<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>12. Replication — Metacat 2.8.4 documentation</title>
<link rel="stylesheet" href="_static/bootstrap.min.css" type="text/css" />
<link rel="stylesheet" href="_static/font-awesome/css/font-awesome.min.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="_static/metacatui.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '2.8.4',
COLLAPSE_MODINDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="top" title="Metacat 2.8.4 documentation" href="index.html" />
<link rel="prev" title="11. Metacat’s Use of Geoserver" href="geoserver.html" />
<link rel="next" title="13. Harvester and Harvest List Editor" href="harvester.html" />
</head>
<body>
<div id="metacatDocs">
<div class="banner">
<a href="index.html"><img class="logo" src="_static/metacat-logo-white.png" /></a>
<a href="index.html"><h1 class="title">Metacat: Metadata and Data Management Server</h1></a>
<img class="logo-right" src="_static/nceas-logo-white.png" />
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right">
<span id="searchbox" style="display: none;">
<form class="search" action="search.html" method="get">
<input type="text" name="q" size="18" />
<input type="submit" value="Go" class="icon-search"/>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</span>
</li>
<script type="text/javascript">$('#searchbox').show(0);</script>
<li class="right">
<a href="genindex.html" title="General Index"
accesskey="I">index</a>
</li>
<li class="right">
<a href="harvester.html" title="13. Harvester and Harvest List Editor"
accesskey="N">next</a>
</li>
<li class="right">
<a href="geoserver.html" title="11. Metacat’s Use of Geoserver"
accesskey="P">previous</a>
</li>
<li class="breadcrumb first"><a href="index.html">Metacat 2.8.4 documentation</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="replication">
<h1>12. Replication<a class="headerlink" href="#replication" title="Permalink to this headline">¶</a></h1>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Note that much of the functionality provided by the replication subsystem in Metacat
has now been generalized and standardized by DataONE, so consider utilizing the
DataONE services for replication as it is a more general and standardized approach
than this Metacat-specific replication system. The Metacat replication system
will be supported for a while longer, but will likely be deprecated in a future
release in favor of using the DataONE replication approach.</p>
</div>
<p>Metacat has a built-in replication feature that allows different Metacat servers
to share data (both XML documents and data files) between each other. Metacat
can replicate not only its home server’s original documents, but also those
that were replicated from partner Metacat servers. When changes are made to
one server in a replication network, the changes are automatically propogated
to the network, even if the network is down.</p>
<p>Replication allows users to manage their data locally and (by replicating them
to a shared Metacat repository) to make those data available to the greater
scientific community via a centralized search. In other words, your Metacat can
be part of a broader network, but you retain control over the local repository
and how it is managed.</p>
<p>For example, the KNB Network (Figure 6.1), which currently consists of ten
different Metacat servers from around the world, uses replication to “join”
the disperate servers to form a single robust and searchable data
repository–facilitating data discovery, while leaving the data ownership and
management with the local administrators.</p>
<div class="figure align-center" id="id1">
<img alt="_images/image059.jpg" src="_images/image059.jpg" />
<p class="caption"><span class="caption-text">A map of the KNB Metacat network.</span></p>
</div>
<p>When properly configured, Metacat’s replication mechanism can be triggered by
several types of events that occur on either the home or partner server: a
document insertion, an update, or an automatic replication (i.e., Delta-T
monitoring), which is set at a user-specified time interval.</p>
<table border="1" class="docutils">
<colgroup>
<col width="28%" />
<col width="73%" />
</colgroup>
<thead valign="bottom">
<tr class="row-odd"><th class="head">Replication Triggers</th>
<th class="head">Description</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>Insert</td>
<td>Whenever a document is inserted into Metacat, the server
notifies each server in its replication list
that it has a new file available.</td>
</tr>
<tr class="row-odd"><td>Update</td>
<td>Whenever a document is updated, the server notifies
each server in its replication list of the update.</td>
</tr>
<tr class="row-even"><td>Delta-T monitoring</td>
<td>At a user-specified time interval, Metacat checks each
of the servers in its replication list
for updated documents.</td>
</tr>
</tbody>
</table>
<div class="section" id="configuring-replication">
<h2>12.1. Configuring Replication<a class="headerlink" href="#configuring-replication" title="Permalink to this headline">¶</a></h2>
<p>To configure replication, you must configure both the home and partner servers:</p>
<ol class="arabic simple">
<li>Create a list of partner servers on your home server using the Replication Control Panel</li>
<li>Create certificate files for the home server</li>
<li>Create certificate files for the partner server</li>
<li>Import partner certificate files to the home server</li>
<li>Import home certificate to the partner server</li>
<li>Update your Metacat database</li>
</ol>
<p>Each step is discussed in more detail in the following sections.</p>
<div class="section" id="using-the-replication-control-panel">
<h3>12.1.1. Using the Replication Control Panel<a class="headerlink" href="#using-the-replication-control-panel" title="Permalink to this headline">¶</a></h3>
<p>To add, remove, or alter servers on your home server’s Replication list, or to
activate and customize the Delta-T handler, use the Replication control panel,
which is accessed via the Metacat Administration interface at the following URL:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">somehost</span><span class="o">.</span><span class="n">somelocation</span><span class="o">.</span><span class="n">edu</span><span class="o">/</span><span class="n">context</span><span class="o">/</span><span class="n">admin</span>
</pre></div>
</div>
<p>“<a class="reference external" href="http://somehost.somelocation.edu/context">http://somehost.somelocation.edu/context</a>” should be replaced with the name
of your Metacat server and context (e.g., <a class="reference external" href="http://knb.ecoinformatics.org/knb/">http://knb.ecoinformatics.org/knb/</a>).
You must be logged in to Metacat as an administrator.</p>
<div class="figure align-center" id="id2">
<img alt="_images/image061.jpg" src="_images/image061.jpg" />
<p class="caption"><span class="caption-text">Replication control panel.</span></p>
</div>
<p>Note that currently, you cannot use the Replication Control Panel to remove a
server after a replication has occurred. To stop replication between two servers,
update the flags that control whether metadata and/or data are replicated.</p>
</div>
<div class="section" id="generating-and-exchanging-security-certificates">
<h3>12.1.2. Generating and Exchanging Security Certificates<a class="headerlink" href="#generating-and-exchanging-security-certificates" title="Permalink to this headline">¶</a></h3>
<p>Before you can take advantage of Metacat’s replication feature, you must
generate security certificates on both the replication partner and home servers.
Depending on how the certificates are generated, the certificates may need to be
exchanged so that each machine “trusts” that the other has replication access.
Certificates that are purchased from a commercial and well-recognized
Certificate Authority do not need to be exchanged with the other replication
partner before replication takes place. Metacat replication relies on SSL with
client certificate authentication enabled. When a replication partner server
communicates with another replication partner, it presents a certificate that
serves to verify and authenticate that the server is trusted.</p>
<p>If you must generate a self-signed certificate, the partner replication server
will need that public certificate (or the certificate of the signing CA) added
to its existing Certificate Authorities.</p>
<div class="section" id="generate-certificates-for-metacat-running-under-apache-tomcat">
<h4>12.1.2.1. Generate Certificates for Metacat running under Apache/Tomcat<a class="headerlink" href="#generate-certificates-for-metacat-running-under-apache-tomcat" title="Permalink to this headline">¶</a></h4>
<p>Note: Instructions are for Ubuntu/Debian systems.</p>
<ol class="arabic">
<li><p class="first">Generate a private key using openssl. The key will be named
<code class="docutils literal"><span class="pre"><hostname>-apache.key</span></code>, where <code class="docutils literal"><span class="pre"><hostname></span></code> is the name of your Metacat
server. Example values for the individual key fields are included in the
table below.</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">openssl</span> <span class="n">req</span> <span class="o">-</span><span class="n">new</span> <span class="o">-</span><span class="n">out</span> <span class="n">REQ</span><span class="o">.</span><span class="n">pem</span> <span class="o">-</span><span class="n">keyout</span> <span class="o"><</span><span class="n">hostname</span><span class="o">>-</span><span class="n">apache</span><span class="o">.</span><span class="n">key</span>
</pre></div>
</div>
<table border="1" class="docutils">
<colgroup>
<col width="26%" />
<col width="74%" />
</colgroup>
<thead valign="bottom">
<tr class="row-odd"><th class="head">Key Field</th>
<th class="head">Description and Example Value</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>Country Name</td>
<td>Two letter country code (e.g., US)</td>
</tr>
<tr class="row-odd"><td>State or Province Name</td>
<td>The name of your state or province spelled in full (e.g., California)</td>
</tr>
<tr class="row-even"><td>Locality Name</td>
<td>The name of your city (e.g., Santa Barbara)</td>
</tr>
<tr class="row-odd"><td>Organization Name</td>
<td>The company or organization name (e.g., UCSB)</td>
</tr>
<tr class="row-even"><td>Organizational Unit Name</td>
<td>The department or section name (e.g., NCEAS)</td>
</tr>
<tr class="row-odd"><td>Common Name</td>
<td>The host server name without port numbers (e.g., myserver.mydomain.edu)</td>
</tr>
<tr class="row-even"><td>Email Address</td>
<td>Administrator’s contact email (e.g., <a class="reference external" href="mailto:administrator%40mydomain.edu">administrator<span>@</span>mydomain<span>.</span>edu</a>)</td>
</tr>
<tr class="row-odd"><td>A challenge password</td>
<td>–leave this field blank–</td>
</tr>
<tr class="row-even"><td>An optional company name</td>
<td>–leave this field blank–</td>
</tr>
</tbody>
</table>
</li>
<li><p class="first">Create the local certificate file by running the command:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">openssl</span> <span class="n">req</span> <span class="o">-</span><span class="n">x509</span> <span class="o">-</span><span class="n">days</span> <span class="mi">800</span> <span class="o">-</span><span class="ow">in</span> <span class="n">REQ</span><span class="o">.</span><span class="n">pem</span> <span class="o">-</span><span class="n">key</span> <span class="o"><</span><span class="n">hostname</span><span class="o">>-</span><span class="n">apache</span><span class="o">.</span><span class="n">key</span> <span class="o">-</span><span class="n">out</span> <span class="o"><</span><span class="n">hostname</span><span class="o">>-</span><span class="n">apache</span><span class="o">.</span><span class="n">crt</span>
</pre></div>
</div>
<p>Use the same <code class="docutils literal"><span class="pre"><hostname></span></code> you used when you generated the key. A file named
<code class="docutils literal"><span class="pre"><hostname>-apache.crt</span></code> will be created in the directory from which you
ran the openssl command. Note: You can name the certificate file anything
you’d like, but keep in mind that the file will be sent to the partner
machine used for replication. The certificate name should have enough
meaning that someone who sees it on that machine can figure out where it
came from and for what purpose it should be used.</p>
</li>
<li><p class="first">Enter the certificate into Apache’s security configuration. This will
be used to identify your server to a replication partner. You must
register the certificate in the local Apache instance. Note that the
security files may be in a different directory from the one used in the
instructions depending on how you installed Apache. Copy the certificate and
key file using the following commands:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="n">cp</span> <span class="o"><</span><span class="n">hostname</span><span class="o">>-</span><span class="n">apache</span><span class="o">.</span><span class="n">crt</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ssl</span><span class="o">/</span><span class="n">certs</span>
<span class="n">sudo</span> <span class="n">cp</span> <span class="o"><</span><span class="n">hostname</span><span class="o">>-</span><span class="n">apache</span><span class="o">.</span><span class="n">key</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ssl</span><span class="o">/</span><span class="n">private</span>
</pre></div>
</div>
</li>
<li><p class="first">Apache needs to be configured to request a client certificate when the
replication API is utilized. The helper file named “metacat-site-ssl.conf” has default
rules that configure Apache for SSL and client certificate authentication.
Set up these SSL settings by copying the metacat-site-ssl.conf file into the <code class="docutils literal"><span class="pre">sites-available</span></code>
directory, editing pertinent values to match your system and running
<code class="docutils literal"><span class="pre">a2ensite</span></code> to enable the site. (Note: some settings in metacat-site-ssl.conf need to be
changed to match the specifics of your system and Metacat deployment.)</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="n">cp</span> <span class="o"><</span><span class="n">metacat_helper_dir</span><span class="o">>/</span><span class="n">metacat</span><span class="o">-</span><span class="n">site</span><span class="o">-</span><span class="n">ssl</span><span class="o">.</span><span class="n">conf</span> <span class="o"><</span><span class="n">apache_install_dir</span><span class="o">>/</span><span class="n">sites</span><span class="o">-</span><span class="n">available</span>
<span class="n">sudo</span> <span class="n">a2ensite</span> <span class="n">metacat</span><span class="o">-</span><span class="n">site</span><span class="o">-</span><span class="n">ssl</span><span class="o">.</span><span class="n">conf</span>
</pre></div>
</div>
</li>
<li><p class="first">Enable the ssl module:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="n">a2enmod</span> <span class="n">ssl</span>
</pre></div>
</div>
</li>
<li><p class="first">Restart Apache to bring in changes by typing:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">init</span><span class="o">.</span><span class="n">d</span><span class="o">/</span><span class="n">apache2</span> <span class="n">restart</span>
</pre></div>
</div>
</li>
<li><p class="first">If using a self-signed certificate, SCP <code class="docutils literal"><span class="pre"><hostname>-apache.crt</span></code> to the
replication partner machine where it will be added as an additional
Certificate Authority.</p>
</li>
</ol>
<p>If using self-signed certificates, after you have created and SCP’d a
certificate file to each replication partner, and received a certificate file
from each partner in return, both home and partner servers must add the
respective partner certificates as Certificate Authorities.</p>
</div>
<div class="section" id="to-import-a-certificate">
<h4>12.1.2.2. To import a certificate<a class="headerlink" href="#to-import-a-certificate" title="Permalink to this headline">¶</a></h4>
<ol class="arabic">
<li><p class="first">Copy it into the Apache directory</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="n">cp</span> <span class="o"><</span><span class="n">remotehostfilename</span><span class="o">></span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ssl</span><span class="o">/</span><span class="n">certs</span><span class="o">/</span>
</pre></div>
</div>
</li>
<li><p class="first">Rehash the certificates for Apache by running:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">cd</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ssl</span><span class="o">/</span><span class="n">certs</span>
<span class="n">sudo</span> <span class="n">c_rehash</span>
</pre></div>
</div>
<p>where the <code class="docutils literal"><span class="pre"><remotehostfilename></span></code> is the name of the certificate file
created on the remote partner machine and SCP’d to the home machine.</p>
</li>
</ol>
</div>
<div class="section" id="to-import-a-certificate-into-java-keystore-for-self-signed-certificates">
<h4>12.1.2.3. To import a certificate into Java keystore (for self-signed certificates)<a class="headerlink" href="#to-import-a-certificate-into-java-keystore-for-self-signed-certificates" title="Permalink to this headline">¶</a></h4>
<ol class="arabic">
<li><p class="first">Use Java’s keytool to import to the default Java keystore</p>
<div class="highlight-default"><div class="highlight"><pre><span></span>sudo keytool -import -alias <remotehostname_alias> -file <remotehostfilename> -keystore $JAVA_HOME/lib/security/cacerts
</pre></div>
</div>
</li>
<li><p class="first">Restart Tomcat</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">init</span><span class="o">.</span><span class="n">d</span><span class="o">/</span><span class="n">tomcat7</span> <span class="n">restart</span>
</pre></div>
</div>
<p>where the <code class="docutils literal"><span class="pre"><remotehostfilename></span></code> is the name of the certificate file
created on the remote partner machine and SCP’d to the home machine and
<remotehostname_alias> is a short memorable alias for this certificate and
$JAVA_HOME is the same as configured for running Tomcat. NOTE: the cacerts path may be different
depending on your exact Java installation.</p>
</li>
</ol>
</div>
<div class="section" id="update-metacat-properties">
<h4>12.1.2.4. Update Metacat properties<a class="headerlink" href="#update-metacat-properties" title="Permalink to this headline">¶</a></h4>
<p>Metacat needs to be configured with the path to both the server certificate and the private key.
1. Edit metacat.properties, modifying these properties to match your specific deployment.</p>
<blockquote>
<div><div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">replication</span><span class="o">.</span><span class="n">certificate</span><span class="o">.</span><span class="n">file</span><span class="o">=/</span><span class="n">etc</span><span class="o">/</span><span class="n">ssl</span><span class="o">/</span><span class="n">certs</span><span class="o">/<</span><span class="n">hostname</span><span class="o">>-</span><span class="n">apache</span><span class="o">.</span><span class="n">crt</span>
<span class="n">replication</span><span class="o">.</span><span class="n">privatekey</span><span class="o">.</span><span class="n">file</span><span class="o">=/</span><span class="n">etc</span><span class="o">/</span><span class="n">ssl</span><span class="o">/</span><span class="n">private</span><span class="o">/<</span><span class="n">hostname</span><span class="o">>-</span><span class="n">apache</span><span class="o">.</span><span class="n">key</span>
<span class="n">replication</span><span class="o">.</span><span class="n">privatekey</span><span class="o">.</span><span class="n">password</span><span class="o">=<</span><span class="n">password</span><span class="p">,</span> <span class="ow">or</span> <span class="n">blank</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">protected</span><span class="o">></span>
</pre></div>
</div>
</div></blockquote>
</div>
</div>
<div class="section" id="update-your-metacat-database">
<h3>12.1.3. Update your Metacat database<a class="headerlink" href="#update-your-metacat-database" title="Permalink to this headline">¶</a></h3>
<p>The simplest way to update the Metacat database to use replication is to use
the Replication Control Panel. You can also update the database using SQL.
Instructions for both options are included in this section.</p>
<div class="figure align-center" id="id3">
<img alt="_images/image063.jpg" src="_images/image063.jpg" />
<p class="caption"><span class="caption-text">Using the Replication Control Panel to update the Metacat database.</span></p>
</div>
<p>To update your Metacat database to use replication, select the “Add this server”
radio button from the Replication Control Panel, enter the partner server name,
and specify how the replication should occur (whether to replicate xml, data,
or use the local machine as a hub).</p>
<div class="section" id="to-update-the-database-using-sql">
<h4>12.1.3.1. To update the database using SQL<a class="headerlink" href="#to-update-the-database-using-sql" title="Permalink to this headline">¶</a></h4>
<ol class="arabic">
<li><p class="first">Log in to the database</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">psql</span> <span class="o">-</span><span class="n">U</span> <span class="n">metacat</span> <span class="o">-</span><span class="n">W</span> <span class="o">-</span><span class="n">h</span> <span class="n">localhost</span> <span class="n">metacat</span>
</pre></div>
</div>
</li>
<li><p class="first">Select all rows from the replication table</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">select</span> <span class="o">*</span> <span class="kn">from</span> <span class="nn">xml_replication</span><span class="p">;</span>
</pre></div>
</div>
</li>
<li><p class="first">Insert the partner server.</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">INSERT</span> <span class="n">INTO</span> <span class="n">xml_replication</span> <span class="p">(</span><span class="n">server</span><span class="p">,</span><span class="n">last_checked</span><span class="p">,</span><span class="n">replicate</span><span class="p">,</span><span class="n">datareplicate</span><span class="p">,</span><span class="n">hub</span><span class="p">)</span> <span class="n">VALUES</span> <span class="p">(</span><span class="s1">'<partner.server/context>/servlet/replication'</span><span class="p">,</span><span class="n">NULL</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mi">0</span><span class="p">);</span>
</pre></div>
</div>
<p>Where <code class="docutils literal"><span class="pre"><partner.server/context></span></code> is the name of the partner server and
context. The values ‘NULL, 1,1,0’ indicate (respectively) the last time
replication occurred, that XML docs should be replicated to the partner
server, that data files should be replicated to the partner server, and
that the local server should not act as a hub. Set a value of ‘NULL,0,0,0’
if your Metacat is only receiving documents from the partner site and not
replicating to that site.</p>
</li>
<li><p class="first">Exit the database</p>
</li>
<li><p class="first">Restart Apache and Tomcat on both home and partner replication machines</p>
</li>
</ol>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
<div class="footerNav">
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right">
<span id="searchbox" style="display: none;">
<form class="search" action="search.html" method="get">
<input type="text" name="q" size="18" />
<input type="submit" value="Go" class="icon-search"/>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</span>
</li>
<script type="text/javascript">$('#searchbox').show(0);</script>
<li class="right">
<a href="genindex.html" title="General Index"
>index</a>
</li>
<li class="right">
<a href="harvester.html" title="13. Harvester and Harvest List Editor"
>next</a>
</li>
<li class="right">
<a href="geoserver.html" title="11. Metacat’s Use of Geoserver"
>previous</a>
</li>
<li class="breadcrumb first"><a href="index.html">Metacat 2.8.4 documentation</a> »</li>
</ul>
</div>
</div>
<div class="small-print">
© Copyright 2012, Regents of the University of California.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.5.2.
</div>
</div>
</div>
</body>
</html>