Create a Coordinating Node

This document describes how to create a new Coordinating Node and configure it to replicate with other existing Coordinating Nodes.

Quick steps:

  1. Create the Coordinating Node virtual machine
  2. Install the Coordinating Node software stack
  3. Configure the software stack

Create the Virtual Machine

Coordinating Nodes are currently being built on Ubuntu 10.04 LTS.

Details for creating KVM or OpenVZ virtual machines are detailed in the document Creating Virtual Machines. Coordinating Nodes should be provisioned with significant resources.

Type RAM Cores SWAP Disk space
Development CN 4096 MB 2 1024 100000 MB
Production CN 32768 MB 8 2048 300000 MB (or more)

After creating the new virtual machine, enable the partner repository by uncommenting or adding the following lines in /etc/apt/sources.list.d/dataone.list:

deb http://archive.canonical.com/ubuntu precise partner
deb-src http://archive.canonical.com/ubuntu precise partner

The following additional packages (beyond the base install described in create_vm) should then be installed:

You may install a handy software package that allows ease of creating source listing entries for apt-get, though it only works for additions to /etc/apt/sources.list, not contents of /etc/apt/sources.list.d/:

sudo apt-get install python-software-properties

The CN software stack is maintained in an apt repository. To install the stack, add the following entries in /etc/apt/sources.list.d/dataone.list:

deb [arch=amd64] http://jenkins-1.dataone.org/ubuntu-unstable precise universe
or
deb [arch=amd64] http://jenkins-1.dataone.org/ubuntu-stable precise universe

and

sudo add-apt-repository "deb [arch=amd64] http://jenkins-1.dataone.org/ubuntu-extra precise universe"

Ensure the system is up to date before continuing:

sudo apt-get update
sudo apt-get upgrade

Lastly, The following packages will need to be installed in this order:

sudo apt-get --no-install-recommends install openjdk-7-jre-headless
sudo update-java-alternatives -s java-7-openjdk-amd64
sudo apt-get --no-install-recommends install libjaxp1.3-java
sudo apt-get --no-install-recommends install libxerces2-java
sudo apt-get --no-install-recommends install ant

sudo apt-get install ssl-cert

The –no-install-recommends switch prevents installation of the Ubuntu preferred JVM (gcj) and so ensures that openjdk-7-jre-headless remains the only JVM available on the system. Unintended installation of gcj appears to have been the cause of some unanticipated behavior of the CN stack.

Install the Coordinating Node Software

Before proceeding with installation of the CN software stack, it is necessary to install a certificate that will be used for enabling HTTPS communications. If the CN has a host name in the dataone.org domain, then the DataONE wild card certificate may be used. This may be retrieved from any existing Coordinating Node or alternatively, contact an administrator (see Server Administration and Contacts). The wild card certificate key should be copied into /etc/ssl/private/dataone_org.key. If the CN has a host name in the .test.dataone.org domain, then the DataONE Test wild card certificate may be used. The test wild card certificate key should be copied into /etc/ssl/private/_.test.dataone_org.key.

Alternatively, a self-signed CN certificate can be used for an individual developer’s virtual development environment for testing and development. The procedure for creating and installing a self-signed certificate goes something like:

cd
pwd
  /home/waltz
mkdir ssl
cd ssl

openssl genrsa -des3 -out server.key 1024
  Generating RSA private key, 1024 bit long modulus
  ...........................................++++++
  .............++++++
  e is 65537 (0x10001)
  Enter pass phrase for server.key: <type something here and remember it>
  Verifying - Enter pass phrase for server.key: <type something here and remember it>

chmod 600 server.key
openssl req -new -key server.key -out server.csr
  Enter pass phrase for server.key: <type something here that you typed above>
  Country Name (2 letter code) [AU]:US
  State or Province Name (full name) [Some-State]:Tennessee
  Locality Name (eg, city) []:Knoxville
  Organization Name (eg, company) [Internet Widgits Pty Ltd]:DataONE.org
  Organizational Unit Name (eg, section) []:
  Common Name (eg, YOUR name) []:cn-dev.dataone.org
  Email Address []:rwaltz@cn-dev

  Please enter the following 'extra' attributes
  to be sent with your certificate request
  A challenge password []: <pressed enter>
  An optional company name []: <pressed enter>

XXX NOTE that the common name is spurious XXX

cp server.key server.key.orig
openssl rsa -in server.key.orig -out server.key
  Enter pass phrase for server.key.orig: <type something here that you typed above>
  writing RSA key

openssl x509 -req -days 3650 -in server.csr -signkey server.key -out server.crt
  Signature ok
  subject=/C=US/ST=Tennessee/L=Knoxville/O=DataONE.org/CN=cn-dev.utk.edu/emailAddress=rwaltz@cn-dev.utk.edu
  Getting Private key

sudo cp server.key /etc/ssl/private/dataone_org.key

sudo cp server.crt /etc/ssl/certs/dataone.org.crt

Next, install the CN software using the usual steps:

sudo apt-get update
sudo apt-get install dataone-cn-os-core dataone-cn-metacat \
dataone-cn-portal dataone-cn-rest-service dataone-cn-processdaemon \
dataone-cn-index dataone-mercury dataone-cn-solr dataone-cn-version-tool

A number of additional packages will be installed as dependencies of the CN software stack. The current (2015-07-21) list of additional packages is provided below:

apache2 apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common authbind
check-mk-agent-logwatch cryptsetup-bin dataone-cn-index dataone-cn-metacat dataone-cn-os-core
dataone-cn-portal dataone-cn-processdaemon dataone-cn-rest-service dataone-cn-solr
dataone-cn-version-tool dataone-mercury dbus-x11 dconf-gsettings-backend dconf-service
debconf-utils fontconfig gawk gconf-service gconf-service-backend gconf2 gconf2-common gvfs
gvfs-common gvfs-daemons gvfs-libs hicolor-icon-theme jsvc ldap-utils libantlr-java
libapache-pom-java libapache2-mod-jk libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap
libasound2 libatasmart4 libatk-wrapper-java libatk-wrapper-java-jni libatk1.0-0 libatk1.0-data
libavahi-glib1 libbackport-util-concurrent-java libbonobo2-0 libbonobo2-common libcairo-gobject2
libcairo2 libcanberra0 libcommons-beanutils-java libcommons-codec-java
libcommons-collections-java libcommons-collections3-java libcommons-csv-java
libcommons-daemon-java libcommons-dbcp-java libcommons-digester-java libcommons-fileupload-java
libcommons-httpclient-java libcommons-io-java libcommons-lang-java libcommons-logging-java
libcommons-parent-java libcommons-pool-java libcommons-validator-java libconvert-asn1-perl
libcryptsetup4 libdatrie1 libdconf0 libdom4j-java libecj-java libencode-locale-perl
libexcalibur-logkit-java libfile-listing-perl libfont-afm-perl libfontenc1 libgconf-2-4
libgconf2-4 libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libgdu0 libgif4 libgl1-mesa-dri
libgl1-mesa-glx libglapi-mesa libgnome-keyring-common libgnome-keyring0 libgnome2-0 libgnome2-bin
libgnome2-common libgnomevfs2-0 libgnomevfs2-common libgtk-3-0 libgtk-3-bin libgtk-3-common
libgtk2.0-0 libgtk2.0-bin libgtk2.0-common libgudev-1.0-0 libhtml-form-perl libhtml-format-perl
libhtml-parser-perl libhtml-tagset-perl libhtml-tree-perl libhttp-cookies-perl
libhttp-daemon-perl libhttp-date-perl libhttp-message-perl libhttp-negotiate-perl libice-dev
libidl-common libidl0 libio-socket-inet6-perl libio-socket-ssl-perl libjasper1 libjaxen-java
libjaxme-java libjdom1-java libjetty-java libllvm3.0 liblog4j1.2-java libltdl7 liblvm2app2.2
liblwp-mediatypes-perl liblwp-protocol-https-perl libmailtools-perl libnet-http-perl
libnet-ldap-perl libnet-ssleay-perl libnoggit-java libodbc1 liborbit2 liboro-java libpango1.0-0
libperl5.14 libpixman-1-0 libpolkit-agent-1-0 libpolkit-backend-1-0 libportlet-api-2.0-spec-java
libpq5 libpthread-stubs0 libpthread-stubs0-dev libservlet2.5-java libservlet3.0-java
libsgutils2-2 libsigsegv2 libslf4j-java libslp1 libsm-dev libsocket6-perl libtdb1 libthai-data
libthai0 libtiff4 libtomcat7-java liburi-perl libvelocity-tools-java libvorbisfile3
libwerken.xpath-java libwww-perl libwww-robotrules-perl libx11-dev libx11-doc libx11-xcb1
libxau-dev libxcb-glx0 libxcb-render0 libxcb-shape0 libxcb-shm0 libxcb1-dev libxcomposite1
libxdamage1 libxdmcp-dev libxft2 libxinerama1 libxom-java libxpp2-java libxpp3-java libxslt1.1
libxt-dev libxtst6 libxv1 libxxf86dga1 mtools openjdk-7-jdk openjdk-7-jre policykit-1
policykit-1-gnome postgresql postgresql-9.1 postgresql-client-9.1 postgresql-client-common
postgresql-common shared-mime-info slapd solr-common7 solr-tomcat7 sound-theme-freedesktop
tomcat7 tomcat7-admin tomcat7-common ttf-dejavu-extra udisks velocity x11-utils x11proto-core-dev
x11proto-input-dev x11proto-kb-dev xmlstarlet xorg-sgml-doctools xtrans-dev

Configuring a Coordinating Node

Temporary fix chown -R openldap.openldap /var/lib/ldap

Creating a Metacat administrative user in LDAP (production only)

After installing the debian packages, but prior to configuring Metacat, the Metacat administrative DN must be added to LDAP for the production environment. The uid=dataone_cn_metacat,o=DATAONE,dc=ecoinformatics,dc=org DN is used in the testing environments. To add the new DN:

  1. Use slappasswd to create an SHA encrypted version of the password:

$ slappasswd -h {SHA} -s <enter-password-here>

2) Create an ldif file named dataone_cn_metacat.ldif with the password appended to the end, similar to:

dn: cn=dataone_cn_metacat,dc=dataone,dc=org objectClass: top objectClass: person objectClass: organizationalPerson objectClass: inetOrgPerson cn: dataone_cn_metacat givenName: DataONE mail: jones@nceas.ucsb.edu sn: Metacat userPassword: {SHA}vx7+jWYTWQAjzRvh0owepFFGdi8=

but replace the value of the userPassword with the output of the command above.

  1. Add the entry to one of the CNs (say, cn-ucsb-1.dataone.org) with:

$ ldapadd -x -D cn=admin,dc=dataone,dc=org -W -f dataone_cn_metacat.ldif

Configuring Metacat authentication (production only)

For the production environment, the Metacat administrative user created above needs to be configured in the Metacat

With apache and tomcat running, go to:

https://<hostname>.dataone.org/knb/admin

The Authentication configuration page should come up. change the follwing fields with the list values:

Authentication Class

**** Add CN contactSubject into dataone (robert’s DN or other) ****

Configuring Solr Cloud

Please see Solr4Cloud for details on solr cloud configuration.

Change SSH server to Certificate only

All servers will have their remote ssh login by password disabled. The only way to login with a password is via console.

You should have a copy of your public key available for installation in your .ssh directory.

The following is copied and edited from _Generate a ssh key and disable password authentication on Ubuntu Server 12.04

Step by Step guide

Obviously, substitute [your username here] with your username, rsmith for instance.
  1. Generate the ssh key pair on your client computer:

    > ssh-keygen > -t rsa > -b 4096 > -C “[your username here] d1 key pair” > -f ~/.ssh/[your username here]_dataone_rsa

    When prompted, create a strong (five+ random word) passphrase. Back this key file up securely in case your drive dies, etc.

  2. Transfer your public key onto each (sigh) dataone VM you want to access

  1. Test your setup from your laptop/workstation
> ssh cn-dev-ucsb-1.test.dataone.org date
  1. Copy the public key to the server
> scp ~/.ssh/[your username here]_dataone_rsa.pub [your username here]@cn-dev-ucsb-1.test.dataone.org:[your username here]_dataone_rsa.pub
  1. Connect to the server:

    > ssh [your username here]@cn-dev-ucsb-1.test.dataone.org

  2. Append the public key to authorized_keys and remove the uploaded copy:

    > cat [your username here]_dataone_rsa.pub >> ~/.ssh/authorized_keys > rm [your username here]_dataone_rsa.pub > chmod 600 .ssh/authorized_keys > chmod 700 .ssh

  3. Edit the ssh server configuration to make sure that public key authentication is enabled (it should be enabled by default):

    > sudo nano /etc/ssh/sshd_config

  1. These entries must be set to yes:
> RSAAuthentication yes > PubkeyAuthentication yes
  1. Reload the configuration:

    > sudo service ssh reload

  2. Disconnect from the server:

    > exit

  3. Try connecting without the need to give the password to the ssh-client:

    > ssh [your username here]@cn-dev-ucsb-1.test.dataone.org

    You might need to give a password now to access your private key file, but you should not need to give the password to the ssh program.

  4. Disable password authentication:

    > sudo nano /etc/ssh/sshd_config

  1. The following settings should be set to no:
ChallengeResponseAuthentication no PasswordAuthentication no UsePAM yes
  1. Reload the configuration:
> sudo service ssh reload
  1. Test that password authentication really is disabled:
  1. Disconnect from the server:
> exit
  1. Try to reconnect to the server with key file authentication disabled:
> ssh [your username here]@cn-dev-ucsb-1.test.dataone.org -o PubkeyAuthentication=no

This should produce a permission denied message: “Permission denied (publickey).”

Updating a Coordinating Node

It will be necessary to update the CN software stack on a regular basis while the CN software stack is under active development. For the most part, this process should be as simple as the normal installed software update process of:

sudo apt-get update
sudo apt-get upgrade

The version information of the cn-buildout packages is automatically updated as they are built by Hudson and so any package updates should be picked up by apt.

When new search fields are added or existing fields modified, it may be necessary to migrate the data in the current search index into a new index with the updated schema. This will be indicated by the exit message of the dataone-cn-index component. Until the migration has completed the previous search index should remain available for users. The migration script is located at:

/usr/share/dataone-cn-index/scripts/migrate-search-index.sh

and requires sudo permission to run.

It may be necessary on occasion to perform a more extensive update that can not be effectively handled through the automated process. Notification of such situations should be made through the developers@dataone.org mailing list.

This material is based upon work supported by the National Science Foundation under Grant Numbers 083094 and 1430508.

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.