Connectivity and Certificates ============================= This document provides an overview of the network ports and certificates used by services operating on DataONE Coordinating Nodes. .. contents:: Contents :local: :backlinks: entry Services and Ports ------------------ In summary: ================ ================ ============= ==================== Service Ports Secured By Using ================ ================ ============= ==================== LDAP 389 TLS :ref:`ENV_FQDN` PostgreSQL 5432 SSL :ref:`WILDCARD` (copy or link) Hazelcast 5701, 5702, 5703 Firewall HTTP 80 No HTTPS 443 TLS :ref:`WILDCARD` Solr (v2) 8983 Uses apache TBD Zookeeper (v2) 9983 SSL TBD DataONE Services 443 TLS :ref:`ENV_CLIENT` Portal-CILogon 443 TLS :ref:`CILOGON_KEY` ================ ================ ============= ==================== LDAP TLS ~~~~~~~~ :Port: 389 :Firewall: Open to CNs only OpenLDAP is used primarily to synchronize information about Member and Coordinating nodes across the Coordianting Nodes. See the ``dataone-os-core`` buildout under ``/user/share/dataone-cn-os-core/debian/ldap`` for details. The ``slapd`` configuration file is ``/etc/ldap/slapd.conf`` and contains the following references to certificates and keys:: TLSCACertificateFile /etc/ssl/certs/{ENV_CHAIN}.crt TLSCertificateFile /etc/dataone/client/certs/{ENV_FQDN}.pem TLSCertificateKeyFile /etc/dataone/client/private/{ENV_FQDN}.key For eache of the ``syncRepl`` entries:: tls_cacert=/etc/ssl/certs/{ENV_CHAIN}.crt tls_cert=/etc/dataone/client/certs/{ENV_FQDN}.pem tls_key=/etc/dataone/client/private/{ENV_FQDN}.key PostgreSQL ~~~~~~~~~~ :Port: 5432 :Firewall: Open to CNs only PostgreSQL is used by the Metacat and Portal applications. The Portal application relies on Postgres mirroring of session state information to ensure user experience consistency across the Coordinating Nodes. See the `PostgreSQL documentation `_ for more information. :: /var/lib/postgresql/9.1/main/server.crt /var/lib/postgresql/9.1/main/server.key The ``server.crt`` is a copy of the :ref:`WILDCARD`, similarly for the key. .. Note:: Can we just symlink these to the actual certificate files? .. _postgresdoc: http://www.postgresql.org/docs/9.1/static/ssl-tcp.html Hazelcast ~~~~~~~~~ :Port: 5701 Storage cluster, group "DataONE" :Port: 5702 Processing cluster, instance "hzProcessInstance", group "hzProcess" :Port: 5703 Portal cluster, group "DataONE" :Firewall: Open to CNs only The storage cluster runs on port 5701 and is configured in:: /etc/dataone/storage/hazelcast.xml The processing cluster runs on port 5702 and is configured in:: /etc/dataone/process/hazelcast.xml which gleans values for many of the properties from:: /etc/dataone/process/dataoneHazelcast.properties The portal cluster runs on port 5703 and is configured in:: /etc/dataone/portal/hazelcast.xml Apache HTTPS ~~~~~~~~~~~~ :Port: 80, 443 :Firewall: Open to anywhere, additional restrictions by apache config. The Apache HTTP server provides the HTTP(S) service for Coordinating Nodes and services all DataONE client interactions. This service is Internet facing, and provides a layer of isolation between the Internet and applications running under Tomcat or other containers. All DataONE services operate through HTTPS which on the Coordinating Nodes, is configured to use the :ref:`WILDCARD` certificate to establish secure communications. The service is configured primarily through two files:: /etc/apache2/mods-enabled/ssl.conf and:: /etc/apache2/sites-enabled/cn-ssl which contains the following certificate references:: SSLCertificateFile /etc/ssl/certs/{WILDCARD}.crt SSLCertificateKeyFile /etc/ssl/private/{WILDCARD}.key SSLCertificateChainFile /etc/ssl/certs/{WILDCARD_CHAIN}.crt SSLCACertificateFile /etc/ssl/certs/{ENV_CHAIN}.crt The SSLCertificateFile, SSLCertificateKeyFile and SSLCertificateChainFile entries are all used to establish TLS. The SSLCACertificateFile is used to verify the client certificate provided by clients when establishing authenticated connections. Solr and Zookeeper (v2) ~~~~~~~~~~~~~~~~~~~~~~~ .. Note:: These services are for Version 2.x infrastructure :Port: 8983 (perhaps not, likely to be over apache 443) :Port: 9983 :: Certs and locations TBD DataONE Services ~~~~~~~~~~~~~~~~ :Port: 443 :Firewall: Open to anywhere. Additional restrictions by apache config. Client certificate is used by various apps for communicating to CN services. The client cert and key:: /etc/dataone/client/certs/{ENV_FQDN}.pem /etc/dataone/client/private/{ENV_CLIENT}.pem Portal and CILogon ~~~~~~~~~~~~~~~~~~ The portal application retrieves a user's client certificate from CILogon after authentication. A public key is generated using the :ref:`WILDCARD` certificate during the ``dataone-cn-portal`` installation process. See :doc:`maintenance/cilogon_portal_setup`. Certificates ------------ Several channels of communication must be secured on CNs to enable normal operations and interactions between local and remove systems. Different certificates, keys, and signing authorities are used. .. list-table:: :widths: 5 8 4 17 :header-rows: 1 * - Document - Location - Used By - Description * - :ref:`ENV_CHAIN `.crt - /etc/ssl/certs/ - LDAP, DataONE Services - The certificate authority chain for the respective DataONE environment used to sign the :ref:`ENV_FQDN` and the :ref:`ENV_CLIENT`. * - :ref:`ENV_FQDN `.pem - /etc/dataone/client/certs/ - LDAP - A :ref:`certificate signed by the DataONE Certificate Authority of the respective environment `. The subject of the certificate is the fully qualified domain name for the server. * - :ref:`ENV_FQDN `.key - /etc/dataone/client/private/ - LDAP - The key for the :ref:`ENV_FQDN`. * - :ref:`WILDCARD `.crt - /etc/ssl/certs/ - HTTPS, PostgreSQL, Portal-CILogon - The :ref:`wild card certificate ` is a commercially provided certificate with a subject that represents the wild card domain for the respective DataONE environment. For production this will be ``*.dataone.org`` and for test environments, this will be ``*.test.dataone.org``. * - :ref:`WILDCARD `.key - /etc/ssl/private/ - HTTPS, PostgreSQL - The key for the :ref:`WILDCARD`. * - :ref:`WILDCARD_CHAIN `.crt - /etc/ssl/certs/ - HTTPS, PostgreSQL, Portal-CILogon - The certificate authority chain issued by the commercial provider of the :ref:`WILDCARD certificate `. Note that this *may* be different between production and test environments depending on the certificate provider and the timing of acquisition. * - :ref:`ENV_CLIENT `.pem - /etc/dataone/client/private/ - DataONE Services - The :ref:`client certificate ` used to identify the node. The certificate subject is the nodeId, and the certificate is signed by the :ref:`ENV_CHAIN` for the respective environment. * - :ref:`CILOGON_KEY `.publickey - /etc/ssl/certs/ - Portal - A public key used by the Portal application to retrieve the client certificate from the CILogon service once a user has authenticated. * - :ref:`CILOGON_KEY `.pk8 - /etc/ssl/private/ - Portal - The private portion of the public key used by the Portal application to retrieve the client certificate from the CILogon service once a user has authenticated. .. _WILDCARD: Server Wild Card Certificate (WILDCARD) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The wild card certificate is a commercially provided certificate with a subject that represents the wild card domain for the respective DataONE environment. For production this will be ``*.dataone.org`` and for test environments, this will be ``*.test.dataone.org``. **Test Environments** :CA: Commercial provider :Chain: ``/etc/ssl/certs/geotrust_intermediate.crt`` :Certificate: ``/etc/ssl/certs/_.test.dataone.org.crt`` :Key: ``/etc/ssl/private/_.test.dataone.org.key`` **Production Environment** :CA: Commercial provider :Chain: ``/etc/ssl/certs/geotrust_intermediate.crt`` :Certificate: ``/etc/ssl/certs/_.dataone.org.crt`` :Key: ``/etc/ssl/private/_.dataone.org.key`` Checking the server certificate. You can save this script locally as "check_cn_certs" and execute it using the command:: ssh some.cn.address "bash -s" -- < ./check_cn_certs The following script will examine the known locations for certificates on the CN and check their status. .. _ENV_CHAIN: DataONE Certificate Authority (ENV_CHAIN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The certificate authority chain for the respective DataONE environment used to sign the :ref:`ENV_FQDN` and the :ref:`ENV_CLIENT` certificates. **Test Environments** :Chain: ``/etc/ssl/certs/DataONETestCAChain.crt`` **Production Environment** :Chain: ``/etc/ssl/certs/DataONECAChain.crt`` See the `DataONE Certificate Authority project `_ for more details. .. _dataoneca: https://repository.dataone.org/software/tools/trunk/ca/ .. _ENV_FQDN: Server FQDN Certificate (ENV_FQDN) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A certificate signed by the :ref:`ENV_CHAIN` of the respective environment with the certificate subject being the fully qualified domain name for the server. This certificate is used by the LDAP service, which apparently does not accept a wild card certificate. **Example for cn-dev-unm-1.test.dataone.org** :CA: :ref:`ENV_CHAIN` :Chain: ``/etc/ssl/certs/DataONETestCAChain.crt`` :Certificate: ``/etc/dataone/client/certs/cn-dev-unm-1.test.dataone.org.pem`` :Key: ``/etc/dataone/client/private/cn-dev-unm-1.test.dataone.org.key`` To generate the certificate: .. code-block:: bash # mount the encrypted volume open ~/Projects/DataONE/data/Security/DataONE_certs_keys.sparseimage cd ~/Projects/DataONE/Operations/ca # Generate the certificate FQDN="the.targetnode.fqdn" ./ca -c Test ${FQDN} # Package and upload cert package ./publish_cert vieglais DataONETestIntCA/certs/cn-dev-unm-1.test.dataone.org-4.pem # log on to cn-dev-unm-1.test.dataone.org, then CERTNAME="cn-dev-unm-1.test.dataone.org-4" FQDN=$(hostname -f) TSTAMP=$(date +%Y%m%dT%H%M%S) mkdir -p ~/private && chmod 0700 ~/private cd private scp project.dataone.org:/var/www/users/${USER}/${CERTNAME}.zip . unzip ${CERTNAME}.zip sudo mv /etc/dataone/client/certs/${FQDN}.pem /etc/dataone/client/certs/${TSTAMP}_${FQDN}.pem.bak sudo mv /etc/dataone/client/private/${FQDN}.key /etc/dataone/client/private/${TSTAMP}_${FQDN}.key.bak # Note that the .crt is renamed .pem sudo cp ${CERTNAME}/${CERTNAME}.crt /etc/dataone/client/certs/${FQDN}.pem sudo cp ${CERTNAME}/private/${CERTNAME}.key /etc/dataone/client/private/${FQDN}.key **Example for cn-ucsb-1.dataone.org** :CA: :ref:`ENV_CHAIN` :Chain: ``/etc/ssl/certs/DataONECAChain.crt`` :Certificate: ``/etc/dataone/client/certs/cn-ucsb-1.dataone.org.pem`` :Key: ``/etc/dataone/client/private/cn-dev-ucsb-1.dataone.org.key`` .. _ENV_CLIENT: Client NodeID Certificate (ENV_CLIENT) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ **Example for cn-dev-unm-1.test.dataone.org** :CA: DataONE CA :Chain: ``/etc/ssl/certs/DataONETestCAChain.crt`` :Certificate: :: /etc/dataone/client/private/urn:node:cnDevUNM1.pem /etc/dataone/client/private/urn_node_cnDevUNM1.pem :Key: :: /etc/dataone/client/private/urn:node:cnDevUNM1.pem /etc/dataone/client/private/urn_node_cnDevUNM1.pem **Example for cn-ucsb-1.dataone.org** :CA: DataONE CA :Chain: ``/etc/ssl/certs/DataONECAChain.crt`` :Certificate: :: /etc/dataone/client/private/urn:node:CNUCSB1.pem /etc/dataone/client/private/urn_node_CNUCSB1.pem :Key: :: /etc/dataone/client/private/urn:node:CNUCSB1.pem /etc/dataone/client/private/urn_node_CNUCSB1.pem .. _CILOGON_KEY: Public key for CILogon (CILOGON_KEY) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Generated by ``dataone-cn-portal`` buildout postinst script from the Wildcard Server Certificate. :Public Key: ``/etc/ssl/certs/_.test.dataone.org.crt.publickey`` :Key: ``/etc/ssl/private/_.test.dataone.org.key.pk8`` See :doc:`maintenance/cilogon_portal_setup`. Keystores --------- Keystores are secure areas that are well known to applications that need access to secure information. Keystores contain sensitive information and so must be afforded the appropriate protection through file system access permissions and other mechanisms where appropriate. Two types of keystore are used by Coordinating Nodes: the file system and the Java keystore. It is necessary to store some security documents in both, and hence it is also necessary during maintenance procedures to ensure that the entries in both are consistent. File System Keystore ~~~~~~~~~~~~~~~~~~~~ Certificates and keys are stored on the file system in several places on coordinating nodes: .. code-block:: bash /etc/ssl/certs # Default location in Ubuntu /private /etc/dataone/client/certs # DataONE managed keys and certs /private /var/lib/postgresql/9.1/main/ # For postgres Folders where keys are stored MUST be protected be appropriate permissions. For example, the default Ubuntu folder for keys (``/etc/ssl/private``) is: .. code-block:: bash chown root:ssl-cert chmod 640 Java Keystore ~~~~~~~~~~~~~ Java applications use the `Java keystore`_ to store certificates an credentials in a well known location that can be accessed by Java applications running on the Coordinating Nodes. The Java keystore is located at:: /etc/ssl/certs/java/cacerts #Note: following used in dataone-cn-os-core postinst, it is a symlink to above /usr/lib/jvm/java-1.7.0-openjdk-amd64/jre/lib/security/cacerts The Java keystore is modified using keytool_ by the ``dataone-cn-os-core`` postinst script to add the :ref:`ENV_CHAIN` and the :ref:`Server wild card certificate ` chain file. .. _Java KeyStore: http://docs.oracle.com/javase/7/docs/api/java/security/KeyStore.html .. _keytool: http://docs.oracle.com/javase/6/docs/technotes/tools/solaris/keytool.html