Components of the DataONE Infrastructure
========================================

The following diagrams indicate code dependencies between various components
that make up the DataONE infrastructure. Functional dependencies are not
depicted.

Common Components
-----------------

**Figure 1.** Core shared components.

.. graphviz::

   digraph core_components {

      fontname = "Bitstream Vera Sans";
      fontsize = 8;

      edge [
        fontname = "Bitstream Vera Sans"
        fontsize = 8
        color = "#888888"
        arrowhead = "open"
        arrowsize = 0.5
        style="solid"
        ];

      node [
        fontname = "Courier"
        fontsize = 8
        fontcolor = "black"];   

      color="#888888";

      schema [label="dataonetypes.xsd" URL="#dataonetypes"];
      d1_common_java;
      d1_common_python;
      d1_libclient_java;
      d1_libclient_python;
      d1_architecture;
      d1_jibx_extensions;
      d1_test_resources;
      foresite_java [style="filled" bgcolor="grey80"];
      foresite_python [style="filled" bgcolor="grey80"];

      d1_architecture -> schema;
      d1_common_java -> schema;
      d1_common_java -> d1_jibx_extensions;
      d1_common_java -> d1_test_resources [style="dashed"];
      d1_common_java -> d1_architecture [style="dotted"];
      d1_common_python -> schema;
      d1_common_python  -> d1_architecture [style="dotted"];
      d1_libclient_java -> d1_common_java;
      d1_libclient_java -> d1_test_resources [style="dashed"];
      d1_libclient_java  -> d1_architecture [style="dotted"];
      d1_libclient_java  -> foresite_java;
      d1_libclient_python -> d1_common_python;
      d1_libclient_python -> d1_architecture [style="dotted"];
      d1_libclient_python -> foresite_python;
   
   }

NEEDS VERIFYING

Integration Test Tools
----------------------

**Figure 2.** Test services.

.. graphviz::

   digraph core_components {

      color="#888888";
      fontname = "Bitstream Vera Sans";
      fontsize = 8;

      edge [
        fontname = "Bitstream Vera Sans"
        fontsize = 8
        color = "#888888"
        arrowhead = "open"
        arrowsize = 0.5
        style="solid"
        ];

      node [
        fontname = "Courier"
        fontsize = 8
        fontcolor = "black"];   

      subgraph cluster_CORE {
        label="Shared Components";
        color="#888888";

        schema [label="dataonetypes.xsd"];
        d1_common_java;
        d1_common_python;
        d1_libclient_java;
        d1_libclient_python;
        d1_architecture;
        d1_jibx_extensions;
        d1_test_resources;
        foresite_java [style="filled" bgcolor="grey80"];
        foresite_python [style="filled" bgcolor="grey80"];

        d1_architecture -> schema;
        d1_common_java -> schema;
        d1_common_java -> d1_jibx_extensions;
        d1_common_java -> d1_test_resources [style="dashed"];
        d1_common_java -> d1_architecture [style="dotted"];
        d1_common_python -> schema;
        d1_common_python  -> d1_architecture [style="dotted"];
        d1_libclient_java -> d1_common_java;
        d1_libclient_java -> d1_test_resources [style="dashed"];
        d1_libclient_java  -> d1_architecture [style="dotted"];
        d1_libclient_java  -> foresite_java;
        d1_libclient_python -> d1_common_python;
        d1_libclient_python -> d1_architecture [style="dotted"];
        d1_libclient_python -> foresite_python;
      }
   
      d1_integration;
      d1_web_test_site;
      d1_instance_generator;
      d1_echo_service;
   
      d1_integration -> d1_common_java;
      d1_integration -> d1_libclient_java;
      d1_integration -> Certificates;

      d1_web_test_site -> d1_integration;
   
      d1_instance_generator -> d1_common_python;
      d1_instance_generator -> d1_libclient_python;
  }


Coordinating Node Components
----------------------------

**Figure 3.** Coordinating node components.

.. graphviz::

   digraph core_components {

      color="#888888";
      fontname = "Bitstream Vera Sans";
      fontsize = 10;

      edge [
        fontname = "Bitstream Vera Sans"
        fontsize = 8
        color = "#888888"
        arrowhead = "open"
        arrowsize = 0.5
        style="solid"
        ];

      node [
        fontname = "Courier"
        fontsize = 10
        fontcolor = "black"];   

      subgraph cluster_CORE {
        label="Shared Components";
        color="#888888";

        schema [label="dataonetypes.xsd"];
        d1_common_java;
        d1_common_python;
        d1_libclient_java;
        d1_libclient_python;
        d1_architecture;
        d1_jibx_extensions;
        d1_test_resources;
        foresite_java [style="filled" bgcolor="grey80"];
        foresite_python [style="filled" bgcolor="grey80"];

        d1_architecture -> schema;
        d1_common_java -> schema;
        d1_common_java -> d1_jibx_extensions;
        d1_common_java -> d1_test_resources [style="dashed"];
        d1_common_java -> d1_architecture [style="dotted"];
        d1_common_python -> schema;
        d1_common_python  -> d1_architecture [style="dotted"];
        d1_libclient_java -> d1_common_java;
        d1_libclient_java -> d1_test_resources [style="dashed"];
        d1_libclient_java  -> d1_architecture [style="dotted"];
        d1_libclient_java  -> foresite_java;
        d1_libclient_python -> d1_common_python;
        d1_libclient_python -> d1_architecture [style="dotted"];
        d1_libclient_python -> foresite_python;
      }
   

      postgres [style="filled" fillcolor="grey80"];
      hazelcast [style="filled" bgcolor="grey80"];
      LDAP [style="filled" bgcolor="grey80"];
      SOLR [style="filled" bgcolor="grey80"];
      cilogon_portal_servlet [style="filled" bgcolor="grey80" label="cilogon-portal-servlet"];
      
      d1_cn_version_tool;
      d1_cn_common;
      d1_cn_index_common;
      d1_cn_index_generator;
      d1_cn_noderegistry;
      d1_mercury_common;
      d1_portal;
      d1_cn_approve_node;
      d1_identity_manager;
      d1_log_aggregation;
      d1_mercury_ui;
      d1_portal_servlet;
      d1_cn_rest;
      d1_cn_rest_proxy;
      d1_cn_service;
      d1_solr_extensions;
      d1_synchronization;
      d1_cn_index_processor;
      d1_replication;
      d1_cn_index_tool;
      d1_process_daemon;
      cn_metacat;

      d1_cn_common -> d1_common_java;
      d1_cn_common -> hazelcast;
      
      d1_cn_index_common -> d1_common_java;
      d1_cn_index_common -> postgres;
      
      d1_cn_index_generator -> d1_cn_index_common;
      d1_cn_index_generator -> d1_common_java;
      d1_cn_index_generator -> d1_cn_common;
      d1_cn_index_generator -> postgres;
      d1_cn_index_generator -> hazelcast;

      d1_cn_noderegistry -> d1_cn_common;
      d1_cn_noderegistry -> LDAP;
      d1_cn_noderegistry -> hazelcast;

      d1_mercury_common -> d1_common_java;
      d1_mercury_common -> d1_libclient_java;
      d1_mercury_common -> d1_portal;

      d1_portal -> d1_common_java;
      d1_portal -> d1_libclient_java;

      d1_cn_approve_node -> d1_cn_noderegistry;
      d1_cn_approve_node -> d1_libclient_java;

      d1_identity_manager -> d1_libclient_java;
      d1_identity_manager -> d1_cn_noderegistry;

      d1_log_aggregation -> d1_libclient_java;
      d1_log_aggregation -> d1_cn_noderegistry;

      d1_mercury_ui -> d1_common_java;
      d1_mercury_ui -> d1_libclient_java;
      d1_mercury_ui -> d1_portal;

      d1_portal_servlet -> cilogon_portal_servlet;
      d1_portal_servlet -> d1_portal;
      d1_portal_servlet -> hazelcast;

      d1_cn_rest_proxy -> d1_cn_common;

      d1_cn_rest -> d1_libclient_java;
      d1_cn_rest -> d1_identity_manager;
      d1_cn_rest -> d1_portal;
      d1_cn_rest -> d1_cn_rest_proxy;
      d1_cn_rest -> d1_cn_noderegistry;
      d1_cn_rest -> d1_cn_common;

      d1_cn_service -> d1_cn_rest;
      d1_cn_service -> d1_cn_rest_proxy;

      d1_solr_extensions -> d1_libclient_java;
      d1_solr_extensions -> d1_cn_common;
      d1_solr_extensions -> d1_identity_manager;
      d1_solr_extensions -> d1_cn_noderegistry;
      d1_solr_extensions -> d1_portal;

      d1_synchronization -> d1_libclient_java;
      d1_synchronization -> d1_cn_noderegistry;
      d1_synchronization -> d1_identity_manager;
      d1_synchronization -> hazelcast;

      d1_cn_index_processor -> d1_cn_common;
      d1_cn_index_processor -> d1_libclient_java;
      d1_cn_index_processor -> d1_cn_index_common;
      d1_cn_index_processor -> d1_cn_index_generator;
      d1_cn_index_processor -> postgres;
      d1_cn_index_processor -> SOLR;

      d1_replication -> d1_cn_noderegistry;
      d1_replication -> d1_cn_common;
      d1_replication -> d1_libclient_java;
      d1_replication -> hazelcast;

      d1_cn_index_tool -> d1_common_java;
      d1_cn_index_tool -> d1_libclient_java;
      d1_cn_index_tool -> d1_cn_common;
      d1_cn_index_tool -> d1_cn_index_common;
      d1_cn_index_tool -> d1_cn_index_generator;
      d1_cn_index_tool -> d1_cn_index_processor;

      d1_process_daemon -> d1_synchronization;
      d1_process_daemon -> d1_log_aggregation;
      d1_process_daemon -> d1_replication;
      d1_process_daemon -> hazelcast;

      cn_metacat -> d1_common_java;
      cn_metacat -> d1_libclient_java;
      cn_metacat -> postgres;
      cn_metacat -> hazelcast;
      cn_metacat -> foresite_java;

   }


Member Node Components and Instances
------------------------------------

**Figure 4.** Member node implementations (ovals) and instances (rectangles).

NEEDS UPDATING

.. graphviz::

   digraph core_components {

      color="#888888";
      fontname = "Bitstream Vera Sans";
      fontsize = 8;

      edge [
        fontname = "Bitstream Vera Sans"
        fontsize = 8
        color = "#888888"
        arrowhead = "open"
        arrowsize = 0.5
        style="solid"
        ];

      node [
        fontname = "Courier"
        fontsize = 8
        fontcolor = "black"];   

      subgraph cluster_CORE {
        label="Shared Components";
        color="#888888";

        schema [label="dataonetypes.xsd"];
        d1_common_java;
        d1_common_python;
        d1_libclient_java;
        d1_libclient_python;
        d1_architecture;
        foresite_java [style="filled" bgcolor="grey80"];
        foresite_python [style="filled" bgcolor="grey80"];

        d1_architecture -> schema;
        d1_common_java -> schema;
        d1_common_java -> d1_jibx_extensions;
        d1_common_java -> d1_test_resources [style="dashed"];
        d1_common_java -> d1_architecture [style="dotted"];
        d1_common_python -> schema;
        d1_common_python  -> d1_architecture [style="dotted"];
        d1_libclient_java -> d1_common_java;
        d1_libclient_java -> d1_test_resources [style="dashed"];
        d1_libclient_java  -> d1_architecture [style="dotted"];
        d1_libclient_java  -> foresite_java;
        d1_libclient_python -> d1_common_python;
        d1_libclient_python -> d1_architecture [style="dotted"];
        d1_libclient_python -> foresite_python;
      }
   
      Metacat;
      Mercury_MN;

      KNB [shape="record"];
      ORNL_DAAC [shape="record"];
      USGS [shape="record"];
      Dryad [shape="record"];
      Merritt [shape="record"];
      SAN_Parks [shape="record"];
      MN_Replication_UNM_1 [shape="record"];
      MN_Replication_UNM_2 [shape="record"];
      MN_Replication_UCSB_1 [shape="record"];
      MN_Replication_UCSB_2 [shape="record"];
      MN_Replication_ORC_1 [shape="record"];
      MN_Replication_ORC_2 [shape="record"];
      
      Metacat -> d1_common_java;
      Metacat -> d1_libclient_java;
      KNB -> Metacat;
      SAN_Parks -> Metacat;
      Merritt -> Metacat;

      Mercury_MN -> d1_common_java;
      ORNL_DAAC -> Mercury_MN;
      USGS -> Mercury_MN;

      GMN -> d1_common_python;
      GMN -> d1_libclient_python;
      MN_Replication_UNM_1 -> GMN;
      MN_Replication_UCSB_1 -> GMN;
      MN_Replication_ORC_1 -> GMN;

      MN_Replication_UNM_2 -> Metacat;
      MN_Replication_UCSB_2 -> Metacat;
      MN_Replication_ORC_2 -> Metacat;
      
      Dryad -> d1_common_java;
      Dryad -> d1_libclient_java;
  }



Investigator Toolkit Components
-------------------------------

**Figure 5.** Investigator toolkit.

NEEDS UPDATING

.. graphviz::

   digraph core_components {

      color="#888888";
      fontname = "Bitstream Vera Sans";
      fontsize = 8;

      edge [
        fontname = "Bitstream Vera Sans"
        fontsize = 8
        color = "#888888"
        arrowhead = "open"
        arrowsize = 0.5
        style="solid"
        ];

      node [
        fontname = "Courier"
        fontsize = 8
        fontcolor = "black"];   

      subgraph cluster_CORE {
        label="Shared Components";
        color="#888888";

        schema [label="dataonetypes.xsd"];
        d1_common_java;
        d1_common_python;
        d1_libclient_java;
        d1_libclient_python;
        d1_architecture;
        foresite_java [style="filled" bgcolor="grey80"];
        foresite_python [style="filled" bgcolor="grey80"];

        d1_architecture -> schema;
        d1_common_java -> schema;
        d1_common_java -> d1_jibx_extensions;
        d1_common_java -> d1_test_resources [style="dashed"];
        d1_common_java -> d1_architecture [style="dotted"];
        d1_common_python -> schema;
        d1_common_python  -> d1_architecture [style="dotted"];
        d1_libclient_java -> d1_common_java;
        d1_libclient_java -> d1_test_resources [style="dashed"];
        d1_libclient_java  -> d1_architecture [style="dotted"];
        d1_libclient_java  -> foresite_java;
        d1_libclient_python -> d1_common_python;
        d1_libclient_python -> d1_architecture [style="dotted"];
        d1_libclient_python -> foresite_python;
      }

      d1_client_cli;
      d1_client_r;
      d1_client_fuse;
      d1_client_dokan;
      
      d1_client_cli -> d1_common_python;
      d1_client_cli -> d1_libclient_python;
      d1_client_r -> d1_common_java;
      d1_client_r -> d1_libclient_java;
      d1_client_fuse -> d1_common_python;
      d1_client_fuse -> d1_libclient_python;
      d1_client_dokan -> d1_client_fuse;
  }



Table of Components
-------------------


.. sqltable:: Components List Table
   :header: Component,Category,Responsible,Description
   :widths: 3 2 2 10
   :driver: xlsx
   :source: source/implementation/data/components.xlsx
   :sql: SELECT Component, Category, Responsible, Description FROM Components;

    

    
.. _d1_architecture: https://redmine.dataone.org/projects/d1/issues?query_id=8
.. _d1_authentication: https://redmine.dataone.org/projects/d1/issues?query_id=9
.. _d1_authorization: https://redmine.dataone.org/projects/d1/issues?query_id=11
.. _d1_schemas: https://redmine.dataone.org/projects/d1/issues?query_id=12
.. _d1_common_java: https://redmine.dataone.org/projects/d1/issues?query_id=13
.. _d1_common_python: https://redmine.dataone.org/projects/d1/issues?query_id=14
.. _metacat: https://redmine.dataone.org/projects/d1/issues?query_id=15
.. _d1_cn_service: https://redmine.dataone.org/projects/d1/issues?query_id=16
.. _d1_indexer: https://redmine.dataone.org/projects/d1/issues?query_id=17
.. _d1_synchronization: https://redmine.dataone.org/projects/d1/issues?query_id=18
.. _d1_mn_generic: https://redmine.dataone.org/projects/d1/issues?query_id=19
.. _d1_mn_daac: https://redmine.dataone.org/projects/d1/issues?query_id=20
.. _d1_mn_dryad: https://redmine.dataone.org/projects/d1/issues?query_id=21
.. _d1_mn_fedora: https://redmine.dataone.org/projects/d1/issues?query_id=22
.. _d1_libclient_java: https://redmine.dataone.org/projects/d1/issues?query_id=23
.. _d1_libclient_python: https://redmine.dataone.org/projects/d1/issues?query_id=24
.. _d1_client_cli: https://redmine.dataone.org/projects/d1/issues?query_id=25
.. _d1_client_fuse: https://redmine.dataone.org/projects/d1/issues?query_id=26
.. _d1_client_r: https://redmine.dataone.org/projects/d1/issues?query_id=27
.. _d1_monitor: https://redmine.dataone.org/projects/d1/issues?query_id=28
.. _debian_packaging: https://redmine.dataone.org/projects/d1/issues?query_id=29

.. _Metacat application: http://knb.ecoinformatics.org/software/
.. _Mercury: http://daac.ornl.gov/mercury.shtml
.. _SOLR: http://lucene.apache.org/solr/
.. _Lucene: http://lucene.apache.org/
.. _Django: http://www.djangoproject.com/
.. _FUSE: http://fuse.sourceforge.net/
.. _R: http://www.r-project.org/
.. _Cacti: http://www.cacti.net/
.. _Debian: http://www.debian.org/
.. _Ubuntu: http://www.ubuntu.com/
.. _issue tracking system: http://redmine.dataone.org/
.. _Dokan: http://dokan-dev.net/en/