Summary

DataONE distinguishes three classes of objects that it will store and manage: data objects, science metadata objects, and resource map documents. Each of these are uniquely identifiable by their persistent identifier (PID), and each has associated SystemMetadata which describes the object type, size, access rules, etc.

• Data objects

  are treated as opaque blobs, and are retrievable via the get method given a persistent identifier (PID). Data objects can be represented in any format, but the repository encourages the use of non-proprietary, open formats such as CSV and netCDF.

• Science metadata objects

  are metadata documents such as EML, FGDC, ISO19115, and so forth that provide metadata describing some data object(s). These are represented in XML according to their respective schema.

• Resource Map objects

  are OAI-ORE documents that describe the aggregations of data and metadata into data packages. Individual data and metadata files can be uploaded to the repository, but to indicate that a set of files is part of an aggregated data package, you must provide a OAI-ORE resource map linking the objects.

  In addition to aggregation, Resource Maps can describe the origin of objects by asserting provenance relationships. These relationships will be displayed on the KNB.

All API access is over HTTPS, and accessed via the https://knb.ecoinformatics.org/knb/d1/mn/v2/ endpoint. The relative path prefix /v2/ indicates that we are currently using version 2 of the DataONE API.

The examples below show calls to the production KNB data repository REST endpoint (https://knb.ecoinformatics.org/knb/d1/mn/v2), but users should not create test data on the production KNB. Instead, please use a test Metacat server to explore the API and create test data (e.g., https://dev.nceas.ucsb.edu/knb/d1/mn/v2).

Quick Reference

Request Format

• GET, HEAD, and DELETE requests only pass parameters as part of the URL. The parameter values must be converted to UTF-8 and appropriately escaped for incorporating into the URL.
• Message bodies (e.g. for POST and PUT requests) are encoded using MIME Multipart, mixed (RFC2046). All information for creating the new object or resource is transmitted in the message body, which is encoded as a MIME multipart/mixed message. We use two types of content in MIME multipart/mixed messages: parameters and files. Parameters are to be used for all simple types (such as a String value). Files are to be used for all complex types (such as an XML structure) and for octet streams.

Response Format

Version 1.0 of the DataONE services only support XML serialization, and this format MUST be used when communicating with the KNB. Request and response documents MUST also be encoded using UTF-8.

Authentication and Authorization

Two mechanisms are supported for authentication:

• Authentication Tokens passed in the HTTP "Authorization" header
• Client-side SSL certificates

Using Authentication Tokens. In this scenario, users sign in to the <%=window.themeTitle%>, and copy an authentication token from their profile page which can then be included in HTTPS requests in the HTTP header "Authorization:".

Users copy an authentication token from their profile page.

The Authentication Token is a long base64-encoded string of characters that encodes the user's credentials in a signed and validatable JWT token. Each language or tool will have is own mechanism for setting HTTP headers. For example, for curl an authenticated request can be made using the '-H' command line option, such as: $ export TOKEN='eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJodHRwOlwvXC9vcmNp...' $ curl -H "Authorization: Bearer $TOKEN" https://knb.ecoinformatics.org/knb/d1/mn/v2/object The token expires in 18 hours, and so a new token can be retrieved beforehand.

The authentication token should be carefully protected just as you would an account password, as it gives the holder full rights to the account. Do not save tokens in code, and don't check them into version control systems, or otherwise make them available to other people.

Using client-side certificates over SSL. Users can log into CILogon to download a client certificate, which can then be included in requests as part of the SSL session with the host. The Subject of the provided certificate will be used by the KNB to determine all access control decisions for accessing, creating, updating, archiving, and deleting objects. If a client-side certificate is not provided, the user will be considered an anonymous public user and will only be able to access public content.

Each language or submission tool will have different mechanisms for setting the client certificate in the SSL session. For example, for Curl the certificate filename is passed in on the command line: curl -X POST --cert /tmp/x509up_u502 ....

The version of curl shipped by Apple on MacOS X 10.9 and later is broken and does not support providing PEM certificates via the command line. Instead, it uses certificates registered in the system keychain, as described on the curl mailing list. Thus, calls to the KNB that require a certificate will fail on the standard Mac curl version, which can be fixed by replacing this with the MacPorts version of curl, or by using a certificate converted to PK12 format. A workaround for these issues is being explored, as the behavior differs in Mavericks and Yosemite.