<article id="API">
	<div class="container">

        <!--
    	<div class="span2 list-group">
      		<a href="#api" class="list-group-item active">DataONE REST API</a>
      		<a href="#apijava" class="list-group-item">Java Client Library</a>
      		<a href="#apipython" class="list-group-item">Python Client Library</a>
      		<a href="#rdataone" class="list-group-item">R Package</a>
    	</div>

    	<div class="span10">
-->
			<div class="row-fluid">
				<div class="text-left">
					<h2>
						<i class="glyphicons-icon cloud"></i>
						KNB Developer Tools
					</h2>
					<p class="lead">The KNB supports the DataONE REST API, and Java, Python, and R libraries for easily creating client tools.  
					</p>	
				</div>
			</div>

				<div class="row-fluid">
					<article class="text-left">
						<h3 id="api">
							<i class="glyphicons-icon cloud"></i> DataONE REST API
						</h3>
						<p class="lead">A REST API for accessing and contributing data.</p>
						<p>
							The KNB supports the DataONE REST API for automating the process of
                            uploading, downloading, and searching for data on the KNB using scripted
                            languages such as shell, R, Matlab, and Python, among others.  This guide
                            is a brief synopsis of the DataONE API, which is more comprehensively 
                            documented in the <a
							href="https://purl.dataone.org/architecture/apis/MN_APIs.html"
							target="_external">DataONE Architecture Documentation</a> (also see the <a
							href="https://purl.dataone.org/architecture-dev/apis/MN_APIs.html"
							target="_external">development version of the architecture guide for future releases</a>).
							This API allows any software
							tool that supports the DataONE API (such as the <a href="https://releases.dataone.org/online/dataone_r/">rDataONE R
							package</a>) to also be able to seamlessly interact with KNB data.
							While DataONE maintains the full technical documentation on the
							API, here is a brief overview for commonly accessed services on the KNB.
						</p>
						<section name="summary">
							<h3>Summary</h3>
							<p>DataONE distinguishes three classes of objects that it
								will store and manage: data objects, science metadata objects,
								and resource map documents. Each of these are uniquely
								identifiable by their persistent identifier (PID), and each has
								associated SystemMetadata which describes the object type, size,
								access rules, etc.</p>
							<ul>
                                <li>
                                	<strong>Data objects</strong>
                                	<p>
	                                	are treated as opaque blobs, and are retrievable
		                                via the <code>get</code> method given a persistent identifier (PID). Data objects
		                                can be represented in any format, but the repository encourages the use of 
		                                non-proprietary, open formats such as CSV and netCDF.
	                                </p>
                            	</li>
                            	<li>    
	                                <strong>Science metadata objects</strong> 
	                                <p>
		                                are metadata documents such as 
		                                <a href="http://knb.ecoinformatics.org/software/eml" target="blank">EML</a>, 
		                                FGDC, ISO19115, and so forth that provide metadata describing some data
										object(s). These are represented in XML according to their
										respective schema. 
									</p>
                            	</li>
	                            <li>    
	                                <strong>Resource Map objects</strong>
		                            <p> are OAI-ORE documents that
										describe the aggregations of data and metadata into
										<code>data packages</code>. Individual data and metadata files can be uploaded
		                                to the repository, but to indicate that a set of files is part of an 
		                                aggregated data package, you must provide a OAI-ORE resource map linking the
		                                objects.
	                                </p>
	                                <p>
										In addition to aggregation, Resource Maps can describe the origin of objects by asserting
										<a href="https://www.dataone.org/webinars/provenance-and-dataone-facilitating-reproducible-science">
										provenance relationships.</a> These relationships will be displayed on the KNB.
                            		</p>
								</li>
							</ul>
							<p>
							<blockquote><i class="glyphicons-icon warning_sign"></i>
							<p class="text-warning">
							All API access is over HTTPS, and accessed via the
							<code>https://knb.ecoinformatics.org/knb/d1/mn/v2/</code>
							endpoint. The relative path prefix
							<code>/v2/</code>
							indicates that we are currently using version 2 of the DataONE
							API.
                            </blockquote>
							<blockquote><i class="glyphicons-icon warning_sign"></i>
							<p class="text-warning">
                            The examples below show calls to the production KNB data
                            repository REST endpoint 
							(<code>https://knb.ecoinformatics.org/knb/d1/mn/v2</code>), but users
						    should not create test data on the production KNB.  Instead, please use
						    a test Metacat server to explore the API and create test data
						    (e.g., <code>https://dev.nceas.ucsb.edu/knb/d1/mn/v2</code>).
						    </p></blockquote>
								
							</p>
							<h4>Quick Reference</h4>
							<table class="endpoints api">
								<tbody>
									<tr>
										<th>URL</th>
										<th>Method</th>
										<th>Example</th>
									</tr>
									<tr>
										<td><code>/object/&lt;pid&gt;</code></td>
										<td>GET</td>
										<td><a href="#api/get">Get an Object</a></td>
									</tr>
									<tr>
										<td><code>/object</code></td>
										<td>POST</td>
										<td><a href="#api/create">Create an Object</a></td>
									</tr>
									<tr>
										<td><code>/object/&lt;pid&gt;</code></td>
										<td>PUT</td>
										<td><a href="#api/update">Update an Object</a></td>
									</tr>
									<tr>
										<td><code>/archive/&lt;pid&gt;</code></td>
										<td>PUT</td>
										<td><a href="#api/archive">Archive an Object</a></td>
									</tr>
									<tr>
										<td><code>/meta/&lt;pid&gt;</code></td>
										<td>GET</td>
										<td><a href="#api/getSystemMetadata">Get System Metadata for an Object</a></td>
									</tr>
									<tr>
										<td><code>/generate</code></td>
										<td>POST</td>
										<td><a href="#api/generateId">Generate an Identifier</a></td>
									</tr>
									<tr>
                                        <td><code>/query/solr/&lt;query&gt;</code></td>
										<td>GET</td>
										<td><a href="#api/query">Search the metadata index</a></td>
									</tr>
									<tr>
										<td><code>/object</code></td>
										<td>GET</td>
										<td><a href="#api/listObjects">List objects</a></td>
									</tr>
									<tr>
										<td><code>/object/&lt;pid&gt;</code></td>
										<td>DELETE</td>
										<td><a href="#api/delete">Delete an Object</a></td>
									</tr>
								</tbody>
							</table>
						</section>
						<section name="Calling Services">
							<h3 id="requestFormat">Request Format</h3>
							<ul>
								<li>GET, HEAD, and DELETE requests only pass parameters as
									part of the URL. The parameter values must be converted to UTF-8
									and appropriately escaped for incorporating into the
									URL.</li>
								<li>Message bodies (e.g. for POST and PUT requests) are
									encoded using MIME Multipart, mixed (RFC2046). All information
									for creating the new object or resource is transmitted in the
									message body, which is encoded as a MIME multipart/mixed
									message. We use two types of content in MIME multipart/mixed
									messages: parameters and files. Parameters are to be used for
									all simple types (such as a String value). Files are to be used
									for all complex types (such as an XML structure) and for octet
									streams.</li>
							</ul>
							<h3 id="responseFormat">Response Format</h3>
							<p>Version 1.0 of the DataONE services only support XML serialization, and 
							this format MUST be used when communicating with the KNB.
							Request and response documents MUST also be encoded using UTF-8.</p>

							<h3 id="auth">Authentication and Authorization</h3>
                            <p>Two mechanisms are supported for authentication:
                            <ul>
                                <li>Authentication Tokens passed in the HTTP "Authorization" header</li>
                                <li>Client-side SSL certificates</li>
                            </ul></p>
                            <p><strong>Using Authentication Tokens</strong>.
                            In this scenario, users sign in to the <a href="#signin"><%=window.themeTitle%></a>, and copy an 
                            <strong>authentication token</strong> from their
                            <a href="#my-profile/s=settings/s=token">profile page</a> which can then be included 
                            in HTTPS requests in the HTTP header "Authorization:".</p>
                            
                            <div>
                            	<a href="img/token-example.gif"><img src="img/token-example.gif" class="img-polaroid figure" /></a>
                            	<p>Users copy an authentication token from their profile page.</p>
                            </div>
                            
                            <p>The Authentication Token is a long base64-encoded string of characters that encodes the user's credentials
                            in a signed and validatable <a href="https://jwt.io" target="_external">JWT token</a>. Each language or 
                            tool will have is own mechanism for setting HTTP headers.  For example, for curl an authenticated request 
                            can be made using the '-H' command line option, such as:<code class="code_example">
$ export TOKEN='eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJodHRwOlwvXC9vcmNp...'
$ curl -H "Authorization: Bearer $TOKEN" https://knb.ecoinformatics.org/knb/d1/mn/v2/object
                            </code>
                            The token expires in 18 hours, and so a new token can be retrieved beforehand.
							<blockquote><i class="icon icon-warning-sign icon-warning"></i>
                                <p class="text-warning">The authentication token should be carefully protected just as you would
                                an account password, as it gives the holder full rights to the account.  Do not save
                                tokens in code, and don't check them into version control systems, or otherwise make them available
                                to other people.
						    </p></blockquote>
                            </p>

                            <p><strong>Using client-side certificates over SSL</strong>.
							Users can 
							<a href="https://cilogon.org/?skin=DataONE" target="_external">log into CILogon</a> 
							to download a client certificate, 
							which can then be included in requests as part of the SSL session with the host.  
                            The Subject of the provided certificate
							will be used by the KNB to determine all access control decisions for accessing,
							creating, updating, archiving, and deleting objects.  If a client-side certificate
							is not provided, the user will be considered an anonymous <code>public</code> user
							and will only be able to access public content.   
							</p>
							<p>Each language or submission tool will have different mechanisms for setting the 
							client certificate in the SSL session.  For example, for Curl the certificate filename 
							is passed in on the command line: <code>curl -X POST --cert /tmp/x509up_u502 ...</code>.
							<blockquote><i class="icon icon-warning-sign icon-warning"></i>
                                <p class="text-warning">The version of <code>curl</code> shipped by Apple on MacOS X 10.9
                                and later is broken and does not support providing PEM certificates via the command line. 
                                Instead, it uses certificates registered in the system keychain, as described on the 
                                <a href="http://curl.haxx.se/mail/archive-2014-10/0053.html">curl mailing list</a>.  
                                Thus, calls to the KNB that require a certificate will fail on the standard Mac curl 
                                version, which can be fixed by replacing this with the MacPorts version
                                of curl, or by using a certificate converted to PK12 format. A workaround for these issues
                                is being explored, as the behavior differs in Mavericks and Yosemite.
						    </p></blockquote>
							</p>
						</section>

						<h3 id="auth">Method Details and Examples</h3>
						<section name="examples">
						
							<section class="example">    
							    <a id="get"></a>
								<h3 name="get">Get an Object</h3>
	                            <p>Each object on DataONE has a persistent identifier (PID), which can be used to get the
	                            bytes of tha object.  Note that PID identifiers must be escaped using URL escaping conventions
	                            if they contain characters that are normally reserved in URLs.  For example, a DOI such as</p>
	                            <code>doi:10.5063/FF1HT2M7Q</code> is a PID which would need to be escaped to 
	                            <code>doi:10.5063%2FF1HT2M7Q</code> when used in a URL.
	
								<code class="code_example">
ENDPOINT="https://knb.ecoinformatics.org/knb/d1/mn/v2"
curl -X GET \
     -H "Accept: text/xml" \
     "${ENDPOINT}/object/doi:10.5063%2FF1HT2M7Q"
								</code>
	
	                            If a certificate is not provided in the request, then the results will only include publicly
	                            accessible content.  To view private content, be sure to include a valid X.509 certificate in
	                            the request (e.g,, in curl, use the <code>--cert</code> argument to provide the path to a 
	                            certificate that that was previously downloaded from CILogon).
	                        </section>

							<section class="example">
								<a id="create"></a>
								<h3 name="create">Create an Object</h3>
	                            An object can be inserted into the repository using the <code>create</code> API call, which
	                            involves POSTing the object to the object collection.  Required parameters include the
	                            <code>pid</code> to be used for the object, the bytes of the <code>object</code> itself,
	                            and an XML 
	                            <a href="https://purl.dataone.org/architecture/apis/Types.html#Types.SystemMetadata">SystemMetadata</a>
	                            (<code>sysmeta</code>) document describing core metadata 
	                            properties about the object, including who owns it, its format, etc. 
								<code class="code_example">
curl -X POST \
     --cert /tmp/x509up_u501 \
     -H "Charset: utf-8" \
     -H "Content-Type: multipart/mixed; boundary=----------4A2D135C-52CC-017FC-B269-B711ED211576_$" \
     -H "Accept: text/xml" \
     -F pid=urn:uuid:56eafcec-8b0a-11e3-a5e8-00334b2a1a0a \
     -F object=@mydatafile.csv \
     -F sysmeta=@mysystemmetadata.xml \
     "${ENDPOINT}/object"
								</code>
							</section>
							
							<section class="example">
								<a id="update"></a>
								<h3 name="update">Update an Object</h3>
	                            An object can be updated in the repository using the <code>update</code> API call, which
	                            involves PUTing the object to the object collection.  Required parameters include the
	                            <code>newPid</code> to be used for the object, the bytes of the <code>object</code> itself,
	                            and an XML 
	                            <a href="https://purl.dataone.org/architecture/apis/Types.html#Types.SystemMetadata">SystemMetadata</a>
	                            (<code>sysmeta</code>) document describing core metadata 
	                            properties about the object, including who owns it, its format, etc. Note that this operation
	                            occurs against the original object by including its <code>pid</code> in the REST URL.
								<code class="code_example">
curl -X PUT \
     --cert /tmp/x509up_u501 \
     -H "Charset: utf-8" \
     -H "Content-Type: multipart/mixed; boundary=----------4A2D135C-52CC-017FC-B269-B711ED211576_$" \
     -H "Accept: text/xml" \
     -F newPid=urn:uuid:21865616-8b0d-11e3-a31f-00334b2a1a0a \
     -F object=@mydatafile.csv \
     -F sysmeta=@mysystemmetadata.xml \
     "${ENDPOINT}/object/urn:uuid:56eafcec-8b0a-11e3-a5e8-00334b2a1a0a"
								</code>
							</section>
							
							<section class="example">
								<a id="archive"></a>
								<h3 name="archive">Archive an Object</h3>
	                            An object can be archived, which moves it out of the search path so it won't be discovered, but is
	                            still accessible to users who know the <code>pid</code> of the object so that citations remain 
	                            viable. To archive an object, call the <code>archive</code> service using an HTTP PUT with the 
	                            <code>pid</code> in the service endpoint.
								<code class="code_example">
curl -X PUT \
     --cert /tmp/x509up_u501 \
     -H "Accept: text/xml" \
     "${ENDPOINT}/archive/urn:uuid:56eafcec-8b0a-11e3-a5e8-00334b2a1a0a"
								</code>
							</section>
							
							<section class="example">
								<a id="getSystemMetadata"></a>
								<h3 name="getSystemMetadata">Get System Metadata for an Object</h3>
	                            Use the <code>getSystemMetadata</code> to access the <a href="https://purl.dataone.org/architecture/apis/Types.html#Types.SystemMetadata">SystemMetadata</a> for an object, which represents critical 
	                            information about each object on the repository, including its identifier, its type, access 
	                            control policies, and replication policies, and other details like size and checksum.
								<code class="code_example">
curl -X GET \
     -H "Accept: text/xml" \
     "${ENDPOINT}/meta/urn:uuid:56eafcec-8b0a-11e3-a5e8-00334b2a1a0a"
								</code>
							</section>
							
							<section class="example">
								<a id="generateId"></a>
								<h3 name="generateId">Generate an identifier</h3>
	                            Creating an object on the repository requires submitting it with a globally unique identifier, 
	                            which can be generated by calling the <code>generateIdentifier</code> service.  This service can
	                            be used to generate identifiers that are UUIDs, DOIs, and that potentially follow other syntax 
	                            conventions. The <code>scheme</code> parameter controls which type of identifier should be 
	                            generated.  Generally, the use of UUIDs is encouraged for fine-grained identification of individual
	                            files within a data package, and the use of DOIs for the identifier for the metadata record for
	                            an overall data package.
								<code class="code_example">
curl -X POST \
     --cert /tmp/x509up_u501 \
     -H "Accept: text/xml" \
     -F scheme=UUID \
     "${ENDPOINT}/generate"
								</code>
							</section>
							
							<section class="example">
								<a id="query"></a>
								<h3 name="query">Search the metadata index</h3>
	                            To search across all of the metadata in the repository, use the <code>query</code> service to 
	                            configure a SOLR query.  The full SOLR syntax is supported, providing the means to create complex 
	                            logical query conditions, and to customize the metadata fields returned. Query results can be 
	                            returned in <code>xml</code> and <code>json</code> formats.  Paging through results is supported
	                            using the <code>rows</code> and <code>start</code> parameters.  To search only the most recent 
	                            version of the metadata, include the <code>-obsoletedBy:*</code> constraint in the SOLR query. And
	                            note that all SOLR queries must be properly URL-escaped and SOLR escaped to be processed correctly
	                            (e.g., spaces in the SOLR query need to be escaped with a '+' or '%20', and colons in a SOLR query
	                            value need to be preceded by a backslash). In addition, to run these commands from curl, shell 
	                            escapes will also need to be added as appropriate (e.g., by quoting strings).
								<code class="code_example">
curl -X GET \
     -H "Accept: text/xml" \
     "${ENDPOINT}/query/solr/q=title:soil+AND+-obsoletedBy:*&amp;fl=identifier,title,origin&amp;rows=30&amp;start=0&amp;wt=xml"
								</code>
	
	                           <p> The searchable SOLR fields that can be used to compose queries are accessible from the query service
	                            as well by accessing the endpoint without any query constraints.</p>
			
								<code class="code_example">
curl -X GET \
     -H "Accept: text/xml" \
     "${ENDPOINT}/query/solr"
								</code>
								<p>Example: To retrieve the download/view counts of a particular object in the KNB, use this Solr query:</p>
								<code class="code_example">
curl -X GET \
     -H "Accept: text/xml" \
     "{ENDPOINT}/query/solr/q=id:{OBJECT_PID}&fl=read_count_i"
								</code>
							</section>
							
							<section class="example">
								<a id="listObjects"></a>
								<h3 name="listObjects">List Objects</h3>
	                            The <code>listObjects</code> service provides a sequential list of objects on a node, and is
	                            minimally filterable.  The <code>query</code> service generally contains more information and is
	                            preferred, but the object list can be useful to see recent activity on the repository.
								<code class="code_example">
curl -X GET -H "Accept: text/xml" "${ENDPOINT}/object?start=0&amp;count=100"
								</code>
							</section>

							<section class="example">
								<a id="delete"></a>
								<h3 name="delete">Delete an Object</h3>
	                            Delete is an administrative service that can not be called by users. Contact an administrator for
	                            appropriate credentials. The <code>delete</code> service is provided to fully remove content from
	                            the repository, particularly when that content violates a law or ethical standard.  When removing
	                            content for scientific reasons, <code>archive</code> is the proper method as it preserves citable
	                            links while still hiding content from search.
								<code class="code_example">
curl -X DELETE \
     --cert /tmp/x509up_u501 \
     -H "Accept: text/xml" \
     "${ENDPOINT}/object/urn:uuid:56eafcec-8b0a-11e3-a5e8-00334b2a1a0a"
								</code>
							</section>
						</section>
					</article>
				</div>

				<div class="row-fluid">
				<div class="text-left">
					<h3 id="apijava">
						<i class="glyphicons-icon cloud"></i>
						DataONE Java Client Library
					</h3>
					<p class="lead">A helper library for calling the REST API using Java.</p>
				</div>
			</div>

			<div class="row-fluid">
				<div class="text-left">
					<h3 id="apipython">
						<i class="glyphicons-icon cloud"></i>
						<a href="http://pythonhosted.org/dataone.libclient/">DataONE Python Client Library</a>
					</h3>
					<p class="lead">A helper library for calling the REST API using Python.</p>
				</div>
			</div>

			<div class="row-fluid">
				<div class="text-left">
					<h3 id="rdataone">
						<i class="glyphicons-icon cloud"></i>
						<a href="https://github.com/DataONEorg/rdataone">DataONE R Package</a>
					</h3>
					<p class="lead">An R package providing classes and methods for calling the API within R.</p>
				</div>
			</div>

			<div class="row-fluid">
				<div class="text-left">
					<h3 id="rdataone">
						<i class="glyphicons-icon cloud"></i>
						<a href="https://github.com/DataONEorg/matlab-dataone">DataONE MATLAB library</a>
					</h3>
					<p class="lead">A MATLAB package providing classes and methods for calling the API within Matlab.</p>
				</div>
			</div>

            <!--		</div> -->
    </div>
</article>
