cdocutils.nodes
document
q)q}q(U nametypesq}q(X wasderivedfromqX usedqX derived data without scriptsqNX# ore model for derived data packagesq NX queriesq
NX indexingq NX# section b of the prov documentationq X derived data using scriptsq
NX examplesqNX
prov ontologyqX generatedqX exampleqNX ontology and modelqNX overviewqNX
sample rdfqNX
wasinformedbyqX wasgeneratedbyquUsubstitution_defsq}qUparse_messagesq]q(cdocutils.nodes
system_message
q)q }q (U rawsourceq U Uparentq cdocutils.nodes
section
q )q!}q"(h U h h )q#}q$(h U h hUsourceq%Xa /var/lib/jenkins/jobs/metacat-unstable/workspace/docs/user/metacat/source/ore-model-expansion.rstq&Utagnameq'Usectionq(U
attributesq)}q*(Udupnamesq+]Uclassesq,]Ubackrefsq-]Uidsq.]q/U#ore-model-for-derived-data-packagesq0aUnamesq1]q2h auUlineq3KUdocumentq4hUchildrenq5]q6(cdocutils.nodes
title
q7)q8}q9(h X# ORE Model for Derived Data Packagesq:h h#h%h&h'Utitleq;h)}q<(h+]h,]h-]h.]h1]uh3Kh4hh5]q=cdocutils.nodes
Text
q>X# ORE Model for Derived Data Packagesq?q@}qA(h h:h h8ubaubcsphinx.addnodes
index
qB)qC}qD(h U h h#h%h&h'UindexqEh)}qF(h.]h-]h+]h,]h1]UentriesqG]qH(UsingleqIX Derived ResourcesUindex-0qJU tqKaUinlineqLuh3K h4hh5]ubcdocutils.nodes
target
qM)qN}qO(h U h h#h%h&h'UtargetqPh)}qQ(h+]h,]h-]h.]qRhJah1]uh3K h4hh5]ubhB)qS}qT(h U h h#h%h&h'hEh)}qU(h.]h-]h+]h,]h1]hG]qV(hIX PROVUindex-1qWU tqXahLuh3K
h4hh5]ubhM)qY}qZ(h U h h#h%h&h'hPh)}q[(h.]h-]h+]h,]h1]Urefidq\hWuh3K h4hh5]ubcdocutils.nodes
definition_list
q])q^}q_(h U h h#h%h&Uexpect_referenced_by_nameq`}h'Udefinition_listqah)}qb(h+]h,]h-]h.]qchWah1]uh3Nh4hUexpect_referenced_by_idqd}qehWhYsh5]qf(cdocutils.nodes
definition_list_item
qg)qh}qi(h X Author
Lauren Walker
h h^h%h&h'Udefinition_list_itemqjh)}qk(h+]h,]h-]h.]h1]uh3K
h5]ql(cdocutils.nodes
term
qm)qn}qo(h X Authorh hhh%h&h'Utermqph)}qq(h+]h,]h-]h.]h1]uh3K
h5]qrh>X Authorqsqt}qu(h X Authorqvh hnubaubcdocutils.nodes
definition
qw)qx}qy(h U h)}qz(h+]h,]h-]h.]h1]uh hhh5]q{cdocutils.nodes
paragraph
q|)q}}q~(h X
Lauren Walkerqh hxh%h&h'U paragraphqh)}q(h+]h,]h-]h.]h1]uh3K
h5]qh>X
Lauren Walkerqq}q(h hh h}ubaubah'U
definitionqubeubhg)q}q(h XO Date
- 20140513 Initial draft of ORE model expansion for derived data packages
h h^h%h&h'hjh)}q(h+]h,]h-]h.]h1]uh3Kh4hh5]q(hm)q}q(h X Dateh hh%h&h'hph)}q(h+]h,]h-]h.]h1]uh3Kh5]qh>X Dateqq}q(h X Dateqh hubaubhw)q}q(h U h)}q(h+]h,]h-]h.]h1]uh hh5]qcdocutils.nodes
bullet_list
q)q}q(h U h)}q(UbulletqX -h.]h-]h+]h,]h1]uh hh5]qcdocutils.nodes
list_item
q)q}q(h XH 20140513 Initial draft of ORE model expansion for derived data packages
h)}q(h+]h,]h-]h.]h1]uh hh5]qh|)q}q(h XG 20140513 Initial draft of ORE model expansion for derived data packagesqh hh%h&h'hh)}q(h+]h,]h-]h.]h1]uh3Kh5]qh>XG 20140513 Initial draft of ORE model expansion for derived data packagesqq}q(h hh hubaubah'U list_itemqubah'U bullet_listqubah'hubeubhg)q}q(h X_ Goal
Accurately describe the relationships of resources that are derived from other resources.
h h^h%h&h'hjh)}q(h+]h,]h-]h.]h1]uh3Kh4hh5]q(hm)q}q(h X Goalh hh%h&h'hph)}q(h+]h,]h-]h.]h1]uh3Kh5]qh>X Goalqq}q(h X Goalqh hubaubhw)q}q(h U h)}q(h+]h,]h-]h.]h1]uh hh5]qh|)q}q(h XY Accurately describe the relationships of resources that are derived from other resources.qh hh%h&h'hh)}q(h+]h,]h-]h.]h1]uh3Kh5]qh>XY Accurately describe the relationships of resources that are derived from other resources.qq}q(h hh hubaubah'hubeubhg)q}q(h X Summary
This proposed expansion of the Metacat ORE model uses the `PROV ontology `_ to describe
the relationships between generated or derived resources and the resources used to generate them.
h h^h%h&h'hjh)}q(h+]h,]h-]h.]h1]uh3Kh4hh5]q(hm)q}q(h X Summaryh hh%h&h'hph)}q(h+]h,]h-]h.]h1]uh3Kh5]qh>X Summaryq̅q}q(h X Summaryqh hubaubhw)q}q(h U h)}q(h+]h,]h-]h.]h1]uh hh5]qh|)q}q(h X This proposed expansion of the Metacat ORE model uses the `PROV ontology `_ to describe
the relationships between generated or derived resources and the resources used to generate them.h hh%h&h'hh)}q(h+]h,]h-]h.]h1]uh3Kh5]q(h>X: This proposed expansion of the Metacat ORE model uses the qq}q(h X: This proposed expansion of the Metacat ORE model uses the h hubcdocutils.nodes
reference
q)q}q(h X6 `PROV ontology `_h)}q(UnameX
PROV ontologyUrefuriqX# http://www.w3.org/TR/prov-overview/qh.]h-]h+]h,]h1]uh hh5]qh>X
PROV ontologyq⅁q}q(h U h hubah'U referencequbhM)q}q(h X& U
referencedqKh hh'hPh)}q(Urefurihh.]qU
prov-ontologyqah-]h+]h,]h1]qhauh5]ubh>Xn to describe
the relationships between generated or derived resources and the resources used to generate them.q텁q}q(h Xn to describe
the relationships between generated or derived resources and the resources used to generate them.h hubeubah'hubeubeubh )q}q(h U h h#h%h&h'h(h)}q(h+]h,]h-]h.]qUoverviewqah1]qhauh3Kh4hh5]q(h7)q}q(h X Overviewqh hh%h&h'h;h)}q(h+]h,]h-]h.]h1]uh3Kh4hh5]qh>X Overviewqq}q(h hh hubaubh|)q}r (h X~ There are two main concepts that are being introduced to Metacat - derived data resources and the activities that create them.r h hh%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh4hh5]r h>X~ There are two main concepts that are being introduced to Metacat - derived data resources and the activities that create them.r r }r (h j h hubaubh|)r }r (h X A derived resource is defined as a entity, usually a data file, that is produced by transforming one entity into another or is the production of an activity that generates a new entity from an existing one.
In Metacat, derived data could be a data table that was composed from analyzing one or more existing data tables.
It could also be a visualization of another resource, such as a graph, chart, or map.r h hh%h&h'hh)}r
(h+]h,]h-]h.]h1]uh3K h4hh5]r h>X A derived resource is defined as a entity, usually a data file, that is produced by transforming one entity into another or is the production of an activity that generates a new entity from an existing one.
In Metacat, derived data could be a data table that was composed from analyzing one or more existing data tables.
It could also be a visualization of another resource, such as a graph, chart, or map.r r
}r (h j h j ubaubh|)r }r (h XU Because derived data is the product of a transformation activity, Metacat's ORE model will need to have the capability to store information on these activities.
These activities could be a script (e.g. an R script) that is used to create a visualization from a data table or to compose a new data table from one or more existing data tables.r h hh%h&h'hh)}r (h+]h,]h-]h.]h1]uh3K!h4hh5]r h>XU Because derived data is the product of a transformation activity, Metacat's ORE model will need to have the capability to store information on these activities.
These activities could be a script (e.g. an R script) that is used to create a visualization from a data table or to compose a new data table from one or more existing data tables.r r }r (h j h j ubaubh|)r }r (h XD These new resources will be documented by metadata, most likely EML.r h hh%h&h'hh)}r (h+]h,]h-]h.]h1]uh3K$h4hh5]r h>XD These new resources will be documented by metadata, most likely EML.r r }r (h j h j ubaubeubh!h )r }r (h U h h#h%h&h'h(h)}r! (h+]h,]h-]h.]r" Uexamplesr# ah1]r$ hauh3KDh4hh5]r% (h7)r& }r' (h X Examplesr( h j h%h&h'h;h)}r) (h+]h,]h-]h.]h1]uh3KDh4hh5]r* h>X Examplesr+ r, }r- (h j( h j& ubaubh )r. }r/ (h U h j h%h&h'h(h)}r0 (h+]h,]h-]h.]r1 Uderived-data-using-scriptsr2 ah1]r3 h
auh3KFh4hh5]r4 (h7)r5 }r6 (h X Derived data using scriptsr7 h j. h%h&h'h;h)}r8 (h+]h,]h-]h.]h1]uh3KFh4hh5]r9 h>X Derived data using scriptsr: r; }r< (h j7 h j5 ubaubh|)r= }r> (h X John Smith is a scientist who has uploaded two data tables and an EML file to a Metacat repository. His package of data is represented below.r? h j. h%h&h'hh)}r@ (h+]h,]h-]h.]h1]uh3KGh4hh5]rA h>X John Smith is a scientist who has uploaded two data tables and an EML file to a Metacat repository. His package of data is represented below.rB rC }rD (h j? h j= ubaubcdocutils.nodes
figure
rE )rF }rG (h U h j. h%h&h'UfigurerH h)}rI (h+]h,]h-]h.]rJ Uid4rK ah1]uh3Nh4hh5]rL (cdocutils.nodes
image
rM )rN }rO (h X .. figure:: images/derived-data-smith-package.png
Figure 3. Smith's original data package using the ORE model Metacat has now
h)}rP (UuriX% images/derived-data-smith-package.pngrQ h.]h-]h+]h,]U
candidatesrR }rS U*jQ sh1]uh jF h5]h'UimagerT ubcdocutils.nodes
caption
rU )rV }rW (h XK Figure 3. Smith's original data package using the ORE model Metacat has nowrX h jF h%h&h'UcaptionrY h)}rZ (h+]h,]h-]h.]h1]uh3KKh5]r[ h>XK Figure 3. Smith's original data package using the ORE model Metacat has nowr\ r] }r^ (h jX h jV ubaubeubh|)r_ }r` (h X Jessica Couture is an analyst who wants to create a graph of John's Smith data. She creates an R script that grabs the data files smith_data.1.1 and smith_data.2.1,
performs some sort of analysis on them and transforms them into a single data file, couture_data.1.1. She then creates another R script to visualize this composed data
into a chart. She describes her process and the data in an EML file and uploads all five files to Metacat. Below is the model that describes Couture's new data package:ra h j. h%h&h'hh)}rb (h+]h,]h-]h.]h1]uh3KMh4hh5]rc h>X Jessica Couture is an analyst who wants to create a graph of John's Smith data. She creates an R script that grabs the data files smith_data.1.1 and smith_data.2.1,
performs some sort of analysis on them and transforms them into a single data file, couture_data.1.1. She then creates another R script to visualize this composed data
into a chart. She describes her process and the data in an EML file and uploads all five files to Metacat. Below is the model that describes Couture's new data package:rd re }rf (h ja h j_ ubaubjE )rg }rh (h U h j. h%h&h'jH h)}ri (h+]h,]h-]h.]rj Uid5rk ah1]uh3Nh4hh5]rl (jM )rm }rn (h X .. figure:: images/Derived-data-example.png
Figure 4. Couture's derived data package with the existing Metacat ORE notation in blue and the new PROV notation in purple.
h)}ro (UuriX images/Derived-data-example.pngrp h.]h-]h+]h,]jR }rq U*jp sh1]uh jg h5]h'jT ubjU )rr }rs (h X| Figure 4. Couture's derived data package with the existing Metacat ORE notation in blue and the new PROV notation in purple.rt h jg h%h&h'jY h)}ru (h+]h,]h-]h.]h1]uh3KSh5]rv h>X| Figure 4. Couture's derived data package with the existing Metacat ORE notation in blue and the new PROV notation in purple.rw rx }ry (h jt h jr ubaubeubeubh )rz }r{ (h U h j h%h&h'h(h)}r| (h+]h,]h-]h.]r} U derived-data-without-scriptsr~ ah1]r hauh3KVh4hh5]r (h7)r }r (h X Derived data without scriptsr h jz h%h&h'h;h)}r (h+]h,]h-]h.]h1]uh3KVh4hh5]r h>X Derived data without scriptsr r }r (h j h j ubaubh|)r }r (h XB Not all derived data will be produced from a script that can be stored in Metacat. An analyst may create a composed data table through a more
manual process, such as transforming values in a spreadsheet program like MS Excel. Or a scientist may create a simple graph of their data with
MS Excel or an online graphing tool.r h jz h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3KWh4hh5]r h>XB Not all derived data will be produced from a script that can be stored in Metacat. An analyst may create a composed data table through a more
manual process, such as transforming values in a spreadsheet program like MS Excel. Or a scientist may create a simple graph of their data with
MS Excel or an online graphing tool.r r }r (h j h j ubaubh|)r }r (h X Let's say John Smith decides to create a time series chart using the data from one of his data files, smith_data.1.1. He adds this image file to
the original data package and edits the metadata to explain his process and the chart. His data package will essentially be using the PROV model without the activity concept.
Smith has the option of creating a new data package with the image and an EML file so that his metadata ID does not have to change.r h jz h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3K[h4hh5]r h>X Let's say John Smith decides to create a time series chart using the data from one of his data files, smith_data.1.1. He adds this image file to
the original data package and edits the metadata to explain his process and the chart. His data package will essentially be using the PROV model without the activity concept.
Smith has the option of creating a new data package with the image and an EML file so that his metadata ID does not have to change.r r }r (h j h j ubaubjE )r }r (h U h jz h%h&h'jH h)}r (h+]h,]h-]h.]r Uid6r ah1]uh3Nh4hh5]r (jM )r }r (h Xm .. figure:: images/PROV-simple-diagram-without-activity.png
Figure 4. The PROV model without an activity
h)}r (UuriX/ images/PROV-simple-diagram-without-activity.pngr h.]h-]h+]h,]jR }r U*j sh1]uh j h5]h'jT ubjU )r }r (h X, Figure 4. The PROV model without an activityr h j h%h&h'jY h)}r (h+]h,]h-]h.]h1]uh3Kah5]r h>X, Figure 4. The PROV model without an activityr r }r (h j h j ubaubeubjE )r }r (h U h jz h%h&h'jH h)}r (h+]h,]h-]h.]r Uid7r ah1]uh3Nh4hh5]r (jM )r }r (h X .. figure:: images/derived-data-smith-package-with-img.png
Figure 5. Smith's new data package with a data visualization. Note that the resource map and metadata have new revisions.
h)}r (UuriX. images/derived-data-smith-package-with-img.pngr h.]h-]h+]h,]jR }r U*j sh1]uh j h5]h'jT ubjU )r }r (h Xy Figure 5. Smith's new data package with a data visualization. Note that the resource map and metadata have new revisions.r h j h%h&h'jY h)}r (h+]h,]h-]h.]h1]uh3Keh5]r h>Xy Figure 5. Smith's new data package with a data visualization. Note that the resource map and metadata have new revisions.r r }r (h j h j ubaubeubeubeubh )r }r (h U h h#h%h&h'h(h)}r (h+]h,]h-]h.]r U
sample-rdfr ah1]r hauh3Khh4hh5]r (h7)r }r (h X
Sample RDFr h j h%h&h'h;h)}r (h+]h,]h-]h.]h1]uh3Khh4hh5]r h>X
Sample RDFr r }r (h j h j ubaubh|)r }r (h X
A sample RDF notation for our above Couture and Smith example (Figure 4) can be :download:`downloaded <_static/resourceMap_couture.1.1.rdf>` or viewed below. Note that the
primary data resource will need to be described in the RDFs for its derivations so that an RDF for a derived dataset will have all the information it needs to relate the primary
resource to the derived resource. This way, a SparQL query can be run within a derived dataset RDF to make the assertion that derived_metadata wasDerivedFrom primary_metadata.h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kih4hh5]r (h>XP A sample RDF notation for our above Couture and Smith example (Figure 4) can be r r }r (h XP A sample RDF notation for our above Couture and Smith example (Figure 4) can be h j ubcsphinx.addnodes
download_reference
r )r }r (h X< :download:`downloaded <_static/resourceMap_couture.1.1.rdf>`r h j h%h&h'Udownload_referencer h)}r (UreftypeX downloadr Urefwarnr U reftargetr X# _static/resourceMap_couture.1.1.rdfr U refdomainU h.]h-]U refexplicith+]h,]h1]Urefdocr X ore-model-expansionr Ufilenamer X resourceMap_couture.1.1.rdfr uh3Kih5]r cdocutils.nodes
literal
r )r }r (h j h)}r (h+]h,]r (Uxrefr j eh-]h.]h1]uh j h5]r h>X
downloadedr r }r (h U h j ubah'Uliteralr ubaubh>X or viewed below. Note that the
primary data resource will need to be described in the RDFs for its derivations so that an RDF for a derived dataset will have all the information it needs to relate the primary
resource to the derived resource. This way, a SparQL query can be run within a derived dataset RDF to make the assertion that derived_metadata wasDerivedFrom primary_metadata.r r }r (h X or viewed below. Note that the
primary data resource will need to be described in the RDFs for its derivations so that an RDF for a derived dataset will have all the information it needs to relate the primary
resource to the derived resource. This way, a SparQL query can be run within a derived dataset RDF to make the assertion that derived_metadata wasDerivedFrom primary_metadata.h j ubeubcdocutils.nodes
literal_block
r )r }r (h Xz
https://cn.dataone.org/cn/v1/resolve/resourceMap_couture.1.1#aggregation
couture_data.1.1
https://cn.dataone.org/cn/v1/resolve/resourceMap_smith .1.1#aggregation
smith_data.1.1
DataONE Aggregation
Java libclient
2013-09-03T09:54:06.392-07:00
resourceMap_couture.1.1
https://cn.dataone.org/cn/v1/resolve/resourceMap_couture.1.1#aggregation
couture_metadata.1.1
https://cn.dataone.org/cn/v1/resolve/resourceMap_couture.1.1#aggregation
couture_img.1.1
https://cn.dataone.org/cn/v1/resolve/resourceMap_couture.1.1#aggregation
couture_script.1.1
https://cn.dataone.org/cn/v1/resolve/resourceMap_couture.1.1#aggregation
couture_composeScript.1.1
h j h%h&h'U
literal_blockr h)}r (U xml:spacer Upreserver h.]h-]h+]h,]h1]uh3Koh4hh5]r h>Xz
https://cn.dataone.org/cn/v1/resolve/resourceMap_couture.1.1#aggregation
couture_data.1.1
https://cn.dataone.org/cn/v1/resolve/resourceMap_smith .1.1#aggregation
smith_data.1.1
DataONE Aggregation
Java libclient
2013-09-03T09:54:06.392-07:00
resourceMap_couture.1.1
https://cn.dataone.org/cn/v1/resolve/resourceMap_couture.1.1#aggregation
couture_metadata.1.1
https://cn.dataone.org/cn/v1/resolve/resourceMap_couture.1.1#aggregation
couture_img.1.1
https://cn.dataone.org/cn/v1/resolve/resourceMap_couture.1.1#aggregation
couture_script.1.1
https://cn.dataone.org/cn/v1/resolve/resourceMap_couture.1.1#aggregation
couture_composeScript.1.1
r r }r (h U h j ubaubeubh )r }r (h U h h#h%h&h'h(h)}r (h+]h,]h-]h.]r Uindexingr ah1]r h auh3Kh4hh5]r (h7)r }r (h X Indexingr h j h%h&h'h;h)}r (h+]h,]h-]h.]h1]uh3Kh4hh5]r h>X Indexingr r
}r (h j h j ubaubh|)r }r
(h XT The Metacat Index will index some of these relationships mainly to inform MetacatUI.r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh4hh5]r h>XT The Metacat Index will index some of these relationships mainly to inform MetacatUI.r r }r (h j h j ubaubh])r }r (h U h j h%Nh'hah)}r (h+]h,]h-]h.]h1]uh3Nh4hh5]r hg)r }r (h X~ New Index Fields
* ``wasGeneratedBy``
* ``generated``
* ``used``
* ``wasDerivedFrom``
* ``wasInformedBy``
* ``hadDerivation``
h j h%h&h'hjh)}r (h+]h,]h-]h.]h1]uh3Kh5]r (hm)r }r (h X New Index Fieldsh j h%h&h'hph)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X New Index Fieldsr r! }r" (h X New Index Fieldsr# h j ubaubhw)r$ }r% (h U h)}r& (h+]h,]h-]h.]h1]uh j h5]r' h)r( }r) (h U h)}r* (hX *h.]h-]h+]h,]h1]uh j$ h5]r+ (h)r, }r- (h X ``wasGeneratedBy``r. h)}r/ (h+]h,]h-]h.]h1]uh j( h5]r0 h|)r1 }r2 (h j. h j, h%h&h'hh)}r3 (h+]h,]h-]h.]h1]uh3Kh5]r4 j )r5 }r6 (h j. h)}r7 (h+]h,]h-]h.]h1]uh j1 h5]r8 h>X wasGeneratedByr9 r: }r; (h U h j5 ubah'j ubaubah'hubh)r< }r= (h X
``generated``r> h)}r? (h+]h,]h-]h.]h1]uh j( h5]r@ h|)rA }rB (h j> h j< h%h&h'hh)}rC (h+]h,]h-]h.]h1]uh3Kh5]rD j )rE }rF (h j> h)}rG (h+]h,]h-]h.]h1]uh jA h5]rH h>X generatedrI rJ }rK (h U h jE ubah'j ubaubah'hubh)rL }rM (h X ``used``rN h)}rO (h+]h,]h-]h.]h1]uh j( h5]rP h|)rQ }rR (h jN h jL h%h&h'hh)}rS (h+]h,]h-]h.]h1]uh3Kh5]rT j )rU }rV (h jN h)}rW (h+]h,]h-]h.]h1]uh jQ h5]rX h>X usedrY rZ }r[ (h U h jU ubah'j ubaubah'hubh)r\ }r] (h X ``wasDerivedFrom``r^ h)}r_ (h+]h,]h-]h.]h1]uh j( h5]r` h|)ra }rb (h j^ h j\ h%h&h'hh)}rc (h+]h,]h-]h.]h1]uh3Kh5]rd j )re }rf (h j^ h)}rg (h+]h,]h-]h.]h1]uh ja h5]rh h>X wasDerivedFromri rj }rk (h U h je ubah'j ubaubah'hubh)rl }rm (h X ``wasInformedBy``rn h)}ro (h+]h,]h-]h.]h1]uh j( h5]rp h|)rq }rr (h jn h jl h%h&h'hh)}rs (h+]h,]h-]h.]h1]uh3Kh5]rt j )ru }rv (h jn h)}rw (h+]h,]h-]h.]h1]uh jq h5]rx h>X
wasInformedByry rz }r{ (h U h ju ubah'j ubaubah'hubh)r| }r} (h X ``hadDerivation``
h)}r~ (h+]h,]h-]h.]h1]uh j( h5]r h|)r }r (h X ``hadDerivation``r h j| h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r j )r }r (h j h)}r (h+]h,]h-]h.]h1]uh j h5]r h>X
hadDerivationr r }r (h U h j ubah'j ubaubah'hubeh'hubah'hubeubaubh|)r }r (h X All of these fields are taken directly from the PROV ontology, except for ``hadDerivation``. Because MetacatUI navigation is metadata-centric
(i.e. metadata are retrieved and listed as search results and details on datasets are viewed via the metadata view [/#view]), it is helpful to
use SparQL to infer relationships between metadata so that it isn't necessary to send multiple queries to the index to retrieve information about
derived datasets.h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh4hh5]r (h>XJ All of these fields are taken directly from the PROV ontology, except for r r }r (h XJ All of these fields are taken directly from the PROV ontology, except for h j ubj )r }r (h X ``hadDerivation``h)}r (h+]h,]h-]h.]h1]uh j h5]r h>X
hadDerivationr r }r (h U h j ubah'j ubh>Xe . Because MetacatUI navigation is metadata-centric
(i.e. metadata are retrieved and listed as search results and details on datasets are viewed via the metadata view [/#view]), it is helpful to
use SparQL to infer relationships between metadata so that it isn't necessary to send multiple queries to the index to retrieve information about
derived datasets.r r }r (h Xe . Because MetacatUI navigation is metadata-centric
(i.e. metadata are retrieved and listed as search results and details on datasets are viewed via the metadata view [/#view]), it is helpful to
use SparQL to infer relationships between metadata so that it isn't necessary to send multiple queries to the index to retrieve information about
derived datasets.h j ubeubh|)r }r (h X( SparQL will infer that the derived metadata wasDerivedFrom the primary source metadata and the Metacat index will index both this relationship and it's inverse,
hadDerivation, for easier querying. The SparQL query used to determine the derived metadata for a primary metadata would look like:r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh4hh5]r h>X( SparQL will infer that the derived metadata wasDerivedFrom the primary source metadata and the Metacat index will index both this relationship and it's inverse,
hadDerivation, for easier querying. The SparQL query used to determine the derived metadata for a primary metadata would look like:r r }r (h j h j ubaubj )r }r (h X SELECT ?primary_metadata_pid ?derived_metadata_pid
WHERE {
?primary_data ore:isDocumentedBy ?primary_metadata .
?primary_metadata dcterms:identifier ?primary_metadata_pid .
?derived_data prov:wasDerivedFrom ?primary_data .
?derived_data ore:isDocumentedBy ?derived_metadata .
?derived_metadata dcterms:identifier ?derived_metadata_pid .
}h j h%h&h'j h)}r (j j h.]h-]h+]h,]h1]uh3Kh4hh5]r h>X SELECT ?primary_metadata_pid ?derived_metadata_pid
WHERE {
?primary_data ore:isDocumentedBy ?primary_metadata .
?primary_metadata dcterms:identifier ?primary_metadata_pid .
?derived_data prov:wasDerivedFrom ?primary_data .
?derived_data ore:isDocumentedBy ?derived_metadata .
?derived_metadata dcterms:identifier ?derived_metadata_pid .
}r r }r (h U h j ubaubh|)r }r (h X More information about inverses in PROV-O and in particular, ``hadDerivation``, see `section B of the PROV documentation `_.r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh4hh5]r (h>X= More information about inverses in PROV-O and in particular, r r }r (h X= More information about inverses in PROV-O and in particular, h j ubj )r }r (h X ``hadDerivation``h)}r (h+]h,]h-]h.]h1]uh j h5]r h>X
hadDerivationr r }r (h U h j ubah'j ubh>X , see r r }r (h X , see h j ubh)r }r (h XS `section B of the PROV documentation `_h)}r (UnameX# section B of the PROV documentationhX* http://www.w3.org/TR/prov-o/#inverse-namesr h.]h-]h+]h,]h1]uh j h5]r h>X# section B of the PROV documentationr r }r (h U h j ubah'hubhM)r }r (h X- hKh j h'hPh)}r (Urefurij h.]r U#section-b-of-the-prov-documentationr ah-]h+]h,]h1]r h auh5]ubh>X .r }r (h X .h j ubeubh )r }r (h U h j h%h&h'h(h)}r (h+]h,]h-]h.]r Uexampler ah1]r hauh3Kh4hh5]r (h7)r }r (h X Exampler h j h%h&h'h;h)}r (h+]h,]h-]h.]h1]uh3Kh4hh5]r h>X Exampler r }r (h j h j ubaubh|)r }r (h X Using the above Couture and Smith example (Figure 4), the following fields and values from the PROV relationships would be indexed.r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh4hh5]r h>X Using the above Couture and Smith example (Figure 4), the following fields and values from the PROV relationships would be indexed.r r }r (h j h j ubaubcdocutils.nodes
table
r )r }r (h U h j h%h&h'Utabler h)}r (h+]h,]h-]h.]h1]uh3Nh4hh5]r cdocutils.nodes
tgroup
r )r }r (h U h)}r (h.]h-]h+]h,]h1]UcolsKuh j h5]r (cdocutils.nodes
colspec
r )r }r (h U h)}r (h.]h-]h+]h,]h1]UcolwidthKuh j h5]h'Ucolspecr ubj )r }r (h U h)}r (h.]h-]h+]h,]h1]UcolwidthKuh j h5]h'j ubj )r }r (h U h)}r (h.]h-]h+]h,]h1]UcolwidthKuh j h5]h'j ubj )r }r (h U h)}r (h.]h-]h+]h,]h1]UcolwidthK uh j h5]h'j ubcdocutils.nodes
thead
r )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r cdocutils.nodes
row
r )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r (cdocutils.nodes
entry
r )r }r
(h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r
}r (h X Objectr h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X Objectr r }r (h j h j
ubaubah'Uentryr ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X
Field Namer h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X
Field Namer r }r! (h j h j ubaubah'j ubj )r" }r# (h U h)}r$ (h+]h,]h-]h.]h1]uh j h5]r% h|)r& }r' (h X
Field Typer( h j" h%h&h'hh)}r) (h+]h,]h-]h.]h1]uh3Kh5]r* h>X
Field Typer+ r, }r- (h j( h j& ubaubah'j ubj )r. }r/ (h U h)}r0 (h+]h,]h-]h.]h1]uh j h5]r1 h|)r2 }r3 (h X Valuer4 h j. h%h&h'hh)}r5 (h+]h,]h-]h.]h1]uh3Kh5]r6 h>X Valuer7 r8 }r9 (h j4 h j2 ubaubah'j ubeh'Urowr: ubah'Utheadr; ubcdocutils.nodes
tbody
r< )r= }r> (h U h)}r? (h+]h,]h-]h.]h1]uh j h5]r@ (j )rA }rB (h U h)}rC (h+]h,]h-]h.]h1]uh j= h5]rD (j )rE }rF (h U h)}rG (h+]h,]h-]h.]h1]uh jA h5]rH h|)rI }rJ (h X couture_metadata.1.1rK h jE h%h&h'hh)}rL (h+]h,]h-]h.]h1]uh3Kh5]rM h>X couture_metadata.1.1rN rO }rP (h jK h jI ubaubah'j ubj )rQ }rR (h U h)}rS (h+]h,]h-]h.]h1]uh jA h5]rT h|)rU }rV (h X wasDerivedFromrW h jQ h%h&h'hh)}rX (h+]h,]h-]h.]h1]uh3Kh5]rY h>X wasDerivedFromrZ r[ }r\ (h jW h jU ubaubah'j ubj )r] }r^ (h U h)}r_ (h+]h,]h-]h.]h1]uh jA h5]r` h|)ra }rb (h X string, multivaluedrc h j] h%h&h'hh)}rd (h+]h,]h-]h.]h1]uh3Kh5]re h>X string, multivaluedrf rg }rh (h jc h ja ubaubah'j ubj )ri }rj (h U h)}rk (h+]h,]h-]h.]h1]uh jA h5]rl h|)rm }rn (h X smith_metadata.1.1ro h ji h%h&h'hh)}rp (h+]h,]h-]h.]h1]uh3Kh5]rq h>X smith_metadata.1.1rr rs }rt (h jo h jm ubaubah'j ubeh'j: ubj )ru }rv (h U h)}rw (h+]h,]h-]h.]h1]uh j= h5]rx (j )ry }rz (h U h)}r{ (h+]h,]h-]h.]h1]uh ju h5]r| h|)r} }r~ (h X smith_metadata.1.1r h jy h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X smith_metadata.1.1r r }r (h j h j} ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh ju h5]r h|)r }r (h X
hadDerivationr h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X
hadDerivationr r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh ju h5]r h|)r }r (h X string, multivaluedr h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X string, multivaluedr r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh ju h5]r h|)r }r (h X couture_metadata.1.1r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X couture_metadata.1.1r r }r (h j h j ubaubah'j ubeh'j: ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j= h5]r (j )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X couture_img.1.1r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X couture_img.1.1r r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X wasGeneratedbyr h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X wasGeneratedbyr r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X string, multivaluedr h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X string, multivaluedr r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X couture_script.1.1r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X couture_script.1.1r r }r (h j h j ubaubah'j ubeh'j: ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j= h5]r (j )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X couture_img.1.1r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X couture_img.1.1r r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X wasDerivedFromr h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X wasDerivedFromr r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X string, multivaluedr h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X string, multivaluedr r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r
(h X couture_data.1.1r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r
h>X couture_data.1.1r r }r (h j h j ubaubah'j ubeh'j: ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j= h5]r (j )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X couture_script.1.1r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X couture_script.1.1r r }r (h j h j ubaubah'j ubj )r! }r" (h U h)}r# (h+]h,]h-]h.]h1]uh j h5]r$ h|)r% }r& (h X usedr' h j! h%h&h'hh)}r( (h+]h,]h-]h.]h1]uh3Kh5]r) h>X usedr* r+ }r, (h j' h j% ubaubah'j ubj )r- }r. (h U h)}r/ (h+]h,]h-]h.]h1]uh j h5]r0 h|)r1 }r2 (h X string, multivaluedr3 h j- h%h&h'hh)}r4 (h+]h,]h-]h.]h1]uh3Kh5]r5 h>X string, multivaluedr6 r7 }r8 (h j3 h j1 ubaubah'j ubj )r9 }r: (h U h)}r; (h+]h,]h-]h.]h1]uh j h5]r< h|)r= }r> (h X couture_data.1.1r? h j9 h%h&h'hh)}r@ (h+]h,]h-]h.]h1]uh3Kh5]rA h>X couture_data.1.1rB rC }rD (h j? h j= ubaubah'j ubeh'j: ubj )rE }rF (h U h)}rG (h+]h,]h-]h.]h1]uh j= h5]rH (j )rI }rJ (h U h)}rK (h+]h,]h-]h.]h1]uh jE h5]rL h|)rM }rN (h X couture_script.1.1rO h jI h%h&h'hh)}rP (h+]h,]h-]h.]h1]uh3Kh5]rQ h>X couture_script.1.1rR rS }rT (h jO h jM ubaubah'j ubj )rU }rV (h U h)}rW (h+]h,]h-]h.]h1]uh jE h5]rX h|)rY }rZ (h X
wasInformedByr[ h jU h%h&h'hh)}r\ (h+]h,]h-]h.]h1]uh3Kh5]r] h>X
wasInformedByr^ r_ }r` (h j[ h jY ubaubah'j ubj )ra }rb (h U h)}rc (h+]h,]h-]h.]h1]uh jE h5]rd h|)re }rf (h X string, multivaluedrg h ja h%h&h'hh)}rh (h+]h,]h-]h.]h1]uh3Kh5]ri h>X string, multivaluedrj rk }rl (h jg h je ubaubah'j ubj )rm }rn (h U h)}ro (h+]h,]h-]h.]h1]uh jE h5]rp h|)rq }rr (h X couture_composeScript.1.1rs h jm h%h&h'hh)}rt (h+]h,]h-]h.]h1]uh3Kh5]ru h>X couture_composeScript.1.1rv rw }rx (h js h jq ubaubah'j ubeh'j: ubj )ry }rz (h U h)}r{ (h+]h,]h-]h.]h1]uh j= h5]r| (j )r} }r~ (h U h)}r (h+]h,]h-]h.]h1]uh jy h5]r h|)r }r (h X couture_script.1.1r h j} h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X couture_script.1.1r r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh jy h5]r h|)r }r (h X generatedr h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X generatedr r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh jy h5]r h|)r }r (h X string, multivaluedr h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X string, multivaluedr r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh jy h5]r h|)r }r (h X couture_img.1.1r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Kh5]r h>X couture_img.1.1r r }r (h j h j ubaubah'j ubeh'j: ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j= h5]r (j )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X couture_data.1.1r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Mh5]r h>X couture_data.1.1r r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X wasGeneratedByr h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Mh5]r h>X wasGeneratedByr r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X string, multivaluedr h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Mh5]r h>X string, multivaluedr r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X couture_composeScript.1.1r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Mh5]r h>X couture_composeScript.1.1r r }r (h j h j ubaubah'j ubeh'j: ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j= h5]r (j )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X couture_data.1.1r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Mh5]r h>X couture_data.1.1r r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X wasDerivedFromr h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Mh5]r h>X wasDerivedFromr r }r (h j h j ubaubah'j ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X string, multivaluedr h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Mh5]r h>X string, multivaluedr r }r (h j h j ubaubah'j ubj )r }r
(h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r
}r (h X smith_data.1.1, smith_data.2.1r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Mh5]r h>X smith_data.1.1, smith_data.2.1r r }r (h j h j
ubaubah'j ubeh'j: ubj )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j= h5]r (j )r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h|)r }r (h X couture_composeScript.1.1r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Mh5]r! h>X couture_composeScript.1.1r" r# }r$ (h j h j ubaubah'j ubj )r% }r& (h U h)}r' (h+]h,]h-]h.]h1]uh j h5]r( h|)r) }r* (h X usedr+ h j% h%h&h'hh)}r, (h+]h,]h-]h.]h1]uh3Mh5]r- h>X usedr. r/ }r0 (h j+ h j) ubaubah'j ubj )r1 }r2 (h U h)}r3 (h+]h,]h-]h.]h1]uh j h5]r4 h|)r5 }r6 (h X string, multivaluedr7 h j1 h%h&h'hh)}r8 (h+]h,]h-]h.]h1]uh3Mh5]r9 h>X string, multivaluedr: r; }r< (h j7 h j5 ubaubah'j ubj )r= }r> (h U h)}r? (h+]h,]h-]h.]h1]uh j h5]r@ h|)rA }rB (h X smith_data.1.1, smith_data.2.1rC h j= h%h&h'hh)}rD (h+]h,]h-]h.]h1]uh3Mh5]rE h>X smith_data.1.1, smith_data.2.1rF rG }rH (h jC h jA ubaubah'j ubeh'j: ubj )rI }rJ (h U h)}rK (h+]h,]h-]h.]h1]uh j= h5]rL (j )rM }rN (h U h)}rO (h+]h,]h-]h.]h1]uh jI h5]rP h|)rQ }rR (h X couture_composeScript.1.1rS h jM h%h&h'hh)}rT (h+]h,]h-]h.]h1]uh3Mh5]rU h>X couture_composeScript.1.1rV rW }rX (h jS h jQ ubaubah'j ubj )rY }rZ (h U h)}r[ (h+]h,]h-]h.]h1]uh jI h5]r\ h|)r] }r^ (h X generatedr_ h jY h%h&h'hh)}r` (h+]h,]h-]h.]h1]uh3Mh5]ra h>X generatedrb rc }rd (h j_ h j] ubaubah'j ubj )re }rf (h U h)}rg (h+]h,]h-]h.]h1]uh jI h5]rh h|)ri }rj (h X string, multivaluedrk h je h%h&h'hh)}rl (h+]h,]h-]h.]h1]uh3Mh5]rm h>X string, multivaluedrn ro }rp (h jk h ji ubaubah'j ubj )rq }rr (h U h)}rs (h+]h,]h-]h.]h1]uh jI h5]rt h|)ru }rv (h X couture_data.1.1rw h jq h%h&h'hh)}rx (h+]h,]h-]h.]h1]uh3Mh5]ry h>X couture_data.1.1rz r{ }r| (h jw h ju ubaubah'j ubeh'j: ubeh'Utbodyr} ubeh'Utgroupr~ ubaubeubh )r }r (h U h j h%h&h'h(h)}r (h+]h,]h-]h.]r Uqueriesr ah1]r h
auh3M h4hh5]r (h7)r }r (h X Queriesr h j h%h&h'h;h)}r (h+]h,]h-]h.]h1]uh3M h4hh5]r h>X Queriesr r }r (h j h j ubaubh|)r }r (h X These indexed fields will be used primarily by MetacatUI to determine relationships between resources. An example use case is the MetacatUI metadata view (/#view/{pid}).
When a user navigates to the metadata view for a primary resource, such as smith_metadata.1.1 from the above example, MetacatUI can send a query to the index to look for
derived resources that the user might be interested in. Some pseduocode to describe this logic:r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3M h4hh5]r h>X These indexed fields will be used primarily by MetacatUI to determine relationships between resources. An example use case is the MetacatUI metadata view (/#view/{pid}).
When a user navigates to the metadata view for a primary resource, such as smith_metadata.1.1 from the above example, MetacatUI can send a query to the index to look for
derived resources that the user might be interested in. Some pseduocode to describe this logic:r r }r (h j h j ubaubj )r }r (h X if(this_metadata.hadDerivation){
hadDerivation = this_metadata.hadDerivation;
sendQuery("q=isDocumentedBy:" + hadDerivation + "+-obsoletedBy:*&fl=id");
}h j h%h&h'j h)}r (j j h.]h-]h+]h,]h1]uh3Mh4hh5]r h>X if(this_metadata.hadDerivation){
hadDerivation = this_metadata.hadDerivation;
sendQuery("q=isDocumentedBy:" + hadDerivation + "+-obsoletedBy:*&fl=id");
}r r }r (h U h j ubaubh|)r }r (h X This query would retrieve the IDs of objects which are documented by metadata that is derived from the metadata currently being viewed. In the above Amith and Couture example (Figure 4), the following IDs
would be returned from this query:r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Mh4hh5]r h>X This query would retrieve the IDs of objects which are documented by metadata that is derived from the metadata currently being viewed. In the above Amith and Couture example (Figure 4), the following IDs
would be returned from this query:r r }r (h j h j ubaubh)r }r (h U h j h%h&h'hh)}r (hX *h.]h-]h+]h,]h1]uh3M h4hh5]r (h)r }r (h X ``couture_composeScript.1.1``r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Nh4hh5]r h|)r }r (h j h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3M h5]r j )r }r (h j h)}r (h+]h,]h-]h.]h1]uh j h5]r h>X couture_composeScript.1.1r r }r (h U h j ubah'j ubaubaubh)r }r (h X ``couture_script.1.1``r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Nh4hh5]r h|)r }r (h j h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3M h5]r j )r }r (h j h)}r (h+]h,]h-]h.]h1]uh j h5]r h>X couture_script.1.1r r }r (h U h j ubah'j ubaubaubh)r }r (h X ``couture_data.1.1``r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Nh4hh5]r h|)r }r (h j h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3M h5]r j )r }r (h j h)}r (h+]h,]h-]h.]h1]uh j h5]r h>X couture_data.1.1r r }r (h U h j ubah'j ubaubaubh)r }r (h X ``couture_img.1.1``
h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3Nh4hh5]r h|)r }r (h X ``couture_img.1.1``r h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3M h5]r j )r }r (h j h)}r (h+]h,]h-]h.]h1]uh j h5]r h>X couture_img.1.1r r }r (h U h j ubah'j ubaubaubeubeubeubeubh%h&h'h(h)}r (h+]h,]h-]h.]r Uontology-and-modelr ah1]r hauh3K(h4hh5]r (h7)r }r (h X Ontology and Modelr h h!h%h&h'h;h)}r (h+]h,]h-]h.]h1]uh3K(h4hh5]r h>X Ontology and Modelr r }r (h j h j ubaubh])r }r (h U h h!h%Nh'hah)}r (h+]h,]h-]h.]h1]uh3Nh4hh5]r hg)r }r (h X The `PROV ontology `_ will be used to describe the following relationships:
* Derived resources and the original resources used to create them
* Derived resources and the activities used to create them
h j h%h&h'hjh)}r (h+]h,]h-]h.]h1]uh3K+h5]r (hm)r }r (h XG The PROV ontology will be used to describe the following relationships:h j h%h&h'hph)}r (h+]h,]h-]h.]h1]uh3K+h5]r (h>X The r r }r (h X The h j ubh)r }r (h X6 `PROV ontology `_h)}r (UnameX
PROV ontologyhX# http://www.w3.org/TR/prov-overview/r h.]h-]h+]h,]h1]uh j h5]r h>X
PROV ontologyr
r }r (h U h j ubah'hubhM)r
}r (h X& hKh j h'hPh)}r (Urefurij h.]r Uid1r ah-]h+]r X
prov ontologyr ah,]h1]uh5]ubh>X6 will be used to describe the following relationships:r r }r (h X6 will be used to describe the following relationships:h j ubeubhw)r }r (h U h)}r (h+]h,]h-]h.]h1]uh j h5]r h)r }r (h U h)}r (hX *h.]h-]h+]h,]h1]uh j h5]r (h)r }r (h X@ Derived resources and the original resources used to create themr! h)}r" (h+]h,]h-]h.]h1]uh j h5]r# h|)r$ }r% (h j! h j h%h&h'hh)}r& (h+]h,]h-]h.]h1]uh3K*h5]r' h>X@ Derived resources and the original resources used to create themr( r) }r* (h j! h j$ ubaubah'hubh)r+ }r, (h X9 Derived resources and the activities used to create them
h)}r- (h+]h,]h-]h.]h1]uh j h5]r. h|)r/ }r0 (h X8 Derived resources and the activities used to create themr1 h j+ h%h&h'hh)}r2 (h+]h,]h-]h.]h1]uh3K+h5]r3 h>X8 Derived resources and the activities used to create themr4 r5 }r6 (h j1 h j/ ubaubah'hubeh'hubah'hubeubaubjE )r7 }r8 (h U h h!h%h&h'jH h)}r9 (h+]h,]h-]h.]r: Uid2r; ah1]uh3Nh4hh5]r< (jM )r= }r> (h Xz .. figure:: images/PROV-simple-diagram.png
Figure 1. A simple diagram illustrating the PROV ontology Metacat will use
h)}r? (UuriX images/PROV-simple-diagram.pngr@ h.]h-]h+]h,]jR }rA U*j@ sh1]uh j7 h5]h'jT ubjU )rB }rC (h XJ Figure 1. A simple diagram illustrating the PROV ontology Metacat will userD h j7 h%h&h'jY h)}rE (h+]h,]h-]h.]h1]uh3K/h5]rF h>XJ Figure 1. A simple diagram illustrating the PROV ontology Metacat will userG rH }rI (h jD h jB ubaubeubh|)rJ }rK (h X This model will allow the original data file to remain unchanged as all assertions made about the derived data can be
described in the derived data RDF.rL h h!h%h&h'hh)}rM (h+]h,]h-]h.]h1]uh3K1h4hh5]rN h>X This model will allow the original data file to remain unchanged as all assertions made about the derived data can be
described in the derived data RDF.rO rP }rQ (h jL h jJ ubaubh])rR }rS (h U h h!h%Nh'hah)}rT (h+]h,]h-]h.]h1]uh3Nh4hh5]rU hg)rV }rW (h X The terms from the PROV ontology that will be introduced to Metacat to describe these relationships:
* `wasGeneratedBy `_ : Generation is the completion of production of a new entity by an activity. This entity did not exist before generation and becomes available for usage after this generation.
* `generated `_ : Generation is the completion of production of a new entity by an activity. This entity did not exist before generation and becomes available for usage after this generation.
* `wasDerivedFrom `_ : A derivation is a transformation of an entity into another, an update of an entity resulting in a new one, or the construction of a new entity based on a pre-existing entity.
* `used `_ : Usage is the beginning of utilizing an entity by an activity. Before usage, the activity had not begun to utilize this entity and could not have been affected by the entity.
* `wasInformedBy `_ : Communication is the exchange of an entity by two activities, one activity using the entity generated by the other.
h jR h%h&h'hjh)}rX (h+]h,]h-]h.]h1]uh3K:h5]rY (hm)rZ }r[ (h Xd The terms from the PROV ontology that will be introduced to Metacat to describe these relationships:h jV h%h&h'hph)}r\ (h+]h,]h-]h.]h1]uh3K:h5]r] h>Xd The terms from the PROV ontology that will be introduced to Metacat to describe these relationships:r^ r_ }r` (h Xd The terms from the PROV ontology that will be introduced to Metacat to describe these relationships:ra h jZ ubaubhw)rb }rc (h U h)}rd (h+]h,]h-]h.]h1]uh jV h5]re h)rf }rg (h U h)}rh (hX *h.]h-]h+]h,]h1]uh jb h5]ri (h)rj }rk (h X `wasGeneratedBy `_ : Generation is the completion of production of a new entity by an activity. This entity did not exist before generation and becomes available for usage after this generation.rl h)}rm (h+]h,]h-]h.]h1]uh jf h5]rn h|)ro }rp (h jl h jj h%h&h'hh)}rq (h+]h,]h-]h.]h1]uh3K6h5]rr (h)rs }rt (h X< `wasGeneratedBy `_h)}ru (UnameX wasGeneratedByhX( http://www.w3.org/ns/prov#wasGeneratedByrv h.]h-]h+]h,]h1]uh jo h5]rw h>X wasGeneratedByrx ry }rz (h U h js ubah'hubhM)r{ }r| (h X+ hKh jo h'hPh)}r} (Urefurijv h.]r~ Uwasgeneratedbyr ah-]h+]h,]h1]r hauh5]ubh>X : Generation is the completion of production of a new entity by an activity. This entity did not exist before generation and becomes available for usage after this generation.r r }r (h X : Generation is the completion of production of a new entity by an activity. This entity did not exist before generation and becomes available for usage after this generation.h jo ubeubah'hubh)r }r (h X `generated `_ : Generation is the completion of production of a new entity by an activity. This entity did not exist before generation and becomes available for usage after this generation.r h)}r (h+]h,]h-]h.]h1]uh jf h5]r h|)r }r (h j h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3K7h5]r (h)r }r (h X2 `generated `_h)}r (UnamehhX# http://www.w3.org/ns/prov#generatedr h.]h-]h+]h,]h1]uh j h5]r h>X generatedr r }r (h U h j ubah'hubhM)r }r (h X& hKh j h'hPh)}r (Urefurij h.]r U generatedr ah-]h+]h,]h1]r hauh5]ubh>X : Generation is the completion of production of a new entity by an activity. This entity did not exist before generation and becomes available for usage after this generation.r r }r (h X : Generation is the completion of production of a new entity by an activity. This entity did not exist before generation and becomes available for usage after this generation.h j ubeubah'hubh)r }r (h X `wasDerivedFrom `_ : A derivation is a transformation of an entity into another, an update of an entity resulting in a new one, or the construction of a new entity based on a pre-existing entity.r h)}r (h+]h,]h-]h.]h1]uh jf h5]r h|)r }r (h j h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3K8h5]r (h)r }r (h X< `wasDerivedFrom `_h)}r (UnameX wasDerivedFromhX( http://www.w3.org/ns/prov#wasDerivedFromr h.]h-]h+]h,]h1]uh j h5]r h>X wasDerivedFromr r }r (h U h j ubah'hubhM)r }r (h X+ hKh j h'hPh)}r (Urefurij h.]r Uwasderivedfromr ah-]h+]h,]h1]r hauh5]ubh>X : A derivation is a transformation of an entity into another, an update of an entity resulting in a new one, or the construction of a new entity based on a pre-existing entity.r r }r (h X : A derivation is a transformation of an entity into another, an update of an entity resulting in a new one, or the construction of a new entity based on a pre-existing entity.h j ubeubah'hubh)r }r (h X `used `_ : Usage is the beginning of utilizing an entity by an activity. Before usage, the activity had not begun to utilize this entity and could not have been affected by the entity.r h)}r (h+]h,]h-]h.]h1]uh jf h5]r h|)r }r (h j h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3K9h5]r (h)r }r (h X( `used `_h)}r (UnamehhX http://www.w3.org/ns/prov#usedr h.]h-]h+]h,]h1]uh j h5]r h>X usedr r }r (h U h j ubah'hubhM)r }r (h X! hKh j h'hPh)}r (Urefurij h.]r Uusedr ah-]h+]h,]h1]r hauh5]ubh>X : Usage is the beginning of utilizing an entity by an activity. Before usage, the activity had not begun to utilize this entity and could not have been affected by the entity.r r }r (h X : Usage is the beginning of utilizing an entity by an activity. Before usage, the activity had not begun to utilize this entity and could not have been affected by the entity.h j ubeubah'hubh)r }r (h X `wasInformedBy `_ : Communication is the exchange of an entity by two activities, one activity using the entity generated by the other.
h)}r (h+]h,]h-]h.]h1]uh jf h5]r h|)r }r (h X `wasInformedBy `_ : Communication is the exchange of an entity by two activities, one activity using the entity generated by the other.h j h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3K:h5]r (h)r }r (h X: `wasInformedBy `_h)}r (UnameX
wasInformedByhX' http://www.w3.org/ns/prov#wasInformedByr h.]h-]h+]h,]h1]uh j h5]r h>X
wasInformedByr r }r (h U h j ubah'hubhM)r }r (h X* hKh j h'hPh)}r (Urefurij h.]r U
wasinformedbyr ah-]h+]h,]h1]r hauh5]ubh>Xv : Communication is the exchange of an entity by two activities, one activity using the entity generated by the other.r r }r (h Xv : Communication is the exchange of an entity by two activities, one activity using the entity generated by the other.h j ubeubah'hubeh'hubah'hubeubaubh|)r }r (h X~ The following diagram fully represents the new model for derived resources. New terms and resources are highlighted in purple.r h h!h%h&h'hh)}r (h+]h,]h-]h.]h1]uh3KX~ The following diagram fully represents the new model for derived resources. New terms and resources are highlighted in purple.r r }r (h j h j ubaubjE )r }r (h U h h!h%h&h'jH h)}r (h+]h,]h-]h.]r Uid3r ah1]uh3Nh4hh5]r (jM )r }r (h X .. figure:: images/Derived-data-packages-model.png
Figure 2. The model for derived data sets with new terms highlighted in purple.
h)}r (UuriX& images/Derived-data-packages-model.pngr h.]h-]h+]h,]jR }r U*j sh1]uh j h5]h'jT ubjU )r }r (h XO Figure 2. The model for derived data sets with new terms highlighted in purple.r h j h%h&h'jY h)}r (h+]h,]h-]h.]h1]uh3K@h5]r h>XO Figure 2. The model for derived data sets with new terms highlighted in purple.r r }r (h j h j ubaubeubeubh%h&h'Usystem_messager h)}r (h+]UlevelKh.]h-]r j aUsourceh&h,]h1]UlineKUtypeUINFOr uh3K,h4hh5]r
h|)r }r (h X0 Duplicate explicit target name: "prov ontology".h)}r
(h+]h,]h-]h.]h1]uh h h5]r h>X0 Duplicate explicit target name: "prov ontology".r r }r (h U h j ubah'hubaubh)r }r (h U h j h%h&h'j h)}r (h+]UlevelKh.]h-]Usourceh&h,]h1]UlineKUtypeUWARNINGr uh3Kh4hh5]r h|)r }r (h X# Literal block expected; none found.h)}r (h+]h,]h-]h.]h1]uh j h5]r h>X# Literal block expected; none found.r r }r (h U h j ubah'hubaubh)r }r (h U h j h%h&h'j h)}r (h+]UlevelKh.]h-]Usourceh&h,]h1]UlineKUtypej uh3Kh4hh5]r! h|)r" }r# (h X# Literal block expected; none found.h)}r$ (h+]h,]h-]h.]h1]uh j h5]r% h>X# Literal block expected; none found.r& r' }r( (h U h j" ubah'hubaubeUcurrent_sourcer) NU
decorationr* NUautofootnote_startr+ KUnameidsr, }r- (hj hj hj~ h h0h
j h j h j h
j2 hj# hhhj hj hj hhhj hj hj uh5]r. (cdocutils.nodes
raw
r/ )r0 }r1 (h X \newpageh hh%h&h'Urawr2 h)}r3 (UformatX latexj j h.]h-]h+]h,]h1]uh3Kh4hh5]r4 h>X \newpager5 r6 }r7 (h U h j0 ubaubh#eh U U transformerr8 NU
footnote_refsr9 }r: Urefnamesr; }r< Usymbol_footnotesr= ]r> Uautofootnote_refsr? ]r@ Usymbol_footnote_refsrA ]rB U citationsrC ]rD h4hU current_linerE NUtransform_messagesrF ]rG (h)rH }rI (h U h)}rJ (h+]UlevelKh.]h-]Usourceh&h,]h1]UlineK Utypej uh5]rK h|)rL }rM (h U h)}rN (h+]h,]h-]h.]h1]uh jH h5]rO h>X- Hyperlink target "index-0" is not referenced.rP rQ }rR (h U h jL ubah'hubah'j ubh)rS }rT (h U h)}rU (h+]UlevelKh.]h-]Usourceh&h,]h1]UlineK Utypej uh5]rV h|)rW }rX (h U h)}rY (h+]h,]h-]h.]h1]uh jS h5]rZ h>X- Hyperlink target "index-1" is not referenced.r[ r\ }r] (h U h jW ubah'hubah'j ubeUreporterr^ NUid_startr_ KU
autofootnotesr` ]ra U
citation_refsrb }rc Uindirect_targetsrd ]re Usettingsrf (cdocutils.frontend
Values
rg orh }ri (Ufootnote_backlinksrj KUrecord_dependenciesrk NU rfc_base_urlrl U https://tools.ietf.org/html/rm U tracebackrn Upep_referencesro NUstrip_commentsrp NU
toc_backlinksrq j U
language_coderr Uenrs U datestamprt NU report_levelru KU _destinationrv NU
halt_levelrw KU
strip_classesrx Nh;NU error_encoding_error_handlerry Ubackslashreplacerz Udebugr{ NUembed_stylesheetr| U output_encoding_error_handlerr} Ustrictr~ U
sectnum_xformr KUdump_transformsr NU
docinfo_xformr KUwarning_streamr NUpep_file_url_templater Upep-%04dr Uexit_status_levelr KUconfigr NUstrict_visitorr NUcloak_email_addressesr U trim_footnote_reference_spacer Uenvr NUdump_pseudo_xmlr NUexpose_internalsr NUsectsubtitle_xformr U source_linkr NUrfc_referencesr NUoutput_encodingr Uutf-8r U
source_urlr NUinput_encodingr U utf-8-sigr U_disable_configr NU id_prefixr U U tab_widthr KUerror_encodingr UUTF-8r U_sourcer h&Ugettext_compactr U generatorr NUdump_internalsr NU smart_quotesr U pep_base_urlr U https://www.python.org/dev/peps/r Usyntax_highlightr Ulongr U input_encoding_error_handlerr j~ Uauto_id_prefixr Uidr Udoctitle_xformr Ustrip_elements_with_classesr NU
_config_filesr ]Ufile_insertion_enabledr U raw_enabledr KU
dump_settingsr NubUsymbol_footnote_startr K h.}r (hhj j j# j j j j j j j jK jF jk jg j~ jz j h!j j
j2 j. hJhNhWh^j j j j j j j j j j j j j j{ j; j7 h0h#j j hhj j uUsubstitution_namesr }r h'h4h)}r (h+]h.]h-]Usourceh&h,]h1]uU footnotesr ]r Urefidsr }r hW]r hYasub.