cdocutils.nodes document q)q}q(U nametypesq}q(Xloggingschema.htmlqX installationqNXexample queriesqNX solr indexq NX introductionq NXsystem metadata updatesq NXusagestatistics.htmlq Xlog recovery processingq NXsolr query processingqNXlog harvest processingqNXlog aggregation overviewqNuUsubstitution_defsq}qUparse_messagesq]qUcurrent_sourceqNU decorationqNUautofootnote_startqKUnameidsq}q(hUloggingschema-htmlqhU installationqhUexample-queriesqh U solr-indexqh U introductionqh Usystem-metadata-updatesqh Uusagestatistics-htmlq h Ulog-recovery-processingq!hUsolr-query-processingq"hUlog-harvest-processingq#hUlog-aggregation-overviewq$uUchildrenq%]q&(cdocutils.nodes block_quote q')q(}q)(U rawsourceq*UUparentq+hUsourceq,NUtagnameq-U block_quoteq.U attributesq/}q0(Udupnamesq1]Uclassesq2]Ubackrefsq3]Uidsq4]Unamesq5]uUlineq6NUdocumentq7hh%]q8cdocutils.nodes comment q9)q:}q;(h*X@startuml images/log-aggregation-activity.png skinparam { titleFofntColor black titleFontSize 16 titleFontStyle normal titleFontName Lucida Bright } title: Log Aggregation Overview (*) -down-> Initialization -down-> recover recent data from other CNs -down-> Register log index top listener -down-> register systemmetadata listener -down-> Loop through MNs if "More MNs to harvest" then -down->[true] "get log records from MN" -down-> "add data from systemmetadata" -down-> "add location data" -down-> send entries to Solr -right-> Loop through MNs else -->[false] ===foo=== endif ===foo=== -down-> (*) @endumlh/}q<(U xml:spaceq=Upreserveq>h4]h3]h1]h2]h5]uh+h(h%]q?cdocutils.nodes Text q@X@startuml images/log-aggregation-activity.png skinparam { titleFofntColor black titleFontSize 16 titleFontStyle normal titleFontName Lucida Bright } title: Log Aggregation Overview (*) -down-> Initialization -down-> recover recent data from other CNs -down-> Register log index top listener -down-> register systemmetadata listener -down-> Loop through MNs if "More MNs to harvest" then -down->[true] "get log records from MN" -down-> "add data from systemmetadata" -down-> "add location data" -down-> send entries to Solr -right-> Loop through MNs else -->[false] ===foo=== endif ===foo=== -down-> (*) @endumlqAqB}qC(h*Uh+h:ubah-UcommentqDubaubh9)qE}qF(h*X@startuml images/log-aggregation-sequence.png skinparam { sequenceTitleFontColor black sequenceTitleFontSize 16 sequenceTitleFontStyle normal sequenceTitleFontName Lucida Bright } title: Log Aggregation Processing LogAggregationScheduleMananger -> LogAggregationScheduleMananger : addHarvest() activate LogAggregationScheduleMananger LogAggregationScheduleMananger -> LogAggregationHarvestJob : execute() deactivate LogAggregationScheduleMananger activate LogAggregationHarvestJob LogAggregationHarvestJob -> LogAggregatorTask : call() deactivate LogAggregationHarvestJob activate LogAggregatorTask LogAggregatorTask -> LogAggregatorTask : retrieve() LogAggregatorTask -> MNode : getLogRecords() activate MNode MNode -> LogAggregatorTask : Log deactivate MNode LogAggregatorTask -> "hzLogEntryTopic : ITopic" : publish(solrItemList) deactivate LogAggregatorTask activate "hzLogEntryTopic : ITopic" deactivate MNode "hzLogEntryTopic : ITopic" -> LogEntryTopicListener : onMessage() deactivate "hzLogEntryTopic : ITopic" activate LogEntryTopicListener LogEntryTopicListener -> indexLogEntryQueue deactivate LogEntryTopicListener activate indexLogEntryQueue indexLogEntryQueue -> indexLogEntryQueue : offer() indexLogEntryQueue -> indexLogEntryQueue : poll() indexLogEntryQueue -> LogEntryQueueTask deactivate indexLogEntryQueue activate LogEntryQueueTask LogEntryQueueTask -> LogEntryIndexTask : call() deactivate LogEntryQueueTask activate LogEntryIndexTask LogEntryIndexTask -> solrServer : addbeans() deactivate LogEntryIndexTask @endumlh+hh,Xi/var/lib/jenkins/jobs/API_Documentation_trunk/workspace/api-documentation/source/design/LogAggregator.txtqGh-hDh/}qH(h=h>h4]h3]h1]h2]h5]uh6KJh7hh%]qIh@X@startuml images/log-aggregation-sequence.png skinparam { sequenceTitleFontColor black sequenceTitleFontSize 16 sequenceTitleFontStyle normal sequenceTitleFontName Lucida Bright } title: Log Aggregation Processing LogAggregationScheduleMananger -> LogAggregationScheduleMananger : addHarvest() activate LogAggregationScheduleMananger LogAggregationScheduleMananger -> LogAggregationHarvestJob : execute() deactivate LogAggregationScheduleMananger activate LogAggregationHarvestJob LogAggregationHarvestJob -> LogAggregatorTask : call() deactivate LogAggregationHarvestJob activate LogAggregatorTask LogAggregatorTask -> LogAggregatorTask : retrieve() LogAggregatorTask -> MNode : getLogRecords() activate MNode MNode -> LogAggregatorTask : Log deactivate MNode LogAggregatorTask -> "hzLogEntryTopic : ITopic" : publish(solrItemList) deactivate LogAggregatorTask activate "hzLogEntryTopic : ITopic" deactivate MNode "hzLogEntryTopic : ITopic" -> LogEntryTopicListener : onMessage() deactivate "hzLogEntryTopic : ITopic" activate LogEntryTopicListener LogEntryTopicListener -> indexLogEntryQueue deactivate LogEntryTopicListener activate indexLogEntryQueue indexLogEntryQueue -> indexLogEntryQueue : offer() indexLogEntryQueue -> indexLogEntryQueue : poll() indexLogEntryQueue -> LogEntryQueueTask deactivate indexLogEntryQueue activate LogEntryQueueTask LogEntryQueueTask -> LogEntryIndexTask : call() deactivate LogEntryQueueTask activate LogEntryIndexTask LogEntryIndexTask -> solrServer : addbeans() deactivate LogEntryIndexTask @endumlqJqK}qL(h*Uh+hEubaubh9)qM}qN(h*X2@startuml images/systemmetadata-listener-activity.png skinparam { titleFofntColor black titleFontSize 16 titleFontStyle normal titleFontName Lucida Bright } title: System Metadata Listener (*) -down-> sysmeta entry added -down-> ===process_sysmeta=== (*) -down-> sysmeta entry updated --> ===process_sysmeta=== -down-> retrieve Solr entries for updated/added pid -down-> update permission fields -down-> update location fields -down-> update spatial field -down-> publish to index processing queue @endumlh+hh,hGh-hDh/}qO(h=h>h4]h3]h1]h2]h5]uh6K^h7hh%]qPh@X2@startuml images/systemmetadata-listener-activity.png skinparam { titleFofntColor black titleFontSize 16 titleFontStyle normal titleFontName Lucida Bright } title: System Metadata Listener (*) -down-> sysmeta entry added -down-> ===process_sysmeta=== (*) -down-> sysmeta entry updated --> ===process_sysmeta=== -down-> retrieve Solr entries for updated/added pid -down-> update permission fields -down-> update location fields -down-> update spatial field -down-> publish to index processing queue @endumlqQqR}qS(h*Uh+hMubaubcdocutils.nodes section qT)qU}qV(h*Uh+hh,hGh-UsectionqWh/}qX(h1]h2]h3]h4]qYh$ah5]qZhauh6K`h7hh%]q[(cdocutils.nodes title q\)q]}q^(h*XLog Aggregation Overviewq_h+hUh,hGh-Utitleq`h/}qa(h1]h2]h3]h4]h5]uh6K`h7hh%]qbh@XLog Aggregation Overviewqcqd}qe(h*h_h+h]ubaubh9)qf}qg(h*Uh+hUh,hGh-hDh/}qh(h=h>h4]h3]h1]h2]h5]uh6Kbh7hh%]ubh')qi}qj(h*Uh+hUh,hGh-h.h/}qk(h1]h2]h3]h4]h5]uh6Nh7hh%]qlcdocutils.nodes paragraph qm)qn}qo(h*X,See subversion log for revision information.qph+hih,hGh-U paragraphqqh/}qr(h1]h2]h3]h4]h5]uh6Kdh%]qsh@X,See subversion log for revision information.qtqu}qv(h*hph+hnubaubaubhT)qw}qx(h*Uh+hUh,hGh-hWh/}qy(h1]h2]h3]h4]qzhah5]q{h auh6Kgh7hh%]q|(h\)q}}q~(h*X Introductionqh+hwh,hGh-h`h/}q(h1]h2]h3]h4]h5]uh6Kgh7hh%]qh@X Introductionqq}q(h*hh+h}ubaubhm)q}q(h*XThe Log Aggregation Facility (LAF) harvests event log information from all Member Nodes registered with a Coordinating Node and aggregates this information into a Solr index that contains information from all member nodes. The Solr index can be queried to create reports of usage statistics for the DataONE network or individual Member Nodes. A description of the data harvested from MNs is described here: ``_h+hwh,hGh-hqh/}q(h1]h2]h3]h4]h5]uh6Kih7hh%]q(h@XThe Log Aggregation Facility (LAF) harvests event log information from all Member Nodes registered with a Coordinating Node and aggregates this information into a Solr index that contains information from all member nodes. The Solr index can be queried to create reports of usage statistics for the DataONE network or individual Member Nodes. A description of the data harvested from MNs is described here: qq}q(h*XThe Log Aggregation Facility (LAF) harvests event log information from all Member Nodes registered with a Coordinating Node and aggregates this information into a Solr index that contains information from all member nodes. The Solr index can be queried to create reports of usage statistics for the DataONE network or individual Member Nodes. A description of the data harvested from MNs is described here: h+hubcdocutils.nodes reference q)q}q(h*X``_h/}q(UnameXLoggingSchema.htmlqUrefuriqhh4]h3]h1]h2]h5]uh+hh%]qh@XLoggingSchema.htmlqq}q(h*Uh+hubah-U referencequbcdocutils.nodes target q)q}q(h*XU referencedqKh+hh-Utargetqh/}q(Urefurihh4]qhah3]h1]h2]h5]qhauh%]ubeubhm)q}q(h*XLAF relies on several technologies: Quartz Scheduler, Hazelcast Data Distribution, Metacat Repository Storage, and the Solr Search platform.qh+hwh,hGh-hqh/}q(h1]h2]h3]h4]h5]uh6Knh7hh%]qh@XLAF relies on several technologies: Quartz Scheduler, Hazelcast Data Distribution, Metacat Repository Storage, and the Solr Search platform.qq}q(h*hh+hubaubhm)q}q(h*X)The DataONE use cases are described here:qh+hwh,hGh-hqh/}q(h1]h2]h3]h4]h5]uh6Krh7hh%]qh@X)The DataONE use cases are described here:qq}q(h*hh+hubaubcdocutils.nodes compound q)q}q(h*Uh+hwh,hGh-Ucompoundqh/}q(h1]h2]qUtoctree-wrapperqah3]h4]h5]uh6Nh7hh%]qcsphinx.addnodes toctree q)q}q(h*Uh+hh,hGh-Utoctreeqh/}q(UnumberedqKU includehiddenqh+Xdesign/LogAggregatorqUcaptionqNUglobqh4]h3]U titlesonlyqh1]h2]h5]Uentriesq]q(NXdesign/UseCases/16_ucqĆqNXdesign/UseCases/17_ucqƆqeUhiddenqȉU includefilesq]q(hheUmaxdepthqKuh6Kth%]ubaubhm)q}q(h*XNAn overview of LAF processing is shown in *Figure 1: Log Aggregation Overview*qh+hwh,hGh-hqh/}q(h1]h2]h3]h4]h5]uh6Kzh7hh%]q(h@X*An overview of LAF processing is shown in qхq}q(h*X*An overview of LAF processing is shown in h+hubcdocutils.nodes emphasis q)q}q(h*X$*Figure 1: Log Aggregation Overview*h/}q(h1]h2]h3]h4]h5]uh+hh%]qh@X"Figure 1: Log Aggregation Overviewqمq}q(h*Uh+hubah-Uemphasisqubeubhm)q}q(h*X$*Figure 1.* Log Aggregation Overviewqh+hwh,hGh-hqh/}q(h1]h2]h3]h4]h5]uh6K|h7hh%]q(h)q}q(h*X *Figure 1.*h/}q(h1]h2]h3]h4]h5]uh+hh%]qh@X Figure 1.q慁q}q(h*Uh+hubah-hubh@X Log Aggregation Overviewq酁q}q(h*X Log Aggregation Overviewh+hubeubcdocutils.nodes figure q)q}q(h*Uh+hwh,hGh-Ufigureqh/}q(h1]h2]h3]h4]h5]uh6Kh7hh%]qcdocutils.nodes image q)q}q(h*X0.. figure:: images/log-aggregation-activity.png h+hh,hGh-Uimageqh/}q(UuriX*design/images/log-aggregation-activity.pngqh4]h3]h1]h2]U candidatesq}qU*hsh5]uh6Kh%]ubaubeubhT)q}q(h*Uh+hUh,hGh-hWh/}q(h1]h2]h3]h4]qhah5]qhauh6Kh7hh%]q(h\)r}r(h*X Installationrh+hh,hGh-h`h/}r(h1]h2]h3]h4]h5]uh6Kh7hh%]rh@X Installationrr}r(h*jh+jubaubhm)r}r (h*X`LAF software is installed from the dataone-cn-processdaemon and dataone-cn-solr Debian packages.r h+hh,hGh-hqh/}r (h1]h2]h3]h4]h5]uh6Kh7hh%]r h@X`LAF software is installed from the dataone-cn-processdaemon and dataone-cn-solr Debian packages.r r}r(h*j h+jubaubhm)r}r(h*X0LAF is started when the Unix service /etc/init.d/d1_processing is launched. LogAggregationScheduleManager is initialized by Spring when d1_processing runs the bootstrap class org.dataone.cn.batch.daemon.SchedulerDaemon. All LAF related beans are configured in /etc/dataone/process/applicationContext.xml.rh+hh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh7hh%]rh@X0LAF is started when the Unix service /etc/init.d/d1_processing is launched. LogAggregationScheduleManager is initialized by Spring when d1_processing runs the bootstrap class org.dataone.cn.batch.daemon.SchedulerDaemon. All LAF related beans are configured in /etc/dataone/process/applicationContext.xml.rr}r(h*jh+jubaubhm)r}r(h*XLAF does not require any manual configuration before it can be run, however Metacat must be installed and configured before LAF starts.rh+hh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh7hh%]rh@XLAF does not require any manual configuration before it can be run, however Metacat must be installed and configured before LAF starts.rr}r(h*jh+jubaubhm)r }r!(h*X;Several LDAP attributes are used to control the execution of LAF, so if necessary the following LDAP entries can be manually edited to alter execution of LAF. This may be useful for testing purposes. Table 1 shows the LDAP attributes in the directory dc=dataone,dc=org,cn=urn:node: directory that LAF usesr"h+hh,hGh-hqh/}r#(h1]h2]h3]h4]h5]uh6Kh7hh%]r$h@X;Several LDAP attributes are used to control the execution of LAF, so if necessary the following LDAP entries can be manually edited to alter execution of LAF. This may be useful for testing purposes. Table 1 shows the LDAP attributes in the directory dc=dataone,dc=org,cn=urn:node: directory that LAF usesr%r&}r'(h*j"h+j ubaubhm)r(}r)(h*X*Table 1* LDAP entriesr*h+hh,hGh-hqh/}r+(h1]h2]h3]h4]h5]uh6Kh7hh%]r,(h)r-}r.(h*X *Table 1*h/}r/(h1]h2]h3]h4]h5]uh+j(h%]r0h@XTable 1r1r2}r3(h*Uh+j-ubah-hubh@X LDAP entriesr4r5}r6(h*X LDAP entriesh+j(ubeubh')r7}r8(h*Uh+hh,hGh-h.h/}r9(h1]h2]h3]h4]h5]uh6Nh7hh%]r:cdocutils.nodes table r;)r<}r=(h*Uh/}r>(h1]h2]h3]h4]h5]uh+j7h%]r?cdocutils.nodes tgroup r@)rA}rB(h*Uh/}rC(h4]h3]h1]h2]h5]UcolsKuh+j<h%]rD(cdocutils.nodes colspec rE)rF}rG(h*Uh/}rH(h4]h3]h1]h2]h5]UcolwidthKuh+jAh%]h-UcolspecrIubjE)rJ}rK(h*Uh/}rL(h4]h3]h1]h2]h5]UcolwidthK uh+jAh%]h-jIubjE)rM}rN(h*Uh/}rO(h4]h3]h1]h2]h5]UcolwidthK'uh+jAh%]h-jIubcdocutils.nodes thead rP)rQ}rR(h*Uh/}rS(h1]h2]h3]h4]h5]uh+jAh%]rTcdocutils.nodes row rU)rV}rW(h*Uh/}rX(h1]h2]h3]h4]h5]uh+jQh%]rY(cdocutils.nodes entry rZ)r[}r\(h*Uh/}r](h1]h2]h3]h4]h5]uh+jVh%]r^hm)r_}r`(h*XAttribute Namerah+j[h,hGh-hqh/}rb(h1]h2]h3]h4]h5]uh6Kh%]rch@XAttribute Namerdre}rf(h*jah+j_ubaubah-UentryrgubjZ)rh}ri(h*Uh/}rj(h1]h2]h3]h4]h5]uh+jVh%]rkhm)rl}rm(h*XValuernh+jhh,hGh-hqh/}ro(h1]h2]h3]h4]h5]uh6Kh%]rph@XValuerqrr}rs(h*jnh+jlubaubah-jgubjZ)rt}ru(h*Uh/}rv(h1]h2]h3]h4]h5]uh+jVh%]rwhm)rx}ry(h*XPurposerzh+jth,hGh-hqh/}r{(h1]h2]h3]h4]h5]uh6Kh%]r|h@XPurposer}r~}r(h*jzh+jxubaubah-jgubeh-Urowrubah-Utheadrubcdocutils.nodes tbody r)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jAh%]r(jU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xd1NodeAggregateLogsrh+jh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xd1NodeAggregateLogsrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X TRUE or FALSErh+jh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X TRUE or FALSErr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X'Enable/Disable this node for harvestingrh+jh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X'Enable/Disable this node for harvestingrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xd1NodeLogLastAggregatedrh+jh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xd1NodeLogLastAggregatedrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X ISO 8601 daterh+jh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X ISO 8601 daterr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*XLast time this MN was harvestedrh+jh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@XLast time this MN was harvestedrr}r(h*jh+jubaubah-jgubeh-jubeh-Utbodyrubeh-Utgrouprubah-UtablerubaubeubhT)r}r(h*Uh+hUh,hGh-hWh/}r(h1]h2]h3]h4]rh!ah5]rh auh6Kh7hh%]r(h\)r}r(h*XLog Recovery Processingrh+jh,hGh-h`h/}r(h1]h2]h3]h4]h5]uh6Kh7hh%]rh@XLog Recovery Processingrr}r(h*jh+jubaubhm)r}r(h*XThe Java class LogAggregationScheduleManager coordinates all scheduling of jobs for LAF. This class first checks if any other CN has more current log information and if so a recovery job is scheduled to query the other CNs for newer log records.rh+jh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh7hh%]rh@XThe Java class LogAggregationScheduleManager coordinates all scheduling of jobs for LAF. This class first checks if any other CN has more current log information and if so a recovery job is scheduled to query the other CNs for newer log records.rr}r(h*jh+jubaubeubhT)r}r(h*Uh+hUh,hGh-hWh/}r(h1]h2]h3]h4]rh#ah5]rhauh6Kh7hh%]r(h\)r}r(h*XLog Harvest Processingrh+jh,hGh-h`h/}r(h1]h2]h3]h4]h5]uh6Kh7hh%]rh@XLog Harvest Processingrr}r(h*jh+jubaubhm)r}r(h*XThen LogAggregationScheduleManager schedules recurring Quartz job (every 24 hours) for LogAggregationHarvestJob which handles running the tasks that harvest log data from the MNs.rh+jh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh7hh%]rh@XThen LogAggregationScheduleManager schedules recurring Quartz job (every 24 hours) for LogAggregationHarvestJob which handles running the tasks that harvest log data from the MNs.rr}r(h*jh+jubaubhm)r}r(h*XNext a Hazelcast listener is registered for the top thehzLogEntryTopic. When log entries are returned from MNs they will posted to Hazelcast so that they are distributed to every CN.rh+jh,hGh-hqh/}r (h1]h2]h3]h4]h5]uh6Kh7hh%]r h@XNext a Hazelcast listener is registered for the top thehzLogEntryTopic. When log entries are returned from MNs they will posted to Hazelcast so that they are distributed to every CN.r r }r (h*jh+jubaubhm)r}r(h*XA Hazelcast systemmetadata listener is also registered so that certain changes to mutable systemmetata fields (see System Metadata Updates for details).rh+jh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh7hh%]rh@XA Hazelcast systemmetadata listener is also registered so that certain changes to mutable systemmetata fields (see System Metadata Updates for details).rr}r(h*jh+jubaubhm)r}r(h*XThe queue indexLogEntryQueue is then setup when the listener LogEntryTopicListener is called. This queue is how tasks that perform the log aggregation and tasks that perform the log indexing communicate.rh+jh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh7hh%]rh@XThe queue indexLogEntryQueue is then setup when the listener LogEntryTopicListener is called. This queue is how tasks that perform the log aggregation and tasks that perform the log indexing communicate.rr}r(h*jh+jubaubhm)r}r(h*XLogAggregationHarvestJob runs the task LogAggregatorTask that queries each MN by calling mn.getLogRecords and retrieving up to 1000 event log records at a time, starting after the last date retrieved that was recorded by the last run, if any.r h+jh,hGh-hqh/}r!(h1]h2]h3]h4]h5]uh6Kh7hh%]r"h@XLogAggregationHarvestJob runs the task LogAggregatorTask that queries each MN by calling mn.getLogRecords and retrieving up to 1000 event log records at a time, starting after the last date retrieved that was recorded by the last run, if any.r#r$}r%(h*j h+jubaubhm)r&}r'(h*X,LogAggregatorTask then modifies each harvested record by adding the following fields: isPublic, dateAggregated, nodeId, readPermissions, formatId, formatType, size, rightsHolder, country, region, city, geohash_1 - geohash_9, location. These records are then published to the Hazelcast topic hzLogEntryTopic so that each CN that is running LAF will have it's LogEntryTopicListener fire, which published them to the queue indexLogEntryQueue. In this way one CN harvests a single MN at a time, and the processing of those records is synced with the other CNs.r(h+jh,hGh-hqh/}r)(h1]h2]h3]h4]h5]uh6Kh7hh%]r*h@X,LogAggregatorTask then modifies each harvested record by adding the following fields: isPublic, dateAggregated, nodeId, readPermissions, formatId, formatType, size, rightsHolder, country, region, city, geohash_1 - geohash_9, location. These records are then published to the Hazelcast topic hzLogEntryTopic so that each CN that is running LAF will have it's LogEntryTopicListener fire, which published them to the queue indexLogEntryQueue. In this way one CN harvests a single MN at a time, and the processing of those records is synced with the other CNs.r+r,}r-(h*j(h+j&ubaubhm)r.}r/(h*XOn the indexing side of processing, LogEntryQueuManager is initialized by Spring when d1_processing init script is run executes the task LogEntryQueueTask that reads and processes entries from the logEntrySolrItemList when this queue has accumulated 100 entries.r0h+jh,hGh-hqh/}r1(h1]h2]h3]h4]h5]uh6Kh7hh%]r2h@XOn the indexing side of processing, LogEntryQueuManager is initialized by Spring when d1_processing init script is run executes the task LogEntryQueueTask that reads and processes entries from the logEntrySolrItemList when this queue has accumulated 100 entries.r3r4}r5(h*j0h+j.ubaubhm)r6}r7(h*X^LogEntryQueueTask then starts task LogEntryIndexTask which sends entries to Solr for indexing.r8h+jh,hGh-hqh/}r9(h1]h2]h3]h4]h5]uh6Kh7hh%]r:h@X^LogEntryQueueTask then starts task LogEntryIndexTask which sends entries to Solr for indexing.r;r<}r=(h*j8h+j6ubaubhm)r>}r?(h*X&*Figure 2.* Log Aggregation Processingr@h+jh,hGh-hqh/}rA(h1]h2]h3]h4]h5]uh6Kh7hh%]rB(h)rC}rD(h*X *Figure 2.*h/}rE(h1]h2]h3]h4]h5]uh+j>h%]rFh@X Figure 2.rGrH}rI(h*Uh+jCubah-hubh@X Log Aggregation ProcessingrJrK}rL(h*X Log Aggregation Processingh+j>ubeubh)rM}rN(h*Uh+jh,hGh-hh/}rO(h1]h2]h3]h4]h5]uh6Kh7hh%]rPh)rQ}rR(h*X0.. figure:: images/log-aggregation-sequence.png h+jMh,hGh-hh/}rS(UuriX*design/images/log-aggregation-sequence.pngrTh4]h3]h1]h2]h}rUU*jTsh5]uh6Kh%]ubaubeubhT)rV}rW(h*Uh+hUh,hGh-hWh/}rX(h1]h2]h3]h4]rYhah5]rZh auh6Kh7hh%]r[(h\)r\}r](h*XSystem Metadata Updatesr^h+jVh,hGh-h`h/}r_(h1]h2]h3]h4]h5]uh6Kh7hh%]r`h@XSystem Metadata Updatesrarb}rc(h*j^h+j\ubaubhm)rd}re(h*XFIn addition to processing harvested records, when an entry in the Hazelcast Systemmetadata map is added or updated, SystemMetadataEntryListener runs. This may happen when one of the mutable fields in systemmetadata changes, such as formatId. In this event, all event log records for that pid must be updated, so SystemMetadataEntryListener retrieves these records, updates them with the current information in systemmetadata and then updates the entries and then publishes them to the IndexLogEntrySolrItem queue, so that they are processed in the same way as new Event Log records.rfh+jVh,hGh-hqh/}rg(h1]h2]h3]h4]h5]uh6Kh7hh%]rhh@XFIn addition to processing harvested records, when an entry in the Hazelcast Systemmetadata map is added or updated, SystemMetadataEntryListener runs. This may happen when one of the mutable fields in systemmetadata changes, such as formatId. In this event, all event log records for that pid must be updated, so SystemMetadataEntryListener retrieves these records, updates them with the current information in systemmetadata and then updates the entries and then publishes them to the IndexLogEntrySolrItem queue, so that they are processed in the same way as new Event Log records.rirj}rk(h*jfh+jdubaubhm)rl}rm(h*X$*Figure 3.* System Metadata Listenerrnh+jVh,hGh-hqh/}ro(h1]h2]h3]h4]h5]uh6Kh7hh%]rp(h)rq}rr(h*X *Figure 3.*h/}rs(h1]h2]h3]h4]h5]uh+jlh%]rth@X Figure 3.rurv}rw(h*Uh+jqubah-hubh@X System Metadata Listenerrxry}rz(h*X System Metadata Listenerh+jlubeubh)r{}r|(h*Uh+jVh,hGh-hh/}r}(h1]h2]h3]h4]h5]uh6Kh7hh%]r~h)r}r(h*X8.. figure:: images/systemmetadata-listener-activity.png h+j{h,hGh-hh/}r(UuriX2design/images/systemmetadata-listener-activity.pngrh4]h3]h1]h2]h}rU*jsh5]uh6Kh%]ubaubeubhT)r}r(h*Uh+hUh,hGh-hWh/}r(h1]h2]h3]h4]rhah5]rh auh6Kh7hh%]r(h\)r}r(h*X Solr Indexrh+jh,hGh-h`h/}r(h1]h2]h3]h4]h5]uh6Kh7hh%]rh@X Solr Indexrr}r(h*jh+jubaubhm)r}r(h*X>Table 2 shows the fields contained in the Event Log Solr indexrh+jh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh7hh%]rh@X>Table 2 shows the fields contained in the Event Log Solr indexrr}r(h*jh+jubaubhm)r}r(h*X*Table 2.* Solr index schemarh+jh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh7hh%]r(h)r}r(h*X *Table 2.*h/}r(h1]h2]h3]h4]h5]uh+jh%]rh@XTable 2.rr}r(h*Uh+jubah-hubh@X Solr index schemarr}r(h*X Solr index schemah+jubeubj;)r}r(h*Uh+jh,cdocutils.nodes reprunicode rX%source/design/EventLogIndexSchema.txtrr}rbh-jh/}r(h1]h2]h3]h4]rUid1rah5]uh6Nh7hh%]r(h\)r}r(h*XSolr index schemarh/}r(h1]h2]h3]h4]h5]uh+jh%]rh@XSolr index schemarr}r(h*jh+jubah-h`ubj@)r}r(h*Uh/}r(h4]h3]h1]h2]h5]UcolsKuh+jh%]r(jE)r}r(h*Uh/}r(h4]h3]h1]h2]h5]UcolwidthKuh+jh%]h-jIubjE)r}r(h*Uh/}r(h4]h3]h1]h2]h5]UcolwidthK uh+jh%]h-jIubjE)r}r(h*Uh/}r(h4]h3]h1]h2]h5]UcolwidthKOuh+jh%]h-jIubjP)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*XNamerh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@XNamerr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X Solr Typerh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X Solr Typerr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*XCommentrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@XCommentrr}r(h*jh+jubaubah-jgubeh-jubah-jubj)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xidrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xidrr}r(h*jh+jubaubah-jgubjZ)r}r (h*Uh/}r (h1]h2]h3]h4]h5]uh+jh%]r hm)r }r (h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xstringrr}r(h*jh+j ubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xadded after harvestrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xadded after harvestrr}r(h*jh+jubaubah-jgubeh-jubjU)r }r!(h*Uh/}r"(h1]h2]h3]h4]h5]uh+jh%]r#(jZ)r$}r%(h*Uh/}r&(h1]h2]h3]h4]h5]uh+j h%]r'hm)r(}r)(h*XdateAggregatedr*h+j$h,jh-hqh/}r+(h1]h2]h3]h4]h5]uh6Kh%]r,h@XdateAggregatedr-r.}r/(h*j*h+j(ubaubah-jgubjZ)r0}r1(h*Uh/}r2(h1]h2]h3]h4]h5]uh+j h%]r3hm)r4}r5(h*Xdater6h+j0h,jh-hqh/}r7(h1]h2]h3]h4]h5]uh6Kh%]r8h@Xdater9r:}r;(h*j6h+j4ubaubah-jgubjZ)r<}r=(h*Uh/}r>(h1]h2]h3]h4]h5]uh+j h%]r?hm)r@}rA(h*Xadded after harvestrBh+j<h,jh-hqh/}rC(h1]h2]h3]h4]h5]uh6Kh%]rDh@Xadded after harvestrErF}rG(h*jBh+j@ubaubah-jgubeh-jubjU)rH}rI(h*Uh/}rJ(h1]h2]h3]h4]h5]uh+jh%]rK(jZ)rL}rM(h*Uh/}rN(h1]h2]h3]h4]h5]uh+jHh%]rOhm)rP}rQ(h*XisPublicrRh+jLh,jh-hqh/}rS(h1]h2]h3]h4]h5]uh6Kh%]rTh@XisPublicrUrV}rW(h*jRh+jPubaubah-jgubjZ)rX}rY(h*Uh/}rZ(h1]h2]h3]h4]h5]uh+jHh%]r[hm)r\}r](h*Xbooleanr^h+jXh,jh-hqh/}r_(h1]h2]h3]h4]h5]uh6Kh%]r`h@Xbooleanrarb}rc(h*j^h+j\ubaubah-jgubjZ)rd}re(h*Uh/}rf(h1]h2]h3]h4]h5]uh+jHh%]rghm)rh}ri(h*X1added after harvest, obtained from systemmetadatarjh+jdh,jh-hqh/}rk(h1]h2]h3]h4]h5]uh6Kh%]rlh@X1added after harvest, obtained from systemmetadatarmrn}ro(h*jjh+jhubaubah-jgubeh-jubjU)rp}rq(h*Uh/}rr(h1]h2]h3]h4]h5]uh+jh%]rs(jZ)rt}ru(h*Uh/}rv(h1]h2]h3]h4]h5]uh+jph%]rwhm)rx}ry(h*XreadPermissionrzh+jth,jh-hqh/}r{(h1]h2]h3]h4]h5]uh6K h%]r|h@XreadPermissionr}r~}r(h*jzh+jxubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jph%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K h%]rh@Xstringrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jph%]rhm)r}r(h*XHadded after harvest, obtained from systemmetadata, filtered during queryrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K h%]rh@XHadded after harvest, obtained from systemmetadata, filtered during queryrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*XentryIdrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K h%]rh@XentryIdrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K h%]rh@Xstringrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xobtained from MN event logrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K h%]rh@Xobtained from MN event logrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xpidrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K h%]rh@Xpidrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K h%]rh@Xstringrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X1added after harvest, obtained from systemmetadatarh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K h%]rh@X1added after harvest, obtained from systemmetadatarr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X ipAddressrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K h%]rh@X ipAddressrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K h%]rh@Xstringrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r (h*X1obtained from MN event log, filtered during queryr h+jh,jh-hqh/}r (h1]h2]h3]h4]h5]uh6K h%]r h@X1obtained from MN event log, filtered during queryr r}r(h*j h+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X userAgentrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K h%]rh@X userAgentrr}r(h*jh+jubaubah-jgubjZ)r }r!(h*Uh/}r"(h1]h2]h3]h4]h5]uh+jh%]r#hm)r$}r%(h*Xstringr&h+j h,jh-hqh/}r'(h1]h2]h3]h4]h5]uh6K h%]r(h@Xstringr)r*}r+(h*j&h+j$ubaubah-jgubjZ)r,}r-(h*Uh/}r.(h1]h2]h3]h4]h5]uh+jh%]r/hm)r0}r1(h*Xobtained from MN event logr2h+j,h,jh-hqh/}r3(h1]h2]h3]h4]h5]uh6K h%]r4h@Xobtained from MN event logr5r6}r7(h*j2h+j0ubaubah-jgubeh-jubjU)r8}r9(h*Uh/}r:(h1]h2]h3]h4]h5]uh+jh%]r;(jZ)r<}r=(h*Uh/}r>(h1]h2]h3]h4]h5]uh+j8h%]r?hm)r@}rA(h*XsubjectrBh+j<h,jh-hqh/}rC(h1]h2]h3]h4]h5]uh6Kh%]rDh@XsubjectrErF}rG(h*jBh+j@ubaubah-jgubjZ)rH}rI(h*Uh/}rJ(h1]h2]h3]h4]h5]uh+j8h%]rKhm)rL}rM(h*XstringrNh+jHh,jh-hqh/}rO(h1]h2]h3]h4]h5]uh6Kh%]rPh@XstringrQrR}rS(h*jNh+jLubaubah-jgubjZ)rT}rU(h*Uh/}rV(h1]h2]h3]h4]h5]uh+j8h%]rWhm)rX}rY(h*X1obtained from MN event log, filtered during queryrZh+jTh,jh-hqh/}r[(h1]h2]h3]h4]h5]uh6Kh%]r\h@X1obtained from MN event log, filtered during queryr]r^}r_(h*jZh+jXubaubah-jgubeh-jubjU)r`}ra(h*Uh/}rb(h1]h2]h3]h4]h5]uh+jh%]rc(jZ)rd}re(h*Uh/}rf(h1]h2]h3]h4]h5]uh+j`h%]rghm)rh}ri(h*Xeventrjh+jdh,jh-hqh/}rk(h1]h2]h3]h4]h5]uh6Kh%]rlh@Xeventrmrn}ro(h*jjh+jhubaubah-jgubjZ)rp}rq(h*Uh/}rr(h1]h2]h3]h4]h5]uh+j`h%]rshm)rt}ru(h*Xstringrvh+jph,jh-hqh/}rw(h1]h2]h3]h4]h5]uh6Kh%]rxh@Xstringryrz}r{(h*jvh+jtubaubah-jgubjZ)r|}r}(h*Uh/}r~(h1]h2]h3]h4]h5]uh+j`h%]rhm)r}r(h*Xobtained from MN event logrh+j|h,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xobtained from MN event logrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X dateLoggedrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X dateLoggedrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xdaterh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xdaterr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xobtained from MN event logrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xobtained from MN event logrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*XnodeIdrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@XnodeIdrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xstringrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xobtained from MN event logrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xobtained from MN event logrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X rightsHolderrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X rightsHolderrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xstringrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*XHadded after harvest, obtained from systemmetadata, filtered during queryrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@XHadded after harvest, obtained from systemmetadata, filtered during queryrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r (h*XformatIdr h+jh,jh-hqh/}r (h1]h2]h3]h4]h5]uh6Kh%]r h@XformatIdr r}r(h*j h+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xstringrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r }r!(h*X1added after harvest, obtained from systemmetadatar"h+jh,jh-hqh/}r#(h1]h2]h3]h4]h5]uh6Kh%]r$h@X1added after harvest, obtained from systemmetadatar%r&}r'(h*j"h+j ubaubah-jgubeh-jubjU)r(}r)(h*Uh/}r*(h1]h2]h3]h4]h5]uh+jh%]r+(jZ)r,}r-(h*Uh/}r.(h1]h2]h3]h4]h5]uh+j(h%]r/hm)r0}r1(h*X formatTyper2h+j,h,jh-hqh/}r3(h1]h2]h3]h4]h5]uh6Kh%]r4h@X formatTyper5r6}r7(h*j2h+j0ubaubah-jgubjZ)r8}r9(h*Uh/}r:(h1]h2]h3]h4]h5]uh+j(h%]r;hm)r<}r=(h*Xstringr>h+j8h,jh-hqh/}r?(h1]h2]h3]h4]h5]uh6Kh%]r@h@XstringrArB}rC(h*j>h+j<ubaubah-jgubjZ)rD}rE(h*Uh/}rF(h1]h2]h3]h4]h5]uh+j(h%]rGhm)rH}rI(h*X1added after harvest, obtained from systemmetadatarJh+jDh,jh-hqh/}rK(h1]h2]h3]h4]h5]uh6Kh%]rLh@X1added after harvest, obtained from systemmetadatarMrN}rO(h*jJh+jHubaubah-jgubeh-jubjU)rP}rQ(h*Uh/}rR(h1]h2]h3]h4]h5]uh+jh%]rS(jZ)rT}rU(h*Uh/}rV(h1]h2]h3]h4]h5]uh+jPh%]rWhm)rX}rY(h*XsizerZh+jTh,jh-hqh/}r[(h1]h2]h3]h4]h5]uh6Kh%]r\h@Xsizer]r^}r_(h*jZh+jXubaubah-jgubjZ)r`}ra(h*Uh/}rb(h1]h2]h3]h4]h5]uh+jPh%]rchm)rd}re(h*Xslongrfh+j`h,jh-hqh/}rg(h1]h2]h3]h4]h5]uh6Kh%]rhh@Xslongrirj}rk(h*jfh+jdubaubah-jgubjZ)rl}rm(h*Uh/}rn(h1]h2]h3]h4]h5]uh+jPh%]rohm)rp}rq(h*X1added after harvest, obtained from systemmetadatarrh+jlh,jh-hqh/}rs(h1]h2]h3]h4]h5]uh6Kh%]rth@X1added after harvest, obtained from systemmetadatarurv}rw(h*jrh+jpubaubah-jgubeh-jubjU)rx}ry(h*Uh/}rz(h1]h2]h3]h4]h5]uh+jh%]r{(jZ)r|}r}(h*Uh/}r~(h1]h2]h3]h4]h5]uh+jxh%]rhm)r}r(h*Xcountryrh+j|h,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xcountryrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jxh%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xstringrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jxh%]rhm)r}r(h*X.added after harvest, determined from ipAddressrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X.added after harvest, determined from ipAddressrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xregionrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xregionrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xstringrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X.added after harvest, determined from ipAddressrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X.added after harvest, determined from ipAddressrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xcityrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xcityrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xstringrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X.added after harvest, determined from ipAddressrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X.added after harvest, determined from ipAddressrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X geohash_1rh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X geohash_1rr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xstringr r }r (h*jh+jubaubah-jgubjZ)r }r (h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X.added after harvest, determined from ipAddressrh+j h,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X.added after harvest, determined from ipAddressrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r }r!(h*X geohash_2r"h+jh,jh-hqh/}r#(h1]h2]h3]h4]h5]uh6Kh%]r$h@X geohash_2r%r&}r'(h*j"h+j ubaubah-jgubjZ)r(}r)(h*Uh/}r*(h1]h2]h3]h4]h5]uh+jh%]r+hm)r,}r-(h*Xstringr.h+j(h,jh-hqh/}r/(h1]h2]h3]h4]h5]uh6Kh%]r0h@Xstringr1r2}r3(h*j.h+j,ubaubah-jgubjZ)r4}r5(h*Uh/}r6(h1]h2]h3]h4]h5]uh+jh%]r7hm)r8}r9(h*X.added after harvest, determined from ipAddressr:h+j4h,jh-hqh/}r;(h1]h2]h3]h4]h5]uh6Kh%]r<h@X.added after harvest, determined from ipAddressr=r>}r?(h*j:h+j8ubaubah-jgubeh-jubjU)r@}rA(h*Uh/}rB(h1]h2]h3]h4]h5]uh+jh%]rC(jZ)rD}rE(h*Uh/}rF(h1]h2]h3]h4]h5]uh+j@h%]rGhm)rH}rI(h*X geohash_3rJh+jDh,jh-hqh/}rK(h1]h2]h3]h4]h5]uh6Kh%]rLh@X geohash_3rMrN}rO(h*jJh+jHubaubah-jgubjZ)rP}rQ(h*Uh/}rR(h1]h2]h3]h4]h5]uh+j@h%]rShm)rT}rU(h*XstringrVh+jPh,jh-hqh/}rW(h1]h2]h3]h4]h5]uh6Kh%]rXh@XstringrYrZ}r[(h*jVh+jTubaubah-jgubjZ)r\}r](h*Uh/}r^(h1]h2]h3]h4]h5]uh+j@h%]r_hm)r`}ra(h*X.added after harvest, determined from ipAddressrbh+j\h,jh-hqh/}rc(h1]h2]h3]h4]h5]uh6Kh%]rdh@X.added after harvest, determined from ipAddressrerf}rg(h*jbh+j`ubaubah-jgubeh-jubjU)rh}ri(h*Uh/}rj(h1]h2]h3]h4]h5]uh+jh%]rk(jZ)rl}rm(h*Uh/}rn(h1]h2]h3]h4]h5]uh+jhh%]rohm)rp}rq(h*X geohash_4rrh+jlh,jh-hqh/}rs(h1]h2]h3]h4]h5]uh6Kh%]rth@X geohash_4rurv}rw(h*jrh+jpubaubah-jgubjZ)rx}ry(h*Uh/}rz(h1]h2]h3]h4]h5]uh+jhh%]r{hm)r|}r}(h*Xstringr~h+jxh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xstringrr}r(h*j~h+j|ubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jhh%]rhm)r}r(h*X.added after harvest, determined from ipAddressrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X.added after harvest, determined from ipAddressrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X geohash_5rh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X geohash_5rr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xstringrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X.added after harvest, determined from ipAddressrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X.added after harvest, determined from ipAddressrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X geohash_6rh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X geohash_6rr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xstringrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X.added after harvest, determined from ipAddressrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X.added after harvest, determined from ipAddressrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X geohash_7rh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X geohash_7rr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@Xstringrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X.added after harvest, determined from ipAddressrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh%]rh@X.added after harvest, determined from ipAddressrr}r(h*jh+jubaubah-jgubeh-jubjU)r}r (h*Uh/}r (h1]h2]h3]h4]h5]uh+jh%]r (jZ)r }r (h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X geohash_8rh+j h,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K h%]rh@X geohash_8rr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xstringrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K h%]r h@Xstringr!r"}r#(h*jh+jubaubah-jgubjZ)r$}r%(h*Uh/}r&(h1]h2]h3]h4]h5]uh+jh%]r'hm)r(}r)(h*X.added after harvest, determined from ipAddressr*h+j$h,jh-hqh/}r+(h1]h2]h3]h4]h5]uh6K h%]r,h@X.added after harvest, determined from ipAddressr-r.}r/(h*j*h+j(ubaubah-jgubeh-jubjU)r0}r1(h*Uh/}r2(h1]h2]h3]h4]h5]uh+jh%]r3(jZ)r4}r5(h*Uh/}r6(h1]h2]h3]h4]h5]uh+j0h%]r7hm)r8}r9(h*X geohash_9r:h+j4h,jh-hqh/}r;(h1]h2]h3]h4]h5]uh6K!h%]r<h@X geohash_9r=r>}r?(h*j:h+j8ubaubah-jgubjZ)r@}rA(h*Uh/}rB(h1]h2]h3]h4]h5]uh+j0h%]rChm)rD}rE(h*XstringrFh+j@h,jh-hqh/}rG(h1]h2]h3]h4]h5]uh6K!h%]rHh@XstringrIrJ}rK(h*jFh+jDubaubah-jgubjZ)rL}rM(h*Uh/}rN(h1]h2]h3]h4]h5]uh+j0h%]rOhm)rP}rQ(h*X.added after harvest, determined from ipAddressrRh+jLh,jh-hqh/}rS(h1]h2]h3]h4]h5]uh6K!h%]rTh@X.added after harvest, determined from ipAddressrUrV}rW(h*jRh+jPubaubah-jgubeh-jubjU)rX}rY(h*Uh/}rZ(h1]h2]h3]h4]h5]uh+jh%]r[(jZ)r\}r](h*Uh/}r^(h1]h2]h3]h4]h5]uh+jXh%]r_hm)r`}ra(h*Xlocationrbh+j\h,jh-hqh/}rc(h1]h2]h3]h4]h5]uh6K"h%]rdh@Xlocationrerf}rg(h*jbh+j`ubaubah-jgubjZ)rh}ri(h*Uh/}rj(h1]h2]h3]h4]h5]uh+jXh%]rkhm)rl}rm(h*Xlocationrnh+jhh,jh-hqh/}ro(h1]h2]h3]h4]h5]uh6K"h%]rph@Xlocationrqrr}rs(h*jnh+jlubaubah-jgubjZ)rt}ru(h*Uh/}rv(h1]h2]h3]h4]h5]uh+jXh%]rwhm)rx}ry(h*X.added after harvest, determined from ipAddressrzh+jth,jh-hqh/}r{(h1]h2]h3]h4]h5]uh6K"h%]r|h@X.added after harvest, determined from ipAddressr}r~}r(h*jzh+jxubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*XinFullRobotListrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K#h%]rh@XinFullRobotListrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xbooleanrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K#h%]rh@Xbooleanrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*XNadded after harvest, determined based on log processing for COUNTER compliancerh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K#h%]rh@XNadded after harvest, determined based on log processing for COUNTER compliancerr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*XinPartialRobotListrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K$h%]rh@XinPartialRobotListrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xbooleanrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K$h%]rh@Xbooleanrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*XNadded after harvest, determined based on log processing for COUNTER compliancerh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K$h%]rh@XNadded after harvest, determined based on log processing for COUNTER compliancerr}r(h*jh+jubaubah-jgubeh-jubjU)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]r(jZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*X isRepeatVisitrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K%h%]rh@X isRepeatVisitrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*Xbooleanrh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K%h%]rh@Xbooleanrr}r(h*jh+jubaubah-jgubjZ)r}r(h*Uh/}r(h1]h2]h3]h4]h5]uh+jh%]rhm)r}r(h*XNadded after harvest, determined based on log processing for COUNTER compliancerh+jh,jh-hqh/}r(h1]h2]h3]h4]h5]uh6K%h%]rh@XNadded after harvest, determined based on log processing for COUNTER compliancerr}r(h*jh+jubaubah-jgubeh-jubeh-jubeh-jubeubeubhT)r}r(h*Uh+hUh,hGh-hWh/}r(h1]h2]h3]h4]rh"ah5]rhauh6Kh7hh%]r(h\)r}r(h*XSolr Query Processingrh+jh,hGh-h`h/}r(h1]h2]h3]h4]h5]uh6Kh7hh%]rh@XSolr Query Processingrr}r(h*jh+jubaubhm)r}r(h*XThe Event Log Solr index can be queried from the service endpoint https://cn.dataone.org/cn/v1/query/logsolr, for example, the following query will return read counts for each node in the network:h+jh,hGh-hqh/}r(h1]h2]h3]h4]h5]uh6Kh7hh%]r (h@XBThe Event Log Solr index can be queried from the service endpoint r r }r (h*XBThe Event Log Solr index can be queried from the service endpoint h+jubh)r }r(h*X*https://cn.dataone.org/cn/v1/query/logsolrrh/}r(Urefurijh4]h3]h1]h2]h5]uh+jh%]rh@X*https://cn.dataone.org/cn/v1/query/logsolrrr}r(h*Uh+j ubah-hubh@XX, for example, the following query will return read counts for each node in the network:rr}r(h*XX, for example, the following query will return read counts for each node in the network:h+jubeubcdocutils.nodes literal_block r)r}r(h*X\https://cn.dataone.org/cn/v1/query/logsolr/select?q=event:read&facet=true&facet.field=nodeIdh+jh,hGh-U literal_blockrh/}r(h=h>h4]h3]h1]h2]h5]uh6Mh7hh%]rh@X\https://cn.dataone.org/cn/v1/query/logsolr/select?q=event:read&facet=true&facet.field=nodeIdrr}r (h*Uh+jubaubhm)r!}r"(h*XThe Event Log Solr index requires authenticated access, because some fields in the log entries contain sensitive information, as shown in *Table 2*.h+jh,hGh-hqh/}r#(h1]h2]h3]h4]h5]uh6Kh7hh%]r$(h@XThe Event Log Solr index requires authenticated access, because some fields in the log entries contain sensitive information, as shown in r%r&}r'(h*XThe Event Log Solr index requires authenticated access, because some fields in the log entries contain sensitive information, as shown in h+j!ubh)r(}r)(h*X *Table 2*h/}r*(h1]h2]h3]h4]h5]uh+j!h%]r+h@XTable 2r,r-}r.(h*Uh+j(ubah-hubh@X.r/}r0(h*X.h+j!ubeubhm)r1}r2(h*X Solr queries are inspected and rewritten by SolrLoggingHandler such that counts for Solr entries will be included only if the entries are publicly accessible, or the user is a CN administrator or the caller's identify has access privileges to the pids of the entries.r3h+jh,hGh-hqh/}r4(h1]h2]h3]h4]h5]uh6Kh7hh%]r5h@X Solr queries are inspected and rewritten by SolrLoggingHandler such that counts for Solr entries will be included only if the entries are publicly accessible, or the user is a CN administrator or the caller's identify has access privileges to the pids of the entries.r6r7}r8(h*j3h+j1ubaubhm)r9}r:(h*X*If the requesting session has a certificate then a list of authorized subjects is obtained from LDAP and access is based on the authorized subjects. Since the Solr index contains access information from systemmetadata, the same access rules that apply to systemmetadata will apply to the event log.r;h+jh,hGh-hqh/}r<(h1]h2]h3]h4]h5]uh6Kh7hh%]r=h@X*If the requesting session has a certificate then a list of authorized subjects is obtained from LDAP and access is based on the authorized subjects. Since the Solr index contains access information from systemmetadata, the same access rules that apply to systemmetadata will apply to the event log.r>r?}r@(h*j;h+j9ubaubhm)rA}rB(h*XIf the authenticated user is the CN administrator then the entire contents of each SOLR document are available in the SOLR result.rCh+jh,hGh-hqh/}rD(h1]h2]h3]h4]h5]uh6Kh7hh%]rEh@XIf the authenticated user is the CN administrator then the entire contents of each SOLR document are available in the SOLR result.rFrG}rH(h*jCh+jAubaubeubhT)rI}rJ(h*Uh+hUh,hGh-hWh/}rK(h1]h2]h3]h4]rLhah5]rMhauh6Kh7hh%]rN(h\)rO}rP(h*XExample QueriesrQh+jIh,hGh-h`h/}rR(h1]h2]h3]h4]h5]uh6Kh7hh%]rSh@XExample QueriesrTrU}rV(h*jQh+jOubaubhm)rW}rX(h*XjA description of how to query the Event Log Solr Index, please see the document ``_.rYh+jIh,hGh-hqh/}rZ(h1]h2]h3]h4]h5]uh6Kh7hh%]r[(h@XPA description of how to query the Event Log Solr Index, please see the document r\r]}r^(h*XPA description of how to query the Event Log Solr Index, please see the document h+jWubh)r_}r`(h*X``_h/}ra(UnameXUsageStatistics.htmlrbhjbh4]h3]h1]h2]h5]uh+jWh%]rch@XUsageStatistics.htmlrdre}rf(h*Uh+j_ubah-hubh)rg}rh(h*XhKh+jWh-hh/}ri(Urefurijbh4]rjh ah3]h1]h2]h5]rkh auh%]ubh@X.rl}rm(h*X.h+jWubeubeubeubeh*UU transformerrnNU footnote_refsro}rpUrefnamesrq}rrUsymbol_footnotesrs]rtUautofootnote_refsru]rvUsymbol_footnote_refsrw]rxU citationsry]rzh7hU current_liner{NUtransform_messagesr|]r}Ureporterr~NUid_startrKU autofootnotesr]rU citation_refsr}rUindirect_targetsr]rUsettingsr(cdocutils.frontend Values ror}r(Ufootnote_backlinksrKUrecord_dependenciesrNU rfc_base_urlrUhttps://tools.ietf.org/html/rU tracebackrUpep_referencesrNUstrip_commentsrNU toc_backlinksrjgU language_coderUenrU datestamprNU report_levelrKU _destinationrNU halt_levelrKU strip_classesrNh`NUerror_encoding_error_handlerrUbackslashreplacerUdebugrNUembed_stylesheetrUoutput_encoding_error_handlerrUstrictrU sectnum_xformrKUdump_transformsrNU docinfo_xformrKUwarning_streamrNUpep_file_url_templaterUpep-%04drUexit_status_levelrKUconfigrNUstrict_visitorrNUcloak_email_addressesrUtrim_footnote_reference_spacerUenvrNUdump_pseudo_xmlrNUexpose_internalsrNUsectsubtitle_xformrU source_linkrNUrfc_referencesrNUoutput_encodingrUutf-8rU source_urlrNUinput_encodingrU utf-8-sigrU_disable_configrNU id_prefixrUU tab_widthrKUerror_encodingrUUTF-8rU_sourcerhGUgettext_compactrU generatorrNUdump_internalsrNU smart_quotesrU pep_base_urlrU https://www.python.org/dev/peps/rUsyntax_highlightrUlongrUinput_encoding_error_handlerrjUauto_id_prefixrUidrUdoctitle_xformrUstrip_elements_with_classesrNU _config_filesr]Ufile_insertion_enabledrU raw_enabledrKU dump_settingsrNubUsymbol_footnote_startrKUidsr}r(hhhhh jghhwh$hUhjjjhjVhjIh!jh#jh"juUsubstitution_namesr}rh-h7h/}r(h1]h4]h3]UsourcehGh2]h5]uU footnotesr]rUrefidsr}rub.