trust threads: provenance for data reuse in long tail science
TRANSCRIPT
Cooperative agreement #OCI0940824
Trust Threads: Provenance for Data Reuse in Long Tail Science���
���Beth Plale���
Data To Insight Center���Indiana University ���
April 23, 2015
The impact that digital data is having on science is increasing because of
phenomenal data growth.
Data driven agent based modeling for labor sharing in Zambia
Enabling new research
Data growth results in
• New and bigger research ques=ons : energy, clean and abundant water, sustainable food, healthy popula=on
• Requires interdisciplinary teams : scien=sts, technologists, medical doctors, social scien=sts, policy makers
• Requires data understood across discipline boundaries
Data successfully interpretable across scien=fic discipline boundaries has likely
seen some cura=on = curated data equates to reusable data =
Data reuse: Data collected for one purpose used for different purpose
Can technology innova=on (tools, data management, knowledge representa=on) accelerate frequency of data reuse?
• Data can carry with it thin threads of informa=on “trust threads” that connect data to both its past and its future.
• In carrying this minimal provenance, data becomes more trustworthy.
• Trustworthiness cri=cal to successful sharing, reuse of data in science and technology research
Provenance suitcase
Data provenance
Data Provenance
Lineage of data product. Like piece of art, tells what has happened to data product through its life. Shows rela=onships between products
Provenance for two polar ice sheet images showing problems with one (on right, missing contribu=ng files)
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom us
edwasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom us
ed
wasDerivedFrom
used
wasDerivedFromused
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom us
ed
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFromus
ed
wasDerivedFromus
ed
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
used
used
wasTriggeredBy
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFromwasGeneratedBy
wasDerivedFrom
wasDerivedFrom wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
wasTriggeredBy
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFromused
wasDerivedFromused
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasTriggeredBy
used
used
used
used
used
used
usedus
ed
used
used
usedus
ed
used
used
used
wasTriggeredBy
wasGeneratedBy
wasTriggeredBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFromwasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom used
wasDerivedFrom
used
wasDerivedFrom us
edwasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
usedused
used
used
used
wasTriggeredBy
wasTriggeredBy
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFromus
edwasDerivedFrom
used
wasDerivedFromwasTriggeredBy
used
wasDerivedFrom
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
wasTriggeredBy
wasGeneratedBy
wasTriggeredBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFromwasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFromused
wasDerivedFrom
wasTriggeredBy
used
wasDerivedFromus
edwasDerivedFrom
used
wasDerivedFromus
edwasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom used
wasDerivedFrom
used
used
used
used
used us
edused
used
used
used
usedused
used
used
used
used
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
used us
ed
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
wasTriggeredBy
wasGeneratedBy
wasTriggeredBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFromwasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasTriggeredBy
used used
used
used
used us
ed
used
used
used us
ed
used
used
usedused
used
used
used
used
used
used
used
used
used
used
used
wasGeneratedBy
wasTriggeredBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFromused
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFromus
edwasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
used
used
used
used used
used
wasTriggeredBy
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
usedused
used
used us
edused
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
wasTriggeredBy
wasTriggeredBy
wasGeneratedBy
wasDerivedFrom wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFromwasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFromwasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
used
used
used
used
used
used
used
used
used
used
wasTriggeredBy
used
used
used
used
used
usedused
used us
ed
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used used
used us
edused
used
used
used
used
used
used
used
used
used
used
used
used
usedused
used
used
used
used
used
wasTriggeredBy
wasTriggeredBy
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFromus
edwasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom us
edwasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom wasTriggeredBy
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
wasTriggeredBy
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFromus
edwasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFromus
edwasDerivedFrom
wasDerivedFrom
used
used
used
used
used used
used
used
used
used
used
used
used
used
used
wasTriggeredBy
wasGeneratedBy
wasGeneratedBy
wasTriggeredBy
wasTriggeredBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFromwasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFromwasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
used
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFromwasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
used
used
used
used
used
used
used
used
used
used
used
wasTriggeredBy
wasTriggeredBy
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFromused
wasDerivedFrom
used
wasDerivedFromused
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom used
wasDerivedFrom
used
wasDerivedFromused
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasTriggeredBy
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
wasTriggeredBy
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFromused
wasDerivedFrom used
wasDerivedFrom
used
wasDerivedFromused
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFromwasGeneratedBy wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
used
used
used
used
used
used
used
used
used
used
usedused
used
used
used
wasTriggeredBy
wasTriggeredBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasTriggeredBy
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
used
used
used
used
used
used
used
used
used
used
used
used us
ed
used
used
used
used
used
used usedus
ed
used
used
used
wasGeneratedBy
wasGeneratedBy
wasTriggeredBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
used
used
used
used
wasTriggeredBy
wasTriggeredBy wasTriggeredBy
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
wasTriggeredBy
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
usedused
used
used
wasTriggeredBy wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
wasDerivedFrom
used
usedusedused
used
used
used
used
used
wasTriggeredBy
wasGeneratedBy
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFromwasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFromwasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasTriggeredBy us
edwasDerivedFrom
used
wasDerivedFrom
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
wasTriggeredBy
wasGeneratedBy
wasTriggeredBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFromwasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
used
wasDerivedFrom
used
wasDerivedFrom us
edwasDerivedFromused
wasDerivedFromused
wasDerivedFrom
used
wasDerivedFrom used
wasDerivedFrom us
edwasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom us
edwasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFromw
asDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
used
used
used
used used
used
used
used
used
used
usedused
used
used
used
wasTriggeredBy
wasTriggeredBy
wasTriggeredBy
used
wasDerivedFrom used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasTriggeredByused
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
used
used
used
used
used
used
used
used
used
used
used
usedused
used used
used
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFromused
wasDerivedFrom
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom w
asDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
used
used
usedused
used
used
used
used
used
used
used
used
used
used
used
used
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFromused
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
used
used
used
used
used used
used
used
used
used
wasTriggeredBy
wasTriggeredBy
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFromused
wasDerivedFrom
used
wasDerivedFrom used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom us
edwasDerivedFrom
used
wasDerivedFrom used
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
used used
used
used
used
used
used
used
usedw
asTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used
used used
wasTriggeredBy
wasTriggeredBy
wasTriggeredBy
used
wasDerivedFrom used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used used
used
used
used
used
used
used
used
used
used us
ed
used
usedused
used
used
wasTriggeredBy
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
used
used used
usedused
used
used
used
used
used
used
used
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFromused
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used us
ed
used
used
used
used
used
used
used
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used
wasDerivedFrom
used
used
used
used
used
usedused used
used
used
used
used
used
used
used
used
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used used
used
used
used
used
used
used
used
used
usedus
ed
used
used
used
used us
ed
used used
used
usedused
used
used
used
used
used
used
wasTriggeredBy
wasTriggeredBy
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFrom
used
wasDerivedFromused
wasDerivedFrom wasGeneratedBy
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
wasDerivedFrom
used used
used
used
used
used
used
used
used
used
used
used
used
BTBT
BT
BTBT
BT
BT
daily
BT
BT
BTBTBT
BT
BT
BT
BTBT
BT
BT
BTBT
BT
BT
BTBT
BT
BTSeaIce12km
BT
@L3seaice
BT
BT
BT
BT
BT
BTBT
SeaIce12km
BT
BT
BT
BT
BT
BTBT
@L3seaiceBT
BTBT
BT
BTBT
BT
BT
BT
BT
BT
BT
BT
BTBT
BT
BT
BT
BT
BT
BT
BT
BT
BTBT
BT
BT
BTBTBT
BTBT
BT
BTdaily
BT
BTBT
BT
BT BTBT
BT
BT
BT
BTBT
BT
BTBT
daily
@L3seaice
BT
BTBT
BTBT
BTBT
BTBT
BT
BTBT
BTBT
SeaIce12km
BT BT
BT
BT
BT
BT
BT
BT BT
BT
BT
@L3seaice
SeaIce12km
BT
BTBT
daily
BTBT
BTBTBT
BT
BTBT
BT
BT
BTBT
BT
BT
BT
BT
BT
BTBT
BTBT
BT
BTBT
daily
BT
@L3seaice
SeaIce12km
BTBT
BTBT
BT
BT
BT
BTBT
BT
BT
BT
BT
BT
BTBT
BTBT
BTBT
BTdaily
BTBT
BTBT
BTBT
BT
BT BT BT
BT
@L3seaice
SeaIce12km
BT
BT
BT
BT
BTBT
BT
BTBT
BT
BTBT
BTBT
BT
BT
BTBT
BT
BT
BT
BT
BT
BT BT
BT BT
@L3seaice
SeaIce12km
BT
BTBTBT
BTBT
daily
BT
BT
BT
BT
BT
BT
daily
@L3seaice
BT
BT
BT
BT
BTBT
BT
BTBT
BT
BT
SeaIce12km
BT
BTBT
BT
BTBT
BT
BTBT
BT
BT
BT
BT
BT
BTBT
BT
BT
BTBT
BT
BT
BTBT
BT
BT
BTBT
BT
BT
BT
BTBT
BT
BT
SeaIce12km
@L3seaice
BT
BT
BT BT
BT
BTBTdaily
BT
BT
BT
BT
@L3seaice
SeaIce12km
BT
BT
BT
BT
BT
BT
BT BT
BTBT
BTBT
BTBT
BT
BT
BT
BT
BT
BT
BT
BTBTBT
BTdaily
BT
SeaIce12km
BT
BTBT
BT
BTBT
BT
BT
BTBT
BTdaily
BTBT
BT@L3seaice
BTBT
BT
BT
BT
BT BT
BT
BT
BT
BT
SeaIce12km
@L3seaice
BT
BT
BT
BTBT
BTBT
BT
BTBT
BT
BT
BTBT
BT
BT
BT
BT
daily
BT
BT
daily
BT
BTBT
BT
BT
BT
BT BT
BT
BTBT
BT
BT
@L3seaice
SeaIce6km
SeaIce12km
BT
BTBT
BTBT
BT
BT
BTBT
BT
BTBT
SeaIce12km
BT
BTBT
BTBT
SeaIce6km
BT
@L3seaice
BT
BTBT
BT
SeaIceDrift
BT
BT
BTBT
BT
BT
BT
BTBT
BT
BT
BT
BT
BTBT
BT
BT
BT
BT
BT
BT
BT
BTBT
BTdaily
@L3seaice
BT
BT
BT
BT BT
BT
BT
BT
BT
BT
BT
BT
BT
BT BT
BT
BT
BT
BT
BT
BT
BT
SeaIce12km
BT
BT
@L3seaice
BT
BT
BT
BT
BT
BT
BT
BT
BT
BT
BT BT
BT
BT
BT
BT
BT
BTBT
BT SeaIce12km
BT
BT
BT
BT
BT
daily
daily
BT
BT BT
BT
BT
BT
BT BT
BT
BT
BT
@L3seaice
BT
BT
BT
BT
BT
BT
BT
BTBT
BT
BT
BT
BT
BT
BTBT SeaIce6km
SeaIce12km
BT
BTBT
BT
BT
BT
BT
BTBT
BTBT
@L3drift
BT
BTBT
BT
BTBT
BTBT
BTBT
BT
BT
daily
BTBT
BTBTBT
BTBT
BT
BTBT
BT
BT
SeaIce12km
daily
BT
@L3seaice
BTBT
BT
SeaIce6km
BT
@L3seaice
SeaIce12km
SeaIce6km
BT
BTBT
BT
BTBT
BTBT
BT
BT
daily
BT
BT
BT
BT
BT
BT
daily
BT
BT
@L3seaice
BT
BTBT
BT
BT
BT
BT
BT
BT
BT
BTBT
SeaIce12km
BT
BT
BT
BT BT
BTBT
BTBT
BTBT
BTBT
BTBT
BTBT
BT
BT
BT
BT
BT
BTdaily
BTBT
BTBT
BT
BT
BT
SeaIce12km
BT
BT
BT
BTBT
BT
BT
BTBT
SeaIce12km
@L3seaice
BT
BTBT
BT
BT BT
BTBT
BT
BT
BT
BT
BTBT
SeaIce12km
BT
BT
BT
BT
BT
BT
BT BTBT
BT
BT
BT
BT
BT@L3seaice
BT
daily
BT
BT
BTBT
BT
daily
BT
BT
BT
BT
SeaIce12km
@L3seaice
BT
BT
BT
BT
BTBT
BTBT
BT
BT
BTBT
BT
BTBT
BT
BT
BT BT
BTBT
BTBT
BT
BT
BT
BTdaily
BT
BT
SeaIce12km @L3seaice
BT
BT
BT
BT
BT
BT
BT
BT
BTBT
BT
BTBT
BTBT
BT
BT
BT
BTBT
BT
BT
BT
BT
BT
BT
BT
BT
BT BT
BT BT
BT
BT
BT
BTSeaIce12km
@L3seaice
BTBT
BT
BT
BT
BT
daily
BT
BT
BT
BT
BT
BT
BTBT
BT
BT
BT
BTBT BT
BT
BTBT
BTBT
SeaIce12km
@L3seaice
BT
BT
BT
BT
BT BTBT BT
BT BT
daily
BT
BT
BT
SeaIce12km
BT
@L3seaice
BTBT
BT
BT
BTBT
BTBT
BT
BTBT
BT
BT
SeaIce12km
BT
BTBT BT
BT
BTBT
BTBT
@L3seaice
BTBT
BTBT
BTBT
BT
daily
BTBT
BTBT
daily
BT
BT
BTBT
BT
BT
BT
BT
BT
BT
BTBT
BTBT
BTBT
BT
BT
BTBT
BT
BT
BT
BT
BT
BT
BT
BT
BT
BTBT
SeaIce12km
@L3seaice
BTBT
BT BT
BT
BT
BT
BT
BT
BT
BT BT
BT
BT
BTBT
BT
daily
BT BT BT BT
BT BTBTBT
BT
daily
BTBT
BT
BTBT
BT
BT
BT
BT BT
BT
BT
SeaIce12km
BT
BT
@L3seaice
BT
BT
BTBT BT
BT
BT
BT
BT
BT
BT
BTBT BT
BT
BT
BT
BT
BT
BTdaily
BTBT
BT
SeaIce12km
BTBT
BT
BT
BT BT
BT
BT
@L3seaice
BTBT
BT
BT
BT
BTBT
daily
BT
BTBT
BTBT
BT
BTBT
BT
BT SeaIce12km
BT BTBT
BTBTdaily
BT
BT
@L3seaice
BT
BTBT
BT
BT
BT
BT
BTBT
BT
BT
BT
BTBT
BT
BTBT
BT
BTBT
daily
BT
BT BTBT BT
BT
BT BT
BTSeaIce12km @L3seaice
BT
BT
BT BT
BT BT
BT
BTBT
BTBT
BT
BT
2012
Month of ice sheet images, linked by template passed from day to day in processing
Long tail of science D
ata
Qua
ntity
Data complexity and dispersion
research center
project
lab
team
solo inves=gator
ci=zen
Giga
Peta
Exa
Tera
Researchers who produce and analyze heterogeneous data that is unique and at fine resolution and granularity.
Researchers who need to collaborate with others to make scientific advances.
Researchers who lack access to reliable cyberinfrastructure for managing, sharing, analyzing, publishing and archiving data.
Publishing Long Tail Science Data
Table of Contents • The Flood • The Publishing Exercise
Terminology
• Data publishing : act of making a data set that is associated with a peer reviewed scien=fic publica=on publicly available
• Data preserva4on : steps undertaken to ensure a data set is available and useable by subsequent genera=ons of scien=sts
• Data sharing : reuse and repurpose of a data set for by a third party
• Long tail science: science u=lizing data from mul=ple sources; data and models can be manipulated on worksta=on or small cluster
Cast of Characters
• Hydrologist (PI level) • Hydrologist Postdoc • Data scien=st (PI level) • Research programmer • Data curator (CLIR Fellow) • University library (data management services)
Extreme Mississippi Flood 2011
Thanks to
SEAD funded by Na4onal Science Founda4on. B. Plale and P. Kumar, co-‐PIs.
-‐-‐ Praveen Kumar and his team at UIUC
Extreme Mississippi Flood 2011*
• Birds Point New Madrid (BPNM) Floodway is an agricultural region located west of Mississippi River just south of its confluence with the Ohio River, near city of Cairo, Illinois.
• Policy established following disastrous 1927 Flood permits levees surrounding BPNM Floodway to be inten=onally breached during extreme floods.
• During historic flood stages of May 2011, Floodway was ac=vated for first =me since 1937. U.S. Army Corps of Engineers used blas=ng agents on May 2, 3 and 5, 2011 to create ar=ficial breaches.
* A. E. Goodwell, Z. Zhu, D. Duca, J. A. Greenberg, P. Kumar, M. H. Garcia, B. L. Rhoads, R. R. Holmes, G. Parker, D. P. Berreca, R. B. Jacobson (2014). Assessment of Floodplain Vulnerability during Extreme River Flood 2011, Environmental Science and Technology.
Why study breach?
• Impact was drama=c. Parts of agricultural floodplain were inundated for over a month.
• Large floods, such as this, create highly variable spa=al pacerns of erosion and deposi=on. These localized areas, or “hotspots”, of change expose underlying landscape vulnerabili=es.
• Study is opportunity to assess causes to guide future ac=ons.
Sudden eleva=on drop at O’Bryan Ridge(B1, B2) resulted in substan=al erosion there.
Oxbow of old river bed (Ten Mile Pond) should have seen substan=al erosion but didn’t because trees had filled into area since 1937.
Grey area is floodplain, blue is Mississippi River. Green circles are USGS installed sensors. Levee breach at point A.
Credit for images: A. E. Goodwell, et al. (2014). Assessment of Floodplain Vulnerability during Extreme River Flood 2011, Environmental Science and Technology.
Sudden eleva=on drop at O’Bryan Ridge resulted in substan=al erosion -‐ see B1, B2.
Part II: Publishing Exercise
• Fall 2014 Postdoctoral Fellow asked to publish datasets associated with 2014 ES&T paper. She used SEAD to ingest into ins=tu=onal repository at Fellow’s university and link dataset and publica=on.
• Ques=ons Fellow had to address: – What do I publish? – Why do I publish? – How do I publish?
Kinds of Data Used • Map of land eleva=on (Digital Eleva=on Model). LIDAR data from 2005 and 2011. US Army Corps of Engineers. Images
• Max velocity of water. Hydrosed 2D hydrology model. Modeled data
• Woody vegeta=on mapping. AVIRIS – airborne visible/infrared imaging. Images
• Water level sensors. USGS Water level sensors. Sensor data
• Soil/landscape sensi=vity to erosion. K/T func=on. Computed values
Published: single Research Object
• 5 Hydrosed model max velocity files (whole Birds Point New Madrid floodway (BPNM), Case 1 and 2: O'Bryan, Case 1 and 2 Ten Mile Pond) – Whole BPNM: Umax_BPNM.tif 1.47 MB – Case 1 O’Bryan: Umax_OBR_Case1.tif 3.23 MB – Case 2 O’Bryan: Umax_OBR_Case2.tif 3.23 MB – Case 1 Ten Mile Pond: Umax_TMP_Case1.tif 6.59 MB – Case 2 Ten Mile Pond: Umax_TMP_Case2.tif 6.59 MB
• K/T Factor soil map: KT_BPNMShape.tif 7.34 MB • Classified AVIRIS map (woody vegeta4on) AVIRIS_50M_BPNMshape.tif
919.48 KB • Original differen4al DEM (1.5 m resolu=on) Sub_Original_5ft.tif
2.84 GB • Final differen4al DEM (10 m resolu=on) Sub_Corrected_10m.tif
72.75 MB • Final differen4al DEM (3 m resolu=on) Sub_Corrected_3m.tif
798.12 MB
Publishing Exercise
– Why do I publish? • A: publish data necessary to reproduce images in published paper
– What do I publish? • A: publish data necessary to reproduce images in published paper
Publishing Exercise
• What do I publish? – Answer: only those files necessary to reproduce images
• Implica<ons?
– All TIF files (geoTIF) – 3.74 GB “Research Object”
• Umax_BPNM.tif 1.47 MB • Umax_OBR_Case1.tif 3.23 MB • Umax_OBR_Case2.tif 3.23 MB • Umax_TMP_Case1.tif 6.59 MB • Umax_TMP_Case2.tif 6.59 MB • KT_BPNMShape.tif 7.34 MB • AVIRIS_50M_BPNMshape.tif 919.48 KB • Sub_Original_5ft.tif 2.84 GB • Sub_Corrected_10m.tif 72.75 MB • Sub_Corrected_3m.tif 798.12 MB
It is existence of content as TIF files that makes data publishing insufficient for data sharing
Size Macers
Implica<ons? • Dspace not well suited to
Research Object (RO) that is 3.74 GB in size
• Published data is 3.74 GB RO, so shared data will be substan<ally larger.
• Libraries need partnerships with large scale storage providers
• Umax_BPNM.tif 1.47 MB • Umax_OBR_Case1.tif 3.23 MB • Umax_OBR_Case2.tif 3.23 MB • Umax_TMP_Case1.tif 6.59 MB • Umax_TMP_Case2.tif 6.59 MB • KT_BPNMShape.tif 7.34 MB • AVIRIS_50M_BPNMshape.tif 919.48 KB • Sub_Original_5ft.tif 2.84 GB • Sub_Corrected_10m.tif 72.75 MB • Sub_Corrected_3m.tif 798.12 MB
Technology frame of reference: SEAD
SEAD Leadership
Cooperative agreement #OCI0940824
Clockwise from top: Margaret Hedstrom, UMich; Beth Plale, IU; Praveen Kumar, UIUC; Jim Myers, UMich; Sandy Payette, UMich
SEAD – Researcher’s Perspective
Collec=on and analysis of data
SEAD Sharing and publica=on of data
Management and sharing of data
SEAD – Researcher’s Perspective
Collec=on and analysis of data
SEAD Sharing and publica=on of data
Management and sharing of data
Time
Cura=on engagement
SEAD Technology
• Active Project Spaces – Collaboration / file sharing space for research projects– Staging area for data curation prior to publishing
• SEAD Virtual Archive– Push publishable data (as Research Objects) from an
active Project Space to appropriate preservation and discovery environment, and back for reuse
• People, Data, Things – Researcher profiles, repository profiles, data profiles– all linked, related, dynamic– JSON LD
Project Spaces ... organize, describe, visualize
geo dashboard
dataset collec=ons with metadata
Virtual Archive
Deposit refines for
discovery
Virtual Archive ... publish and archive data
Deposit refines for
discovery
The 3.6 GB
Research Objec
t of
data for Mississipp
i
River Flood 20
11
deposited to ID
EALS
repository
Cloud Storage
IR
Project Space
Project
Project Space Project
Space
Match-‐ Maker
Cura=on Work Flow Submit
Submit
Computa=onal Environments
Tools Apps
Project Space
Project Space
Project Space
Long Tail of D
ata Prod
ucers
Domain Repository
Scien=
fic W
orkflow
VA
Registry Services
social network
• At point in =me when researcher ready to publish her data, she must make decision about what data – from amongst all data sources used, consulted, and created, should be include in publishable result.
• At this point (at latest) is when data curator gets engaged
• Works in-‐situ and alongside researcher to assemble publishable object – Occurs in SEAD Project Spaces
Back to Hydrology Researcher
Tracking life of research object
Research Object
Unique ID
Agents
StatesRelationships
Content
Research Object Framework
• Data creator• Curator• Data re-use scientist
• Live• Curated• Published
• Aggregates • Related to • Describes • Derived from • Versioned from
• Files• Bitstreams• Pointers• Annotative
information
Bundles of resources that use common standards and services to transfer and consume them
Research Object Publish reuse lifecycle
• RO begins as Live Object (LO), this is when all future content is s=ll in state of high flux.
• Live Object state is collabora=ve, mul=-‐person project team working together in single shared project space in SEAD environment
Research Object Publish reuse lifecycle
• At point in =me when researcher begins assembly for publishing, she will prune and organize material to publish into new directory or set of directories, or tag specific files.
• This pruned content becomes the CuraAon Object (CO)
Research Object Publish reuse lifecycle
• Once researcher and digital curator agree that content and descrip=ons of research product are ready, researcher signals intent to publish whereupon research object moves from its state as a Cura=on Object to a new state as a Published Object (PO).
• PO exists in “control zone” where changes are tracked in a rigorous manner.
Publish reuse lifecycle
Behavior diagram (aka “state machine”)
Cloud Storage
IR
Project Space
Project
Project Space Project
Space
Match-‐ Maker
Cura=on Work Flow Submit
Submit
Computa=onal Environments
Tools Apps
Project Space
Project Space
Project Space
Long Tail of D
ata Prod
ucers
Domain Repository
Scien=
fic W
orkflow
VA
Registry Services
social network
Cloud Storage
IR
Project Space
Project Space Project
Space
Match-‐ Maker
Cura=on Work Flow Submit
Submit
Computa=onal Environments
Tools Apps
Project Space
Project Space
Project Space
Long Tail of D
ata Prod
ucers
Domain Repository
Scien=
fic W
orkflow
VA
Registry Services
social network
Wild West Boundary Waters Control Zone
Cloud Storage
IR
Project Space
Project Space Project
Space
Match-‐ Maker
Cura=on Work Flow Submit
Submit
Computa=onal Environments
Tools Apps
Project Space
Project Space
Project Space
Long Tail of D
ata Prod
ucers
Domain Repository
Scien=
fic W
orkflow
VA
Registry Services
social network
Wild West Boundary Waters Control Zone
Requires two way communica4on with partner repositories
From point in =me where data is first conceived as ready for publica=on through to its first reuse is
cri=cal stage in data’s life.
If we are unable to bring technical rigor to handling object cura=on, rela=onships, and deriva=ons in this early stage, we lose cri=cal
opportunity for enhancing trustworthiness of data object:
an opportunity that is irrecoverable.
With special thanks
• Hydrologist postdoc Allison Goodwell, Professor Praveen Kumar, UIUC
• Dr. Inna Kouper, IU • IU SEAD developer team: Scoc McCaulay, Isuru Suriarachchi, Aravindh Varadharaju, Charitha Arachchige, Yuan Luo
• Na=onal Science Founda=on