data curation at nees

Site Operations Manager’s Workshop – October 23rd-24th, 2007

Data Curation at NEES

Claude TrottierData Curator

Site Operations Managers Site Operations Managers WorkshopWorkshop


Presentation Outline

• Data curation accomplishments• What data curation looks like• Data curation workflow• How researchers and local site personnel

can contribute to data curation• Proposed new Data Archiving and Sharing

Plan – Attachment E of the RPA• Future goals and tasks


NEES Data CurationAccomplishments Since 9/2006

• Data Curation Application Designed, Coded, and Implemented in NEEScentral

• Surveyed and “registered” all existing projects in NEEScentral

• Identified Initial Set of “Mature Projects/Experiments” to Curate

• Curated 19 Experiments of 6 Projects• Curated 636 files


NEES Data CurationProject PI Curation

StatusPublished?

Tsunami Inundation Dan Cox Completed Yes

Soil Structure Foundation Interaction

Bruce Kutter In Progress 4 Experiments

Building Performance on Softened Ground

Shideh Dashti In Progress No

Dynamic Behavior of Slickensided Surfaces

Christopher Meehan

In Progress No

Soil Structure Foundation Interaction

Mehdi Saiidi In Progress No

Zipper Frames Roberto Leon 5 Experiments Completed

No


Data Curation Reports


Definition of Data Curation

Preparation of NEES research project contents, and subsequent publications, for facilitated indefinite accessibility and research replication


Res

earc

her 2.2a

Interim+ Repositoryuncurated

Equ

ipm

ent S

iteD

ata

Libr

aria

n

5 Publish4 Curate3 EncapsulateAfter

2 Post-processDuring Experiment

1 AcquireBefore0 Plan-1 Pre-award

0.2Design

0.3Build

directdata

storeddata

document process

0.1Goals

1.2Organize

App

rova

l

predefinedprocess

4.1Register

Var

iabl

e

ResearcherData Librarian

Dat

a A

rchi

ving

& S

harin

gP

lan

By 6 months fromend of experimentor simulation

Repeat for each Experiment

Repeat for each Trial

*Responsibility variable depending on project and Equipment Site practices. Responsibilities to be defined in the Research Participation Agreement

VerifyAggregateConvert

2.1

Equipment Site

1.3aInitial

Repository

1.3bBackup

Repository

3.2Central

Repositoryuncurated

4.2Assign Tags

4.3Certify

4.4PermanentRepository

curated

Release

1.1RecordDAQ

2.2bdocument

Primary Responsibility to Assure Task Completion:

Deviations to be specified in the Research Participation Agreement

All decisions at the time of experiment that do not affect Equipment Site safety are the

responsibility of the Researcher.

Data Conduit Reference Model

3.1Encapsulate

Mapping to Global Data Model

Mapping to Local Data Model

Pri

mar

y R

ole

in P

erfo

rmin

g T

ask

RAWDATA

CURATEDDATA

STRUCTURED DATA

+ Interim Repository may reside at the: Equipment Site, Researcher Institution, or Central Repository

CONVERTED DATA


Data Curation WorkflowExperimentExecuted

ResearchersCurate Experiment

Data

Researchers Prepare Data for

Uploading to NEEScentral

Experiment DataUploaded to NEEScentralWithin Six Months AfterExperiment Completion

Data CuratorCurates the

Experiment Files

Data CurationReport Sent

to Researcher

Researcher Reviewsthe Curation and Sends

Feedback to theData Curator

Data CuratorRevises Experiment

Data Curation

Researcher Reviews Experiment Data

Curation and Publishes or Returns to

Data Curator


Data Curation Checklist

• Guide as to what data curation researchers should do prior to uploading experiment files to NEEScentral

• Not all items on the checklist apply to every project

• Can be used as a communication tool between the researchers and the data curator


Data Curation Checklist FormNEEScentral Project Data Curation Preparation Task NEEScentral Project Data Curation Preparation Task

ChecklistChecklistStatusStatus Status DateStatus Date NoteNote

Valid values for Status are: Complete, Incomplete, In Progress, and N/A

Data Management Team/Contact Identified

Project Basic Definition in NEEScentral (post RPA approval)

Experiment Basic Metadata and Schedule in NEEScentral (post RPA approval)

Project Public Directory populated with project level documents (including a final report – if applicable)

Project Public Directory files have annotations (Description attribute has descriptive text)

Project Documentation Directory populated with project level documents which should not be publicly available until Project is published

Project Documentation Directory files have annotations (Description attribute has descriptive text)

Project Analysis Directory populated with project level analysis

Project Analysis Directory files have annotations (Description attribute has descriptive text)

Experiment-1 Documentation Directory populated with experiment level documents (including a final report – if applicable)

Experiment-1 Documentation Directory files have annotations (Description attribute has descriptive text)

Full form available in NEEScentral under the NEES Data Curation project/ Public Directory/ Data Curation Checklist folder


Detailed Data CurationTask List

• Survey the experiment NEEScentral content• Read documentation• Determine ontology terms• Examine contents of files to decide on title,

description, and ontology terms• Form descriptive data file titles based on

experiment and/or trial name or title• Associate relevant ontology terms to the file• Associate converted, corrected, and derived files

to the source unprocessed (raw) data file


Proposed New Data Archiving and Sharing Plan (DASP)

• More structured information than current DASP form

• Informs the researchers early on as to what they need to do to conform to the data curation standards

• Set file type standards for curation and archiving


Data Curation - the Future

• Curate all currently completed experiments• Enhance the Data Curation Application • Publish/present data curation information to the

NEES community and wider earthquake engineering universe

• Establish productive communication channel with the researchers

• Provide NEES management and NSF with visibility into experiment and curation status

• Develop long term archiving strategy


Thank You

data curation at nees

Documents