the alliance for data archive technologies: looking towards a common future myron gutmann, icpsr ben...
TRANSCRIPT
![Page 1: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/1.jpg)
The Alliance for Data Archive Technologies: Looking towards
a Common Future
Myron Gutmann, ICPSRBen Evans, ASSDA
Deborah Mitchell, ASSDAKevin Schürer, UK Data Archive
![Page 2: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/2.jpg)
Overview
• Why?• What?• Why Now?• Early Steps• Understanding Process• Understanding Needs• Next Steps
![Page 3: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/3.jpg)
Why?
• Data curation has been an ad hoc process, with local practices & expertise
• Since the 1990s– Enormous investment in technology– Significant successes in social science
(SDA, Nesstar, DVN, IPUMS, even ICPSR)– Major new ways to find & use content (Google) &
architectures to deliver content (web services)
![Page 4: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/4.jpg)
More Why
• Proprietary systems unsustainable• Market too small for commercial systems• Partnerships will help avoid unnecessary
duplication of effort & assure efficiency• Need to be truly global
![Page 5: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/5.jpg)
What?
• New organization to support technologies for curation, preservation, & delivery that are:– Open– Community-developed– Standards-based
• Built on existing networks of social science data archives & technology centers, and …
• Open to all who want to contribute
![Page 6: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/6.jpg)
Why Now? Three Standards
• DDI – Metadata Standard• OAIS – Preservation Reference Model• Repository Architecture Standards:
- Fedora, D-Space & Duraspace
• Organizational models like the DDI Alliance, CESSDA, Data-PASS (even the new Hathi Trust)
![Page 7: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/7.jpg)
Why Now? Community Tech
• Community-developed software has become widely used
• Examples: Drupal/Plone• Examples: Fedora• Examples: SOLR/Lucene
• But we shouldn’t ignore all the challenges that this software has faced
![Page 8: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/8.jpg)
Why Now? Workflows
• Improved workflow technologies are operating in many of our institutions
• Some are shared in CESSDA & Data-PASS• And in other communities: Virtual
Observatory
• Another challenge: not the same as sharing business practices in complex organizations
![Page 9: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/9.jpg)
Why Now? Progress So Far
• SDA• Nesstar• DVN
• All used in more than one archive• Not all open-source• Potential shared technologies that we can
leverage in the future
![Page 10: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/10.jpg)
1st Steps: October 2008 Meeting
• ICPSR• ASSDA• UKDA• Roper Center - UConn• Odum Ins. – N. Carolina• Harvard - IQSS• Minnesota Pop. Center• Berkeley – SDA• DANS – Netherlands
• DDA Denmark• Gesis – ZA• South Africa• DDI Alliance• IASSIST• Library of Congress• U.S. NSF• U.S. NIH• Canadian SSHRC
***Thanks to Library of Congress for hosting
![Page 11: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/11.jpg)
1st Steps: After October, 2008
• Solicit needs in the form of wish lists• Authorize creation of an organization at an
appropriate time• Work on raising money and finding common
ground for future work
![Page 12: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/12.jpg)
Process: Begin with OAIS Model
![Page 13: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/13.jpg)
Design OAIS for ICPSR
![Page 14: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/14.jpg)
Focus on Ingest
![Page 15: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/15.jpg)
ICPSR: Standards Compliance
OAIS Workflow• Ingest tools• AIP Creation-Validation• SIP Creation-Validation• DIP Creation-Validation• Audit tools
DDI Workflow• Tools for full variable-
level metadata creation not dependent on proprietary software (such as SPSS)
• DDI Editor• DDI Converter • DDI 2 to 3 translator
![Page 16: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/16.jpg)
Needs: Wish Lists from …
• ICPSR• UKDA• ASSDA• Harvard• Roper Center• Odum Institute
• DANS (Netherlands)• DDA (Denmark)• GESIS (Germany)• NSD (Norway)• Minnesota Pop.
Center
![Page 17: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/17.jpg)
Needs: A Catalog
Ingest
Data Management
Archival Storage
Access
Storage fabric/architecture (FEDORA or ?)Replication (LOCKSS)Persistent identifiersContent model development
Storage fabric/architecture (FEDORA or ?)Replication (LOCKSS)Persistent identifiersContent model development
Open metadata curationConfidentialitySoftware/algorithm archiving
Open metadata curationConfidentialitySoftware/algorithm archiving
Open metadata curationData format curationData management & analysisQualitative data managementData integrationMetadata registriesSurvey question managementData citation
Open metadata curationData format curationData management & analysisQualitative data managementData integrationMetadata registriesSurvey question managementData citation
Data format conversionSetup file creationInternational data sharingCommunity data/User comments/Web 2.0SearchConfidentialityPersistent identifiersVisualizationData citationSemantic data accessSecurity
Data format conversionSetup file creationInternational data sharingCommunity data/User comments/Web 2.0SearchConfidentialityPersistent identifiersVisualizationData citationSemantic data accessSecurity
AdministrationIdentity managementOAIS workflow & audit (SIP/AIP/DIP)Identity managementOAIS workflow & audit (SIP/AIP/DIP)
ProductionData producer toolsData producer tools
![Page 18: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/18.jpg)
Next Steps: Canberra Meeting
• Prime Goal: Strategic Planning • What’s the business model?• What are the links to… – Standards?– Security?– Archiving practice & workflows?– Training & Research?
• How do we measure success?
![Page 19: The Alliance for Data Archive Technologies: Looking towards a Common Future Myron Gutmann, ICPSR Ben Evans, ASSDA Deborah Mitchell, ASSDA Kevin Schürer,](https://reader036.vdocument.in/reader036/viewer/2022070306/5519fcea55034619378b4775/html5/thumbnails/19.jpg)
Three Major Outcomes
• Goal 1: A few critical decisions– Standards, repository framework, software
approaches
• Goal 2: Initial Common Interests. Examples:– Fedora data/content models– Open source metadata tools (DDI 3?)
• Goal 3: How do we collaborate?