a community approach to preservation: experiences with social science data asist summit 2010...

A Community Approach to Preservation: “Experiences with Social Science Data”

ASIST Summit 2010

Jonathan Crabtree

April 9, 2010

The Odum Institute

• Oldest Institute or Center at UNC-CH Founded 1924

• Mission: Teaching, research, & service for social sciences

• Cross-disciplinary focus

Odum’s Multidisciplinary Mission

• Diverse areas of study• Culture of multidisciplinary collaboration• Focus on data reuse• History of data stewardship• Successful at relationship building

Social Science Archives in Collaboration for Preservation

• Strategic partnership agreements• Coordinated operations• Joint best practices• Shared federated catalog• Shared tools & technologies

The Plan• Identify significant data collections (classic)

• Identify important contemporary data (“at risk”)

• Develop common standards and procedures across partnership

Partnership Goals

• Use technological advances to encourage metadata standards and a shared catalog

• Establish a replicated storage network• Develop common standards and activities• Determine how the partnership can expand

Dataverse Network

• Open source platform

• OAI server

• DDI metadata standards

• Federated approach

Overview of How Catalog Works

Access DataWith Extraction and Analysis, Through CatalogDirect to Partner Sites

View Information on DataThrough CatalogLink to Data at Partner Site

Search Shared Catalog

Data Mirror

MetadataCatalog

Harvester

Online Catalog

Online Analysis

Multi-Archival: Syndicated Storage Platform

Dual Approach to Replication

• Private LOCKSS Network– Peer to Peer

• iRODS Datagrid– Client Server

Syndicated Storage Platform (SSP)

iRODS extension of the NARA Transcontinental Persistent Archive Prototype (TPAP)

• OAI-PMH based transfer• XML/XSL Translation• iRODS rules and microservices

Nexuses for Preservation Failure• Technical

– Media failure: storage conditions, media characteristics– Format obsolescence– Preservation infrastructure software failure– Storage infrastructure software failure– Storage infrastructure hardware failure

• External Threats to Institutions– Third party attacks – Institutional funding– Change in legal regimes

Replication as Part of a Multi-Institutional Preservation and Sustainability Strategies

There are potential single points of failure in both technology, organization and legal regimes:

• Diversify your portfolio: multiple software systems and hardware

• Find diverse partners – diverse business models, funding sources, and legal regimes

• Use strategic partnerships to achieve a balance• Diversify your holdings• Develop macro appraisal policies• Develop data-transfer agreements that ensure sustainability

Summary

• Replication ameliorates institutional risks to preservation

• Data PASS requires policy based, auditable, asymmetric replication commitments

• Sustainability is attained through diversification• Strategic partnerships must be made• Data reuse is enhanced through macro appraisal

policies

Contact Information

Jonathan Crabtree

Website: http://www.odum.unc.edu

E-mail: jonathan_crabtree@unc.edu