Evolution of Data Documentation

ICPSR 2012

Evolution of Data Documentation

Codebooks!

Codebooks!

Codebooks!

Codebooks!

Description of study Description of files Description of variables Survey instrument Classification Standard

Codebooks! Location of variables in the data file

Codebooks! Values, Labels, Frequencies…

PDF

Osiris dictionary

SPSS cards

CBLT

Book

Osiris

SPSS

Problems of early digital codebooks(part 1)

PDF

Osiris dictionary

SPSS cards

CBLT

Book

Osiris

SPSS

(user has to re-create information inorder to re-use information)

Machine “readable” but not

Machine “actionable”

DDI 1 and 2

1.0 DOCUMENT DESCRIPTION 2.0 STUDY DESCRIPTION 3.0 DATA FILES DESCRIPTION 4.0 VARIABLE DESCRIPTION 5.0 OTHER STUDY-RELATED MATERIALS

Built to emulate early code BOOKS and digital Codebooks…

Problems of early digital codebooks (part 2)

Static, inflexible. Meant to document the end point of research --

Views research as linear. Hard to re-use the information for new research.

Problems of DDI 1 and 2

Emulated the Code Book Not flexible enough We could do so much more…

Three Stages of Technological Change

Type of Change Characterized by

Modernization Doing what we’ve always done, but using technology to do more and to increase efficiency

Innovation Doing things we’ve wanted to do, but could not do without the technology

Transformation Doing things that we didn’t imagine until technology made it possible.

Three Stages of Technological Change

Type of Change Characterized by

Early digital codebooks

Doing what we’ve always done, but using technology to do more and to increase efficiency

DDI 1 and 2 Doing things we’ve wanted to do, but could not do without the technology

DDI 3 Doing things that we didn’t imagine until technology made it possible.

Three Stages of Technological Change

Type of Change Characterized by

Early digital codebooks

Making codebooks machine readable

DDI 1 and 2 Making codebooks re-usable, even machine actionable…

DDI 3 Re-thinking “documentation”

Re-thinking the research process

DDI 1 and 2

Document Description Study Description Data Files Description Variable Description Other Study-Related

Materials

DDI 1 and 2

Document Description Study Description Data Files Description Variable Description Other Study-Related

Materials

Study Concept Data Collection Data Processing Data Distribution Data Archiving Data Discovery Data Analysis Repurposing

DDI 3