Data without BoundariesA short Overview

Coordination : Roxane SilbermanCNRS/Réseau Quetelet

Presented by Mike Priddy/DANSIris Alfredsson/SND

Cologne, ESSnet, 2011-10-27

The DwB Project, a Short Overview

Toward a European Research Infrastructure• A four-year EU-funded FP7-13 project (2011-2015)• Aims:

Linking the capacity of the research community with the important resources of the official micro data in Europe

Enhancing researchers access to official micro data in Europe Surveys and administrative datasets, combined files Focus on confidential (highly detailed) data Focus on crossing national boundaries

• Mechanism = Coordination of existing infrastructures CESSDA Data Archives, and the ESS (NSIs coordinated by Eurostat, ECB) Based on volunteers

The DwB Project, a Short OverviewIntroduction

Project Focus and Mechanism

Partners

• Coordination: Roxane SILBERMAN (CNRS/Réseau Quetelet)

• 27 partners 1/3 CESSDA Archives: CNRS/RQ, GESIS, NSD, SND, FSD, DANS, UKDA, FORS,

EKKE, CIS, RODA 1/3 NSIs and Statistical departments: ONS, CBS, INSEE/GENES, SORS, IAB, SCB,

DESTATIS, CSIC, CNPS-INS 1/3 Universities: URV, UL, UPC, ULL, SOTON, CIS (IPUMS) + MT (SME)

The DwB Project, a Short OverviewIntroduction

Partnership

From Current Situation …• Access to official statistics both anonymized and highly detailed is still

uneven in Europe, both at national and at European levels

• Access to Eurostat highly anonymized datasets is still burdensome

• Increasing level of anonymization does not meet the researchers needs

• Though crucial for comparative Research, crossing borders is even worse: different legal frameworks, institutional arrangements and criteria for

accreditation, different providers (NSIs, Archives), different modes of access (no access, safe centres, remote execution, remote

access), different languages, different views about security, anonymization, output checking…

The DwB Project, a Short OverviewContext

… To DwB Project Main Issues• Building a central point of access: what are the available data? How can

they be accessed?

• Metadata standards and interoperability: NSIs tend to use SDMX as a standard for metadata exchange, CESSDA Archives use DDI as a standard for documentation

• Legal issues and accreditation: towards a European accreditation

• Servicing the use of OS data: provide tools (format, routines for harmonization), train the researchers for using European micro data

• Technical, standardization and methodological issues in developing a European distributed remote access both for national and for European micro data, flexible to national institutional arrangements (NSI or data archives as provider): propose and implement a test case

The DwB Project, a Short OverviewContext

Three Blocks, Twelve Work Packages

• Block 1: Access Facilities (WP3, WP4, WP9, WP10 and WP11)

• Block 2: Front Office (WP5, WP7, WP8 and WP12)

• Block 3: Enlarging Cooperation (WP6)

+ WP1 (Project Management)

+ WP2 (Internal & External Communication)

The DwB Project, a Short OverviewProject Architecture

WP 7 – Standards DevelopmentThe central purpose is to create a common platform for lasting cooperation between NSIs and data archives.

•Objective 1 – Interaction between data archives and NSIs relating their use of metadata standards

•Objective 2 – Interaction with standards groups for administrative and preservation metadata

•Objective 3 – Identification of similar cross disciplinary standards activities and collaboration with this as appropriate

WP 7 – Tasks 1-4• Task 1 - A survey of the present state of the art in metadata usage in NSIs and data

archives, as well as their plans for the coming 3-4 years.

• Task 2 - Establish which metadata standard meets the majority of needs and which related vocabularies and coding schemes may be beneficial across all sectors.

• Task 3 - Explore and define a set of standards with future relevance for European social science data infrastructure needs, and to make an assessment of the different standards applicability to specific purposes.

• Task 4 - Identify key areas where the NSIs and data archives have issues that are not sufficiently covered by present standards.

WP 7 – Tasks 5-7• Task 5 – Define specific rules and best practices for key areas of metadata standard

selection and usage.

• Task 6 - Discover and describe specific issues involved in software development to specific widely used metadata standards.

• Task 7 - Build and maintain effective collaboration with the DDI Technical Implementation Committee and the SDMX (Statistical Data and Metadata eXchange).

WP7 – Tasks 8-9• Task 8 - Identify metadata standards and practices in related disciplines to support

extension of existing social science metadata and interdisciplinary use of research results. Identify further needs.

• Task 9 - Identify sources of contextual metadata and identify regulative standards for linking data and publications (beyond PID system) and cooperation with respective initiatives and projects like DatapluS of the SURFfoundation. Create the basis for linking towards other data types and links towards reports.

WP 8 – Improving Resource Discovery for OS Data

The CESSDA portal is a discovery tool and gateway to the data holdings of the network of CESSDA data archives. In relation to Official Statistics (OS) data it is clearly incomplete. The aim is to bring the disparate and variable information on the availability of OS research data together from across the European Research Area.

•Objective 1 – To investigate the possibilities and problems associated with harvesting NSI metadata on OS data and making them available through an enhanced CESSDA portal. •Objective – To create a metadata model incorporating SDMX and DDI as well as any system-specific enrichment required to deliver extended portal functionality. •Objective 3 – To develop functional requirements for effective resource discovery on data harvested from the NSIs.

WP 8 – Tasks 1-4• Task 1 - Investigate the desired portal resource discovery functionality.

• Task 2 - Evaluate the disparate body of metadata on Official Statistics (OS) data available including export/interchange formats currently offered.

• Task 3 – Construct an object model based on metadata available suitable for describing the disparate resources.

• Task 4 - Provide a consistent mapping between SDMX and DDI3 and identify any metadata enrichment required by the system, which goes beyond that contained within SDMX and DDI3.

WP 8 – Tasks 5-7• Task 5 – Draft a metadata model applicable across all NSI data to be harvested.

• Task 6 - Develop appropriate workflows and dataflows including enrichment processes encompassing: direct harvesting of metadata from NSI’s and harvesting of NSI data from an intermediary data archive.

• Task 7 - Propose portal resource discovery functionality, which could be provided based on the available metadata.

To Summarize …• A challenging project:

Need to build trust and common understanding between NSIs, Archives and Research Communities

Need to agree on standards, provide a model and implement a pilot Need to enlarge cooperation and strong coordination with other initiatives &

ongoing discussions

• A crucial step toward a European research infrastructure within the context of the CESSDA ERIC: Building a single point of entry, Paving the way for a European accreditation, Enhancing access to anonymized official data, Providing a flexible infrastructure for accessing confidential data

The DwB Project, a Short OverviewConclusions

Thanks for ListeningContact:

iris.alfredsson@snd.gu.semike.priddy@dans.knaw.nl

Website: http://www.dwbproject.org/