METIS 2004(Geneva, 9-11 February 2004)

Inter-agency cooperation

for the dissemination and exchangeof standard metadata

Invited PaperSubmitted by

Eurostat, IMF, and OECD

Overview

• Growth of Internet-based modes of data and metadata dissemination

• Widespread recognition of the need for precise and robust tools to:– promote information access– improve the quality (and quantity) of the

information retrieved– enhance comparability of data from multiple

sources

The DSBB & Interagency Cooperation

• The IMF, in collaboration with the ECB and Eurostat, disseminate harmonized statistical metadata in SDDS format on the Internet. – The ECB’s Euro area metadata (

http://dsbb.imf.org/Applications/web/euronote/)

• Financial and external sector metadata covering 7 statistical domains for the 12 participating Member States

– Eurostat’s Indicators (http://dsbb.imf.org/Applications/web/eurostatnote/)

• Euro and structural indicators covering 14 statistical domains for the 15 participating Member States

The ECB’s Euro area metadata (accessible on the DSBB)

• Financial sector – Euro area banking sector – Eurosystem: ECB and national central banks (NCBs) of the 12

participating Member States– Euro area interest rates

• External sector– Euro area balance of payments– Euro area international reserves and foreign currency liquidity– Euro area international investment position– Euro area exchange rates

Eurostat’s Euro Indicators (accessible from the DSBB)

• Access to Euro indicators metadata by domain– National accounts– Industry, Commerce, Services and Energy– Harmonized Indices of Consumer Prices – Labour market – Monetary and financial indicators – Balance of payments – External trade– Business and Consumer surveys

• Access to metadata is also provided to Eurostat’s Structural Indicators: General economic background; Employment; Innovation and Research; Economic reform; Social cohesion; Environment

• On May 1, 2004 the number of countries will increase: from EU15 to EU25, thereby dramatically increasing access to metadata in SDDS format

Metadata within the OECD

• A decentralised system of collection, storage and dissemination for 30 Member countries (plus a limited number of non-member states)

• Only limited metadata are stored in databases actually linked to the statistics they describe

• A corporate metadata facility and metadata model are under development: METASTORE

OECD Metastore

• Allows metadata stored by different Directorates to be linked to various outputs

• Includes links to metadata maintained by other international organizations and national agencies

• Permits search and query• Will be linked to other elements of the OECD’s

corporate data environment – OECD.STAT and the OECD Glossary of Statistical Terms

• Goal: develop governance practices for accessing the repositories of other organizations

DSBB Content: 3 types of metadata

• Subscription to the SDDS has grown to 55 countries• Participation in the General Data Dissemination

System (GDDS): Metadata for 68 countries • Reports on the Observance of Standards and Codes:

Data Modules prepared using the Data Quality Assessment Framework (DQAF) for 29 countries—including 20 SDDS subscribers and 7 GDDS participants—on the Data Quality Reference Site (DQRS)

SDDS/GDDS/DQRS: the IMF’s main metadata repositories

• SDDS and GDDS metadata (18 data categories), which cover many of the same statistical domains as the metadata maintained by ECB and Eurostat

• SDDS and GDDS metadata may be queried—separately now—but soon as a common repository

• DQRS based metadata (ROSC reports) will soon become queryable—and mapped to the SDDS/GDDS metadata repository

PROBLEM: These three repositories are not harmonized against a common metadata model nor are they linked to other organizations’ repositories

SDDS Metadata Query FacilityThe DSBB allows users to tap into SDDS information in four ways:

• by Metadata Dimension(s) and Element(s) for one or more Countries and Data Categories• by Key Concepts within Metadata Element(s) for one or more Countries and Data Categories• View Advance Release Calendar information for one or more data categories and countries• View Summary of Observance information: cross-country practices vis-à-vis the SDDS

However, the metadata cannot be interrogated and repurposed to meet specific users’ needs.

Statistical Data and Metadata Exchange (SDMX: www.sdmx.org)

• ISSUE: How to achieve the open exchange of metadata within the IMF and among institutions?

• FOCUS: business practices in the field of statistical information that promote efficient processes for exchange and sharing of data and metadata

• GOAL: to establish common e-standards and standardization activities that result in: – gains in efficiency – avoiding duplication of effort in our own work and the

work of others in the field of statistical information.

SDMX Metadata Projects

1. Metadata Common Vocabulary– Development of a metadata terminology

standard focusing on descriptions of statistical concepts and methodologies used in the collection, processing, and dissemination of statistical data

• Improves consistency in the metadata compiled

• Can be used independently from any specific metadata model

• Facilitates information interchange

Metadata Common Vocabulary

• An example:Timeliness– Definition: Timeliness refers to the speed of dissemination of the

data - i.e., the lapse of time between the end of a reference period (or a reference date) and dissemination of the data. It reflects many factors, including some that are related to institutional arrangements, such as the preparation of accompanying commentary and printing.

– Source: International Monetary Fund (IMF), "Guide to the Data Dissemination Standards, Module 1: The Special Data Dissemination Standard", Washington, May 1996

– Hyperlinks: http://dsbb.imf.org/Applications/web/sddshome/ – Context: The third quality component in the Eurostat Definition is

"timeliness and punctuality".– Related term: Data, Quality, Punctuality

SDMX Metadata Projects

2. Metadata Repositories– Development of a platform-independent metadata

exchange system providing fluid and unobstructed access to information from a large number of content providers

• Improved metadata access, analysis, and repurposing

• A common model or structure for describing all metadata elements– Expressed in UML (Unified Modeling Language)

– Rendered using XML

– Linked to the MCV

Metadata RepositoriesSDMX Model – An Application to the SDDS

• SDDS – As it standsDIMENSION ELEMENTS KEY CONCEPTS

QUALITY Dissemination ofdocumentation on methodologyand sources used in preparingstatistics

Publication(s)

Dissemination of component detail, reconciliations with related data, and statisticalframeworks that supportstatistical cross-checksand provide assurance ofreasonableness

Publication(s)

Component breakdown -

Metadata Repositories SDMX Model – An Application to the SDDS

• SDDS – Revisions related to SDMXDIMENSION ELEMENTS Level 3 Level 4

QUALITY REFERENCES

Dissemination of reference documentation on methodological soundness and types of data sources used in preparing statistics

Title(s) and frequency of methodological publication (DQAF 5.2.1 & 5.2.2)

Hardcopy (ISBN)

Electronic (URL)

Dissemination of reference documentation that support statistical cross-checks and provide assurance of coherence and consistency over time

Title(s) and frequency of statistical publication (DQAF 4.2.2)

Hardcopy (ISBN)

Electronic (URL)

Consistency over time /Within the dataset (DQAF 4.2.1 & 4.2.2)

Breaks in time series data

Consistency with other indicators/data sets (DQAF 4.2.3)

Hardcopy (ISBN)

Electronic (URL)

Quality assurance references / Benchmark (ROSC/DQAF, etc.)

Hardcopy (ISBN)

Electronic (URL)

Toward an “SDMX” SDDS model

• Describes SDDS metadata concept families, building on the four dimensions of data characteristics, quality, access, and integrity, plus methodology and dissemination formats

• Prescribes—in four levels—greater detail and specificity in the metadata provided

• Promotes comparability and the sharing of content among providers and users

• Can be linked to other metadata systems and extended, using XML and a common terminology

• Serves as an input for the XML tagging of metadata across SDMX organizations– Facilitates information management across diverse systems– Enables full or partial metadata sharing

Issues for discussion

• Next phase: Combining outputs from all SDMX activities to refine and extend data/metadata exchange for selected “pilot” domains

• Strategies for migrating metadata from the “old” to the “new” system

• Does the SDMX approach parallel initiatives underway in national agencies? How can these efforts be harmonized?