SDMX and Metadata

SDMX Basics Course12 April 2013

Daniel SuranyiEurostat B5Management of statistical data and metadata

The main types of metadata

Structural metadata are:

acting as identifiers and descriptors of the data, such as:

• dimensions of statistical cubes• variables• titles of tables• navigation tree

Structural metadata must always be associated with the data to allow their identification, retrieval and browsing.

Example for structural metadata

The main types of metadata

Reference metadata are:

acting only as descriptors of the data, they don’t help to actually identify the data.

They can be of different kinds:

• conceptual metadata• methodological metadata• quality metadata (process and output)

Reference metadata can be exchanged in-dependently from the data they are related to, but are however often linked to them.

Example for reference metadata

Metadata and the ESS vision

The ESS vision is based on the Commission Communication 404/2009 “on the production methods of EU statistics: a vision for the next decade”.

Some main ideas of this vision are:

• From statistical ‘stove pipes’ to more integrated statistical production processes;

• Better integration of the ESS in terms of IT infrastructure, IT tools, data quality, metadata, methodology etc. (both in terms of horizontal and vertical integration);

• Broader use of administrative data sources in the statistical data production processes;

• Statistical legislation should also be cross-cutting in covering larger statistical domains (first cross-cutting legislation drafted).

Standardisation of structural metadata

• Code lists describe dimensions in data tables, giving a meaning to the data.

• Are based on

• official statistical classifications such as NACE, NUTS, ISCO…

• the SDMX Content Oriented Guidelines

• Domain specific codifcations

• A standard code list is a code list already harmonised

• Standard code lists should be used all along the statistical business process

• data design, collection, aggregation, dissemination, archiving…

8

Example of a harmonised code list (NACE Rev. 1.1)

 

Old version (before harmonisation)

New version(after harmonisation)

Domains Old codes Old label_en New codes New label_en

hrst, htec MA_TOTAL Manufacturing sector

D Manufacturing

fats MAN Manufacturing industries

theme3 RD Manufacturing industry

theme4 B0200 Manufacturing industry

theme8 SE0_4 Manufacturing industry

theme9 TOT_MANUF Manufacturing industry

ds, hrst, htec MA_LOW_TEC Low technology manufacturing sector

D_LTCLow-technology manufacturingfats / inn LOT

Low Technology (incl. following NACE codes: 15-22; 36, 37)

inn I_LOW_TECLow tech industries: NACE Rev.1 codes 15 to 22, 36 and 37

hrst, htec SE_TOTALServices: NACE Rev. 1.1 sections G to Q = 50 to 99 G-Q Services

fats SER Services sector

Impact on the statistical business processes

Better comparability: same codes for the same concepts

Increase efficiency: less transcoding; less code lists; clean lists

Improve accuracy: facilitate data management and exchange and reduce the number of errors

Re-usability and integration of the data: data warehouse are only possible if codes corresponding to the same concept are the same

SDMX implementation: it is essential for the implementation of a SDMX data/metadata exchange process.

The ESS standard code lists will also be made available in the Euro SDMX Registry(currently Ramon)

http://ec.europa.eu/eurostat/ramon

Standard Code Lists in RAMON

Standardisation of Reference Metadata

12

Standardisation of reference metadata

The Euro SDMX Metadata Structure (ESMS)

Standardisation of reference metadata

The ESS Standard for Quality Reports Structure (ESQRS)

Dissemination of reference metadata

Dissemination of national reference metadata

The ESS Metadata Handler

Common user Interface

Output produced for the Eurostat Web

Other output for Eurostat or external users

Metadata from the Eurostat domain manager

Eurostat as main administrator

NRME EMIS

RAMON CODED

ESS – Metadata HandlerE

uro

SD

MX

Reg

istr

y

Domain specific non harmonized metadata (inserted from the

Eurostat production databases)

Input from national metadata

Common user Interface

Output produced for the Eurostat Web

Other output for Eurostat or external users

Metadata from the Eurostat domain manager

Eurostat as main administrator

NRME EMIS

RAMON CODED

ESS – Metadata HandlerE

uro

SD

MX

Reg

istr

y

Input from national metadata

The National Reference Metadata Editor (NRME)

Impact on the statistical business processes

ESS reference metadata standards are integrated into the National Reference Metadata Editor for production, exchange and dissemination of reference metadata in the ESS, allowing:

+ More AUTOMATIC PRODUCTION of the reference metadata in the ESS.

+ Information collected ONLY ONCE and reused(ESQRS -> ESMS; NSI->Eurostat; Eurostat-> IMF/OECD).

+ More harmonised and better availability of metadata on quality.

+ Full SDMX compliance(metadata creation, exchange and dissemination).

+ Cost and resources savings in the ESS.

Questions?

20

• SDMX Support Team• ESTAT-Support-SDMX@ec.europa.eu

• SDMX Website• http://www.sdmx.org

• Eurostat SDMX Info Space• https://webgate.ec.europa.eu/fpfis/mwikis/sdmx