![Page 1: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/1.jpg)
2. An overview of SDMX(What is SDMX? Part I)
1
Edward CookEurostatUnit B5: “Central data and metadata services”
SDMX Basics course, 27-29 October 2015
![Page 2: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/2.jpg)
Eurostat
• A model to describe statistical data and
metadata
• A standard for automated communication from machine to machine
• A technology supporting standardised IT tools
What is SDMX?
2
for which statisticians agree to use common descriptions and guidelines
driven by these common descriptors for all to reuse
developed as wide-ranging open source software
![Page 3: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/3.jpg)
Eurostat
• The SDMX Information model:
What is the information model underlying the data and metadata exchange between the partners?
• Content-oriented guidelines:
How to increase the interoperability and statistical harmonisation?
• IT Architecture for Data Exchange:
How to exchange the data?
Presentation of SDMX
3
![Page 4: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/4.jpg)
Eurostat
THE INFORMATION MODEL
4
![Page 5: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/5.jpg)
Eurostat
The Information Model:
… is a representation of concepts, relationships, constraints, rules and operations.
… is a formal way to:- express and design information needs- communicate with IT people- give specifications to reporting agents- document the system- drive the software
5
![Page 6: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/6.jpg)
Eurostat
What things does SDMX need to model?
• Statistical data• Through descriptor concepts. These concepts can be
further classified into dimensions, attributes and measures.
• Metadata• Structural metadata (such as concept names etc.)• Reference (or explanatory) metadata
• Data exchange processes 6
![Page 7: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/7.jpg)
Eurostat
Modelling statistical data in SDMX
7
![Page 8: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/8.jpg)
Eurostat
8
![Page 9: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/9.jpg)
Eurostat
Key SDMX object concerning data:
Data Structure Definition (DSD)
• Identification of dimensions, attributes and measures
• Use of common codelists
• Integration into concept schemes
9
![Page 10: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/10.jpg)
Eurostat
10
Modelling reference metadata in SDMX
So much descriptive information. It needs to be expressed in a common, standard way.
Quality descriptions
Process descriptions
Methodological descriptions
Administrative descriptions
![Page 11: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/11.jpg)
Eurostat
The standard way is the Metadata Structure Definition (MSD)
A Metadata Structure Definition describes how metadata sets, containing reference metadata are organised.
In particular, it defines:- which metadata are being exchanged; - how these concepts relate to each other;- how they are represented (free text or coded values);- with which object types (agencies, data flows, data
providers, subsets of data flows, or others) they are associated.
11
![Page 12: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/12.jpg)
Eurostat
12
Modelling reference metadata in SDMX
![Page 13: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/13.jpg)
Eurostat
13
![Page 14: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/14.jpg)
Eurostat
THE CONTENT-ORIENTED GUIDELINES
14
![Page 15: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/15.jpg)
Eurostat
Content-oriented guidelines
• The content-oriented guidelines are a set of recommendations within the scope of the SDMX standard in order to produce maximum interoperability.
• The SDMX standards:- provide essential support to statisticians;- maximise the amount of information through to
users;- allow an automation of the process;- allow web-service queries.
15
![Page 16: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/16.jpg)
Eurostat
There are three main areas in the content-oriented guidelines:
1. Statistical subject-matter domains.
2. Cross-domain concepts (and code lists).
3. A Metadata Common Vocabulary.
16
![Page 17: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/17.jpg)
Eurostat
Statistical subject-matter domains
• Statistical subject matter domains is a high level classification of statistical areas.
• They refer to statistical activities that have common characteristics with respect to variables, concepts and methodologies for data collection.
• Examples: price statistics, national accounts, environment statistics or education statistics.
• It is intended to cover the universe of official statistics.
17
![Page 18: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/18.jpg)
Eurostat
Functions of the classification of statistical domains.
• A standard against which domain lists of national and international organisations can be mapped to facilitate the exchange of data and metadata.
• Provides an identifier for registering and searching statistical data on SDMX registries.
• Navigation aide for the identification and organisation of corresponding domain groups.
18
![Page 19: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/19.jpg)
Eurostat
19
![Page 20: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/20.jpg)
Eurostat
Cross-domain concepts
• They are a list of statistical concepts, related to statistical processes and data quality.
• The list is based on the concepts used by the contributing international organisations.
• The concepts can be used at the data side as well as at the metadata side.
20
![Page 21: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/21.jpg)
Eurostat
21
![Page 22: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/22.jpg)
Eurostat
Examples of cross-domain concept
22
![Page 23: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/23.jpg)
Eurostat
Examples of cross-domain concept
23
![Page 24: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/24.jpg)
Eurostat
• A cross-domain concept may have a code list as presentation.
• This means that the concept might take a limited set of possible values enumerated in its corresponding code list.
• The code lists associated with cross-domain concepts are called cross-domain code lists.
24
![Page 25: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/25.jpg)
Eurostat
• Code lists have a general description, a list of codes, their description and annotations that provide additional information on the codes.
• Examples of cross-domain concepts and code list:• FREQ and its associated code list CL_FREQ.• SEX and its associated code list CL_SEX.
25
![Page 26: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/26.jpg)
Eurostat
26
![Page 27: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/27.jpg)
Eurostat
27
![Page 28: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/28.jpg)
Eurostat
Metadata Common Vocabulary
• The Metadata Common Vocabulary (MCV) is a vocabulary that recommends a common terminology to be used in order to facilitate communication and understanding
• The MCV is closely linked to the cross-domain concepts as it also contains all these concepts, stating their definitions and context descriptions.
28
![Page 29: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/29.jpg)
Eurostat
Metadata common vocabulary
• The MCV covers a selected range of metadata concepts:• General metadata concepts.• Metadata terms decribing statistical
methodologies and data quality.• Terms referring specifically to data and metadata
exchange.
29
![Page 30: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/30.jpg)
Eurostat
Examples of Metadata Common Vocabulary
30
![Page 31: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/31.jpg)
Eurostat
Examples of Metadata Common Vocabulary
31
![Page 32: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/32.jpg)
Eurostat
IT Architecture for data exchange
32
![Page 33: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/33.jpg)
Eurostat
• Standard formats for the exchange of data and metadata.• SDMX-EDI• SDMX-ML
• Architectures for data exchange:• Push• Pull• Data-hub
• SDMX Tools
33
![Page 34: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/34.jpg)
Eurostat
Push mode
34
![Page 35: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/35.jpg)
Eurostat
Pull mode
35
![Page 36: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/36.jpg)
Eurostat
Data Hub
36
![Page 37: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/37.jpg)
Eurostat
SDMX tools
• Eurostat tools at our SDMX Info Space https://webgate.ec.europa.eu/fpfis/mwikis/sdmx/index.php/Main_Page• SDMX Registry (a central repository for storing and sharing
SDMX artefacts). • SDMX Data Structure Wizard (used to create, edit and test
SDMX artefacts).• SDMX Converter (converts data files between SDMX formats
and other file formats). • SDMX Reference Infrastructure (SDMX-RI) (set of tools
that allows to connect your IT systems to the SDMX world). • SDMX Mapping Assistant (mapping and transcoding of the
contents of an existing database to SDMX data structures).
37
![Page 38: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/38.jpg)
Eurostat
SDMX toolsOther tools available in the community (
www.sdmxtools.org)
38
![Page 39: 2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 27-29 October 2015](https://reader035.vdocument.in/reader035/viewer/2022070413/5697bfe21a28abf838cb4754/html5/thumbnails/39.jpg)
Questions?