![Page 1: Putting SDMX into practice - unescap.org concepts... · 4/30/2014 2 SDMX • Standard formats for data and metadata • Content guidelines • IT architecture for exchange of data](https://reader031.vdocument.in/reader031/viewer/2022021810/5c66ec1309d3f2e33b8ccb92/html5/thumbnails/1.jpg)
4/30/2014
1
SDMX – Concepts and toolsPutting SDMX into practice
ADB/ESCAP SDMX Capacity Building InitiativeBangkok, 28-30 April 2014
Why do we need SDMX ?
• Common language for statistical d t d t d t hdata and metadata exchange
– Variety of formats used within NSI
– International data exchange
• Common data transmission format for statistical data and metadata
![Page 2: Putting SDMX into practice - unescap.org concepts... · 4/30/2014 2 SDMX • Standard formats for data and metadata • Content guidelines • IT architecture for exchange of data](https://reader031.vdocument.in/reader031/viewer/2022021810/5c66ec1309d3f2e33b8ccb92/html5/thumbnails/2.jpg)
4/30/2014
2
SDMX
• Standard formats for data and metadata
• Content guidelines
• IT architecture for exchange of data and metadata
Organizations are free to make use of whichever elements of SDMX are most
appropriate in a given case
Statistical data
![Page 3: Putting SDMX into practice - unescap.org concepts... · 4/30/2014 2 SDMX • Standard formats for data and metadata • Content guidelines • IT architecture for exchange of data](https://reader031.vdocument.in/reader031/viewer/2022021810/5c66ec1309d3f2e33b8ccb92/html5/thumbnails/3.jpg)
4/30/2014
3
Dimensions, attributes and measures
• Dimensions describe the data and form a the id tifi (ID) f th l t d d tidentifier (ID) of the related data
• Attributes provide additional information to qualify the data (typically: unit, status of the data (provisional, estimate…)
• Measure: value of the phenomenon observedMeasure: value of the phenomenon observed
Code list
• Most of dimension values are defined in code li tlist
– Makes SDMX language independent
• Attributes can be defined in a code list or as free text
![Page 4: Putting SDMX into practice - unescap.org concepts... · 4/30/2014 2 SDMX • Standard formats for data and metadata • Content guidelines • IT architecture for exchange of data](https://reader031.vdocument.in/reader031/viewer/2022021810/5c66ec1309d3f2e33b8ccb92/html5/thumbnails/4.jpg)
4/30/2014
4
Example of code list
![Page 5: Putting SDMX into practice - unescap.org concepts... · 4/30/2014 2 SDMX • Standard formats for data and metadata • Content guidelines • IT architecture for exchange of data](https://reader031.vdocument.in/reader031/viewer/2022021810/5c66ec1309d3f2e33b8ccb92/html5/thumbnails/5.jpg)
4/30/2014
5
Metadata
• Structural metadata
– Concepts used in the description and identification of statistical data and metadata
• Reference metadata
– Additional explanatory metadata, for example onAdditional explanatory metadata, for example on the methodology used or quality aspects
Structural Metadata
• Identify and describe the data
• Corresponds to a dimension in a data cube
• Arrange in Structure Definitions
– Data Structure Definitions (DSD)
– Metadata Structure Definitions (MSD)
![Page 6: Putting SDMX into practice - unescap.org concepts... · 4/30/2014 2 SDMX • Standard formats for data and metadata • Content guidelines • IT architecture for exchange of data](https://reader031.vdocument.in/reader031/viewer/2022021810/5c66ec1309d3f2e33b8ccb92/html5/thumbnails/6.jpg)
4/30/2014
6
Reference Metadata
• Conceptual metadata, describing the concepts used and their practical implementationused and their practical implementation;
• Methodological metadata, describing methods used for the generation of the data;
• Quality metadata, describing the different quality aspects of the statistical dataquality aspects of the statistical data.
• Structured according to a Metadata Structure Definition (MSD)
![Page 7: Putting SDMX into practice - unescap.org concepts... · 4/30/2014 2 SDMX • Standard formats for data and metadata • Content guidelines • IT architecture for exchange of data](https://reader031.vdocument.in/reader031/viewer/2022021810/5c66ec1309d3f2e33b8ccb92/html5/thumbnails/7.jpg)
4/30/2014
7
Content oriented guidelines
• Guidelines with the scope of SDMX
– To achieve better interoperability between organizations
– Use is encouraged
• 3 areas
– Cross‐domain conceptsCross domain concepts
– Statistical subject‐matter domains
– A Metadata common vocabulary
Cross‐domain concepts
• List of statistical concepts related to statistical d d t litprocess and data quality
Example
![Page 8: Putting SDMX into practice - unescap.org concepts... · 4/30/2014 2 SDMX • Standard formats for data and metadata • Content guidelines • IT architecture for exchange of data](https://reader031.vdocument.in/reader031/viewer/2022021810/5c66ec1309d3f2e33b8ccb92/html5/thumbnails/8.jpg)
4/30/2014
8
Statistical subject‐matter domains
• high level classification of statistical areas
• starting point for organizing the exchange of statistical data and metadata
![Page 9: Putting SDMX into practice - unescap.org concepts... · 4/30/2014 2 SDMX • Standard formats for data and metadata • Content guidelines • IT architecture for exchange of data](https://reader031.vdocument.in/reader031/viewer/2022021810/5c66ec1309d3f2e33b8ccb92/html5/thumbnails/9.jpg)
4/30/2014
9
Metadata Common Vocabulary
• contains concepts and related definitions used i t t l d f t d t fin structural and reference metadata of international organizations and national data producing agencies.
– General metadata concepts
– Metadata terms describing statistical gmethodologies and data quality
– Terms referring specifically to data and metadata exchange
Metadata Common Vocabulary
![Page 10: Putting SDMX into practice - unescap.org concepts... · 4/30/2014 2 SDMX • Standard formats for data and metadata • Content guidelines • IT architecture for exchange of data](https://reader031.vdocument.in/reader031/viewer/2022021810/5c66ec1309d3f2e33b8ccb92/html5/thumbnails/10.jpg)
4/30/2014
10
IT Architecture
IT architecture
![Page 11: Putting SDMX into practice - unescap.org concepts... · 4/30/2014 2 SDMX • Standard formats for data and metadata • Content guidelines • IT architecture for exchange of data](https://reader031.vdocument.in/reader031/viewer/2022021810/5c66ec1309d3f2e33b8ccb92/html5/thumbnails/11.jpg)
4/30/2014
11
XML
• Extensible Mark‐up Language
• used to describe the content and structure of data in a document
• XML is not designed for use directly by people, but is instead intended to deliver documents to computer applications over the Internetto computer applications over the Internet.
• Uses tags to describe the meaning and hierarchical structure of data
![Page 12: Putting SDMX into practice - unescap.org concepts... · 4/30/2014 2 SDMX • Standard formats for data and metadata • Content guidelines • IT architecture for exchange of data](https://reader031.vdocument.in/reader031/viewer/2022021810/5c66ec1309d3f2e33b8ccb92/html5/thumbnails/12.jpg)
4/30/2014
12
Web service
Mapping process
• Data described differently by data providers d b d t ll tand by data collectors
• Need to harmonize structural metadata
• Mapping is necessary when local concepts are different from the corresponding concept in global DSDsglobal DSDs
![Page 13: Putting SDMX into practice - unescap.org concepts... · 4/30/2014 2 SDMX • Standard formats for data and metadata • Content guidelines • IT architecture for exchange of data](https://reader031.vdocument.in/reader031/viewer/2022021810/5c66ec1309d3f2e33b8ccb92/html5/thumbnails/13.jpg)
4/30/2014
13
Ressources on SDMX
• www.sdmx.org
• https://webgate.ec.europa.eu/fpfis/mwikis/sdmx/index.php/Main_Page
SDMX Reference InfrastructureSDMX‐RI
• Universal framework for modern data provision d hand exchange
• Set of pick‐and‐choose reusable building blocks allowing a statistical office to expose data to the external world based on access rights
• Designed to provide data and structuralDesigned to provide data and structural metadata based on mappings to each organization's dissemination data warehouse
• Uses SDMX standards incl. one for Web Services
![Page 14: Putting SDMX into practice - unescap.org concepts... · 4/30/2014 2 SDMX • Standard formats for data and metadata • Content guidelines • IT architecture for exchange of data](https://reader031.vdocument.in/reader031/viewer/2022021810/5c66ec1309d3f2e33b8ccb92/html5/thumbnails/14.jpg)
4/30/2014
14