SDMX at United Nations Statistics Division

Experience, Challenges, Strategies

2

Background United Nations Statistics Division is an SDMX

Sponsor agency Has used SDMX in its business processes

since 2005 (UN Comtrade) Participates in the development of global Data

Structure Definitions (DSDs) Uses SDMX in dissemination

3

Development of Global Data Structures Under the auspices of Interagency Expert

Group on Millennium Indicators, led the development of DSD for MDG indicators

Participates in the development of DSD for National Accounts

Leads the development of DSD for International Merchandise Trade Statistics

Development of DSD for Demography is in the initial stages

4

SDMX Exchange Exchange of IMTS data between UNSD and OECD Pilot project underway to exchange National

Accounts data with IMF, Eurostat, OECD; full-scale exchange to be established after the pilot

As part of the UNSD-DfID CountryData project, exchange of development indicator data was established with 11 countries Several non-project countries have also joined the

exchange The system has been in production since 2011 and

receives new data on a regular basis SDMX-related modules (mapping, registry) were

developed for the DevInfo software

5

SDMX Dissemination Commodity Trade web site has disseminated

international merchandise trade data since 2005

CountryData disseminates development indicator data supplied by approximately 100 countries

Implementation of SDMX is underway at UNdata, UNSD’s flagship dissemination platform 6 UNdata datasets are currently available,

including World Population Prospects and Millennium Development Goals

Eurostat’s SDMX Reference Infrastructure is used as the engine

6

SDMX Issues: Technology Extensibility

Countries very often wish to extend a standard DSD to customize it for their own needs, while some classifications explicitly provide for user-defined codes.

Metadata In terms of tools available, limited support for

metadata exchange in versions prior to SDMX 2.1 Metadata is often stored and maintained outside

of a database, which complicates the automation of metadata exchange.

7

SDMX Issues: National Statistical Office Lack of IT expertise

Many of the developing countries are unfamiliar with SDMX or underlying technologies, as well as with the principles of data modeling.

Lack of established aggregated data processing and dissemination Many countries do not use databases to store aggregated

data, which complicates both reporting and dissemination. Standards

Not all countries adhere to the latest versions of international classifications.

Lack of coordination between IT and Statisticians SDMX is often treated as a technology project, and

statisticians are not involved to the extent required – or at all.

8

SDMX Exchange Issues: International Agency Complexity of data structures

Global Data Structure Definitions can be very comprehensive but also difficult to have the countries adopt and use them.

Mapping errors Can lead to issues such as incorrectly coded or

duplicate series, etc. This is a problem even for moderately complex DSDs.

Uncertainty Scepticism about SDMX applicability at the global

level given its perceived complexity

9

SDMX Exchange Solutions:Technology Further refinement of the standard

Improve support for modern and emerging technological standards.

Better documentation Better support for metadata New tools oriented at beginners Implementing SDMX support for existing tools

10

SDMX Exchange Solutions: National Statistical Office Capacity building with assistance from

international agencies Improving data processing and dissemination

Formulating and implementing aggregated data dissemination strategy

Using standard tools and platforms As a way to lower the barrier to entry

11

SDMX Exchange Solutions:International Agency Address the complexity of comprehensive

data structures IMTS Working Group (UNSD, Eurostat, OECD,

UNCTAD and ITC) broke data exchange into manageable data-flows, which results in less complexity than a single overarching DSD would create.

Support capacity building at national level Support further development of the standard Support further development of tools

12

SDMX Exchange: Recent Developments Many capacity building projects underway (Eurostat,

IMF, UNSD, ADB, …) New tools greatly simplify dissemination and help

lower the costs of implementation SDMX Reference Infrastructure, Fusion Registry, DevInfo

Eurostat’s SDMX InfoSpace is an excellent resource to help master the technology

Global DSDs gradually become available Greater awareness of SDMX and willingness to

participate in exchange on the part of both countries and international agencies

Rapidly growing dissemination Global SDMX Registry

13

SDMX Exchange: The Future It is expected that SDMX Exchange will be

established for Post-2015 development indicators, both with international agencies and countries Building on MDG and CountryData experience Opportunities for capacity building and increasing

SDMX adoption rates at country level Brings to the fore sub-national geography issues

Gradual implementation of exchange with Member States Reducing reporting burden Eurotrace software (used by 80 developing countries)

planned to be adapted for SDMX Exchange in International Merchandise Trade Statistics

14

SDMX Exchange: The Future (2) Expanding dissemination

Not only popularizes SDMX but enables and encourages use of advanced technologies such as API/Web services

Expanding exchange using the DevInfo software, available for over 100 countries SDMX tools developed for the UNSD-DfID project

are now part of the standard DevInfo package Explore the possibilities for the use of SDMX in

the context of Data Revolution

15

Thank you!