modernization of statistical information systems global initiatives · 2015. 1. 30. ·...
TRANSCRIPT
-
1
Modernization of Statistical fInformation systemsGlobal initiatives
Eric Hermouet, Statistics Division, ESCAP
1
Presentation
Why modernization? What is modernization? What is modernization? How: Global initiatives and collaboration mechanisms Main outputs of those mechanisms
-
2
Modernization
Modernization of statistical information systems Industrialization of statistics Industrialization of statistics Modernization of statistical production and services
The data deluge
The internet hadThe internet had 1800 exabytes of data in 2011
Exabyte=1018
Or 1 million terabytesy
-
3
The data deluge
50 000 exabytes by 202050 000 exabytes by 2020
Even if only 0.1% of these data is useful, that leave millions of terabytes for possible statistical purposes
New data providers
Google – Real-time price indices– Real-time price indices– Public data explorer– First point of reference for the “data generation”
Facebook Telecom companies
– 4 billion mobile phones in Asia and the Pacificp
How can statistical offices access and use those data sources?
-
4
Changing user expectations
Expectation of data available at a faster rate– Real-time – Real-time
Ability to customize datasets– Linking coherently datasets across domains– With a high degree of detail
Data presentation addressing different target groups– Governments– General public
IT progress
IT developments offers new solutions– In processing huge amounts of data– In processing huge amounts of data– In accessing new sources of data
In a more efficient way– In terms of speed– In terms of financial costs
-
5
So what is modernization?
Common generic processes Common tools Common tools Common methodologies Recognizing that all statistics are produced in a similar way
– No special domains
Increased flexibility to adapty– To access new data sources– To generate new statistical products
-
6
A shared production environment
From To
Custom design Use of information bank andCustom design Use of information bank and modular design
Built solutions Assembled solutionsDesign for a collection mode Source independent design
“designed-in” quality Working with existing data and varying level of qualityvarying level of quality
Survey cycles with clear start and end points
Continuous approach of on-going collection, processing, and release
A shared production environment
From To
Direct data collection Tapping into existing dataDirect data collection supplemented with data from administrative source
Tapping into existing data, using direct data collection to link sources and bridge gaps
Individually crafted data structure
Use of agreed standard approaches
Management of data Management of data, metadata, and paradata
A large field workforce Smaller field workforce with specialized interviewing skills
-
7
A shared production environment
From To
Spending effort and resources More emphasis on specifyingSpending effort and resources on collection and processing
More emphasis on specifying needs, design, and analysis
Understanding user needs and designing collection instruments
Understanding data characteristics and negotiating solutions that bridges gaps between existing data and user requirementsuser requirements
What is modernization and why is it needed?
Questions so far ?
14
-
8
Global initiatives & collaboration mechanisms
15
Global initiatives & collaboration mechanisms
High-Level Group for the Modernization of Statistical Production and Services (HLG)Production and Services (HLG)
Formerly HLG-BAS Objectives
– To promote common standards, models, tools and methods to support the modernization of official statistics;
– To drive new developments in the production, d d f ff lorganization and products of official statistics, ensuring
effective coordination and information sharing within official statistics, and with relevant external bodies;
-
9
HLG
Gosse van der Veen (Netherlands) - Chairman Brian Pink (Australia) Brian Pink (Australia) Eduardo Sojo Garza-Aldape (Mexico) Enrico Giovannini (Italy) Mr Park Hyungsoo (Republic of Korea) Irena Križman (Slovenia) Katherine Wallman (United States)( ) Walter Radermacher (Eurostat) Martine Durand (OECD) Lidia Bratanova (UNECE)
HLG reference documents
Strategic vision– http://www1 unece org/stat/platform/display/hlgbas/S– http://www1.unece.org/stat/platform/display/hlgbas/S
trategic+Vision Strategy to implement the vision of the HLG
– http://www1.unece.org/stat/platform/display/hlgbas/HLG+Strategy
-
10
Technical groups - MSIS
Management of Statistical Information Systems (MSIS)– Objectives: j
• To provide, through regular meetings and other means such as the MSIS Wiki, a forum for exchange of experiences and good practices among information systems managers from national and international statistical organizations.
• To contribute to the coordination of activities of different national and international organizations in the area of statistical information systems.y
• To facilitate and encourage implementation of international standards and recommendations in the field of statistical computing among national and international statistical organizations.
Technical groups - MSIS
Annual meetings jointly organized by UNECE, Eurostat, OECD. And ESCAP for this 2013 MSISOECD. And ESCAP for this 2013 MSIS– MSIS meetings consider issues related to information
technology governance and management, system architecture, accessibility and usability.
– First meeting in 2000– Secretariat: UNECE
O ll UN d l Open to all UN countries and international organizations Reports to the Conference of European Statisticians
-
11
Technical Group - METIS
Steering Group on Statistical Metadata – METIS– Objectives:
P t th i l t ti f t d t t b d l i • Promote the implementation of metadata systems by developing advocacy targeting the senior management level and subject-matter staff of NSOs;
• Oversee the maintenance of the Common Metadata Framework (CMF) directing it towards a practical guide serving national statistical offices;
• Facilitate collection, discussion and dissemination of best practices in the field of statistical metadata
– Established in 1990Established in 1990– Work sessions every 2-3 years– Workshops in-between– Organized with Eurostat / OECD– Open to all UN member countries and international organizations
Technical Groups - SAB
Sharing Advisory Board– Objectives – Objectives
• To promote harmonization of business and information systems architectures;
• To support collaboration for the development of statistical software
• To provide guidelines and tools to assess new statistical f l dsoftware tools and components
• To assist in the improvement of the technical statistical infrastructure of countries both within and outside the UNECE region as required.
-
12
Technical groups - others
– Statistical Data editing– Statistical Data collection– Statistical Data collection
Not directly overseen by HLG – Statistical Network– SDMX Expert Group– Statistical Open Standards Group– Working group on quality in statistics g g p q y– ….
Main outputs
Emerging “statistical industry” standards– GSBPM– GSBPM– GSIM– SDMX– DDI
-
13
GSBPM
Generic Statistical Business Process Model– To define and describe statistical processes in a coherent – To define and describe statistical processes in a coherent
way– To standardize process terminology– To compare and benchmark processes within and between
organisations– To identify synergies between processes– To inform decisions on systems architectures and
organisation of resources
GSBPM
Applicability– All activities undertaken by producers of official statistics – All activities undertaken by producers of official statistics
which result in data outputs– National and international statistical organizations– Independent of data source, can be used for:
• Surveys / censuses• Administrative sources / register-based statisticsg• Mixed sources
-
14
GSBPM
Not a linear model Sub-processes do not have to be followed in a strict order Sub-processes do not have to be followed in a strict order It is a matrix, through which there are many possible paths,
including iterative loops within and between phases Some iterations of a regular process may skip certain sub-
processes
-
15
GSBPM
Examples of use– Harmonizing statistical computing systems – Harmonizing statistical computing systems – Facilitating sharing of statistical software– Framework for process quality management– Structure for storage of documents – Measuring operational costs
-
16
GSIM
GSIM contains objects which specify information about the real world – 'information objects‘ – Examples include data and
metadata (such as classifications) as well as the rules and parameters needed for production processes to run (for example, data editing rules) editing rules).
GSIM identifies around 150 information objects, which are grouped into four top-level groups
GSIM
Generic Statistical Information Model To describe data and metadata objects and flows within the To describe data and metadata objects and flows within the
statistical business process Implementation of GSIM, in combination with GSBPM, allow
– Creating an environment to prepare for reuse and sharing of methods, components and processes;
– Implementing rule based process control, thus minimizing h h dhuman intervention in the production process;
– Economies of scale through development of common tools by the community of statistical organizations.
-
17
SDMX and DDi
Standard Data and Metadata eXchange– UNSD/DFID project in Cambodia Laos Thailand Viet – UNSD/DFID project in Cambodia, Laos, Thailand, Viet
Nam Data Documentation Initiatives
– World Bank Microdata Management Toolkit
SDMX and DDI
-
18
What’s next?
Launch of “Plug and Play” project – February 2013– To create a common statistical production architecture To create a common statistical production architecture – To create a standardized architecture for statistical
production solutions, including processes, information and systems, coherent with the Generic Statistical Business Process Model (GSBPM) and the Generic Statistical Information Model (GSIM),
– To enable and advance the sharing of production processes t th d i tor components, thus reducing costs.
– To provide the basis for a central inventory or repository with life cycle management of sharable production processes and components.