SDE 2014

UNITED NATIONSECONOMIC COMMISSION FOR EUROPE

CONFERENCE OF EUROPEAN STATISTICIANS

Work Session on Statistical Data Editing(Paris, France, 28-30 April 2014)HLG Modernization Committeeon Production and Methods

Steven Vale (UNECE)Claude Poirier (Canada)1

UNECE Economic Commission for Europe Introducing the HLGHigh-level Group for the Modernisation of Statistical Production and ServicesCreated by the Conference of European Statisticians in 2010Vision and strategy endorsed by CES in 2011/2012Who are the HLG members?Pdraig Dalton (Ireland) - ChairmanTrevor Sutton (Australia)Wayne Smith (Canada)Emanuele Baldacci (Italy)Bert Kroese (Netherlands)Park, Hyungsoo (Republic of Korea)GenovefaRui (Slovenia)Walter Radermacher (Eurostat)Martine Durand (OECD)Lidia Bratanova (UNECE)

What does the HLG do?Oversees activities that support modernisation of statistical organisationsStimulates development of global standards and international collaboration activitiesWithin the official statistics community ... take a leadership and coordination roleQuotes from the Terms of Reference of the HLG5Why is the HLG needed?Before the HLGNowMany expert groupsClear visionLittle coordinationAgreed prioritiesNo overall strategyStrategic leadershipLimited impactReal progress6What has the HLG achieved?2012Generic Statistical Information Model2013Common Statistical Production ArchitectureFrameworks and Standards for Statistical Modernisation2014 - Work in ProgressImplementation of the Common Statistical Production ArchitectureBig Data in Official StatisticsThese projects are chosen based on feedback from the CES and an annual workshop with representatives of around 25 international expert groups.

I will give a short summary of what as achieved in 2013, and what is currently in progress.7HLG Activities Engagement Map

Governance

SDEHLG Modernization Committeeon Production and MethodsThe membershipChairs: Marton Vucsan, Rune GlersenParticipants: Australia, Canada, Hungary, Ireland, Italy, Moldova, Netherlands, New Zealand, Norway, Poland, Spain, Eurostat, OECD, UNIDOThe 2014 work planCSPA, BIG Data, Statistical Services, Survey on current tools, Development of official statistics, Machine learning, Strategic development (interconnected world) 10Machine LearningOpportunities for Methodologists and IT specialists to co-workGet methodology vision with respect to modern servicesIdentify current methodology assetsArticulate methodology development around IT strengths

Influence IT initiative to enable state-of-the-art methods

Production Less manual interventions Machine learning Get ready for BIG Data

_______________________________

MACHINE LEARNING The initiative on Machine Learning will be an opportunity for methodologists to share their thoughts with IT specialists and to coordinate efforts to develop a vision. So methodologists will document their vision on where they think modern statistical services should be oriented. The current methodology assets (or state-of-the-art methods) will be identified. Well then try to see how we can articulate methodology development around IT strengths and vice versa : how IT initiatives can be influenced by new methods.

11Machine Learning (contd)Machine learning:Data editing Automated coding Record linkageHigh potential to reduce if not eliminate manual workEnvironmental scan to identify where we are atIdentification of local development plansRoadmap to identify where we should go

Participants:Canada, Australia, Netherlands,Hungary, Poland, UNIDO

Lead:Canada

_______________________________

MACHINE LEARNING Machine learning may apply to many processes. In the current context, wed like to maximise the cost-benefit of the initiative, thus focussing on data editing, automated coding, and record linkage. These three processes offer tremendous potential gain, especially with respect to reducing manual intervention in the processing of secondary data.

An environmental scan will provide us with the big picture: where are each NSO at with respect to editing, coding and linkage. The national development plans will be identified. Duplication of effort will be identified amongst NSOs, and a roadmap will be drafted to identify where we should go as a statistical community.

Interested:ABS, StatCan, UNIDO, Netherlands, Hungary, PolandLead:Claude Poirier (Canada)Priority:MUST, this can lead to quite unexpected results if done right.

12Machine Learning (contd)DeliverablesA report reviewing state-of-the-art machine learning methods and algorithms for editing, coding and linkage A development roadmap taking advantage of local initiatives while avoiding duplication of works, and being IT-realistic

ScheduleOctober 2014_______________________________

Deliverable 1: A report reviewing state of the art machine learning methods and algorithms relevant to automated coding, record linkage and editing "old" and "new" data.Deliverable 2: A development roadmap taking advantage of local initiatives while avoiding duplication of works, and being IT-realistic. This will list the most interesting developments in the field of machine learning and the most effective way for the NSIs to get there. This ultimate roadmap will be presented to the Executive Board to get its support. You may notice that, in this plan, there's nothing specific to data sources (social, economic, admin or big data). These areas will be considered when identifying gaps. With respect to timelines, we'd have to adhere to HLG expectations - this means some kind of a sprint.

Planning: Q3

13What does this mean for the future of the SDE group?

The Statistical Data Editing group has been around for many yearsSome successes, particularly sharing ideas and good practicesCould a different approach or a different format accelerate progress?14Some ideas

Silos not just for subject matter people?We should bring together:MethodologistsIT expertsInformation architectsData scientists....Multi-disciplinary approach to problem solving within organisations15So why keep silos for international meetings?

What sort of event would have the most value foryour organisation?