statistics estonia on its way to improving efficiency un ece seminar on new frontiers for...
TRANSCRIPT
Statistics Estonia on its way to improving efficiencyUN ECE Seminar on
New Frontiers for Statistical Data Collection
Geneva, 31.10‒02.11.2012
Tuulikki Sillajõe
Outline
Organization of the data collection Production system as a whole Conclusions and future plans
02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Data EntryService
Central Data Collection Department since 2004
Seminar on New Frontiers for Statistical Data Collection02.11.2012
Data CollectionDevelopmentManager
Data CollectionService
FieldworkOrganisationService
Data CollectionDepartmentManager
Collection of administrative data (I)
Data Collection Department is not responsible for collection of administrative data
Data Processing Systems Department is responsible for a single entry point for
administrative data runs pre-agreed data processing makes the data available for statistical domains
02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Collection of administrative data (II)
Methodology Department consolidates the needs of statistical domains conducts negotiations with the holders of
administrative registers organises the conclusion of agreements with
these holders is in charge of the description of data in a
central metadata system Statistics Estonia used about 100 different
administrative registers (2012)
02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Functionalities of ADAM (data collecting system for administrative data)
Automatic extraction of detailed personalized data from administrative sources using
X-road (data exchange layer) ftp, etc.
Storing data in raw data databases Data processing
coding duplicate removal
Making data available for in-house applications
02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Milestones of Data Collection in Statistics Estonia, 2000‒2012
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
HOUSEHOLDS
ENTERPRISES
ABOLISHMENT OF REGIONALBUREAUS
CENTRAL DEPARTMENT
CAPI
WEB-BASED COLLECTION (eSTAT)
CALL CENTER FOR ENTERPRISES
FAILURE WITH 1st CAWI
1st MIXED MODE CAPI+ CATI
TERMINATION OF SENDING Q.-S
PILOT PHC 2011 CAWI+CAPI
AGRI-CENSUS CAWI+CAPI
PRE-FILLING WITH ADMIN-DATA
PHC 2011 PAPER-LESS
OTHER CAWI PILOTS
02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Functionality of eSTAT for external users (I) to view the list of statistical questionnaires, which a
particular economic entity has to present to Statistics Estonia during the current year
to view deadlines for presenting these statistical questionnaires
to order reminders, which notify by e-mail about upcoming deadlines
to compile statistical questionnaires, i.e. to fulfil cells on the web with data or download and upload CSV-files
to run controls, i.e. check whether they have compiled the statistical questionnaires required
02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Functionality of eSTAT for external users (II) to correct statistical questionnaires immediately upon
compilation thereof to submit statistical questionnaires to look at all earlier statistical questionnaires
submitted to Statistics Estonia via eSTAT by a respondent concerned
to print out a paper copy of a compiled statistical questionnaire
to administer users, i.e. to create, change and cancel rights and access
to accept or correct one’s contact information02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Functionality of eSTAT for internal users (I) to define statistical questionnaires in the system, i.e.
to describe and add them to define controls for statistical questionnaires
described in the system to follow the inflow of statistical questionnaires and
send reminders to register contacts with respondents to see the time and the content of contacts with
respondents to see the same information as external user and
help them online02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Functionality of eSTAT for internal users (II) to see and correct contact information of economic
entity who can deliver questionnaires via the system to compile statistical questionnaires (e.g. when
receiving them by phone) to view and correct statistical questionnaires
compiled by respondents to administer external main users and internal users to create empty statistical questionnaires in pdf-
format for printing out or saving them as a file for different administrative purposes
02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Questionnaires received from economic entities by channel, 2008–2012
Seminar on New Frontiers for Statistical Data Collection02.11.2012
Pre-filling of questionnaires (I)
Annual statistical questionnaires for the year 2012, i.e. for the reference year 2011 data collection, are prefilled using administrative data
structural business statistics (EKOMAR) agriculture, forestry and fishing financial intermediation and activities auxiliary
to financial services and insurance activities non-profit institutions
Data providers have to fill in only the gaps, i.e. information not available from annual bookkeeping report
02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Pre-filling of questionnaires (II)
The information from annual reports is preloaded to eSTAT every hour
First results 52% of the questionnaires for structural business
statistics had been pre-filled from annual reports 80% of the fields were pre-filled, and respondents
had to fill in the remaining 20% (only fields filled in with a number other than zero are taken into account)
Average compiling time of a statistical questionnaire has been reduced twice (from 3 hours to 1,5 hour) compared to previous year
02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Mixed mode was used for PHC 2011
Modes of questioning e-Census on the web (CAWI) → 66% Interview (CAPI)
Institutions filled in a special questionnaire Different data sources
Population and Housing Census 2000 Administrative registers Data collection from persons
New software was developed next to eSTAT
02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Seminar on New Frontiers for Statistical Data Collection
New software for data collection (VVIS)
Supports data collection process for various surveys and census, dividing it into three sub-processes
Preparation work for data collection Data collection on the web (CAWI) and
fieldwork (CAPI) Support and management of the whole process
02.11.2012
Seminar on New Frontiers for Statistical Data Collection
Applications of VVIS
Questionnaire definition application Interviewers’ (enumerators’) application Public web application Management application Interfaces with external systems
02.11.2012
Seminar on New Frontiers for Statistical Data Collection
Questionnaire definition application
A tool for preparing questionnaires Built on top of Eclipse Functionality
Every survey can use a different, custom data model Several questionnaires can be built on top of data
model (e.g. different questionnaires for CAWI and CAPI)
Advanced navigation and validation rules within questionnaires
Use of common classifications in different surveys (e.g. occupations: ISCO, education: ISCED, etc.)
Data model and questionnaires saved in open XML format
Multi-language support for questionnaires
02.11.2012
Seminar on New Frontiers for Statistical Data Collection
Interviewers’ application
Stand-alone desktop application can be used in laptop or desktop computer (Delphi)
Synchronises all data for offline work over encrypted channel (HTTPS) Functionality
Advanced validation and navigation rules within questionnaire Navigation between different questionnaires Questionnaires in multiple languages Planning / scheduling of work Reminders Communication with direct supervisor Map info for location of subjects, GPS positioning, work
planning Overview of general fieldwork progress for interviewers Help information for interviewers Automatic software updates (distributed from central server)
02.11.2012
Seminar on New Frontiers for Statistical Data Collection
Public web application
Used by survey subjects independently Public system, accessible over the web (Java,
Weblogic) Functionality:
Authentication with ID card or through bank link Choice between questionnaires assigned to the
subject Support for all rules used in questionnaires, the
same questionnaires can be used that are defined for interviewers’ application
Background information and help texts about the surveys and questionnaires
02.11.2012
Seminar on New Frontiers for Statistical Data Collection
Management application
Used by fieldwork management, statisticians, help desk Internal web application (Java, Weblogic), accessible from within
internal network Functionality:
Authentication using LDAP or any other authentication method Creation of survey object, configuration of methodology,
fieldwork hierarchy and other characteristics of survey Role management Possibility to work simultaneously with several surveys Overview of fieldwork progress, task management Definition of milestones (deadlines) for fieldwork organisation Help desk functionality Data processing tools (e.g. for classification of data) Communication (messages from one user to another)
02.11.2012
Seminar on New Frontiers for Statistical Data Collection
Interfaces with external systems
Information database Meta-info about surveys Background survey information (displayed on the web) Classifications used in questionnaires
Authentication system (e.g. Active Directory) Statistical register
Import of sample Pre-filling of questionnaires Export of sample (changes in subjects’ information)
GIS database Maps Location info of buildings Hierarchy of district division
02.11.2012
Generic StatisticalBusiness Process Model
Seminar on New Frontiers for Statistical Data Collection02.11.2012
Architecture of the information system
Seminar on New Frontiers for Statistical Data Collection02.11.2012
Dissemination
Statistical registers
Metadata system
Data collection ProcessingStatistical analysis
Persons
Administrative registers
Users
Economic entities
Data Warehouse
iMETA
VVISADAM eGeostat
SRS
VAISeSTAT
PX-Web
Census-HUB
KUNDE
Analyse
Seminar on New Frontiers for Statistical Data Collection
2002Statistical
Farm Register
1993
1993
1994Statistical Business Register
1994economic entities
2004persons
2004
2004economic
entities
2006economic entities
2001metadata
management
2011iMeta
project started 2011 system for
statistical registers
project started2011
project started2011 persons
planned
Generic Statistical Business Process Model
project started
2012
02.11.2012
Sources for efficiency gains of Statistics Estonia Central data collection department, i.e.
standardisation of processes Generic office-wide software Administrative data instead of survey data if
applicable Pre-filling of statistical questionnaires with
administrative data
02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Small developments, big efforts
Reminders sent before the deadline instead of after the deadline
Informing economic entities simultaneously about all the questionnaires they have to fill in next year
Centralisation of the preparation of questionnaires from statistical departments to IT Department and within a few years to Methodology Department
Standardisation and simplification of instructions about questionnaires for economic entities
Creation of a list of input variables
Seminar on New Frontiers for Statistical Data Collection02.11.2012
Next practical steps
Introduction of CAWI as the main data collection method for surveys on individuals (2013)
Introduction of CATI for data collection from both types of respondents: economic entities and individuals (step by step, starting from 2012)
Implementation of generic office-wide software for other functions than data collection
Reuse of data within the statistical office
02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Strategic directions
Training of data suppliers (economic entities, individuals, registers, etc.), incl. about their personal and public gains
Closer cooperation between the data collection function and dissemination function within the organisation, for better communication with data suppliers (based on the experience of PHC 2011)
Simplification of statistical reports (harmonisation of concepts, deadlines, practices, etc.)
Development of infrastructure for selling data collection services
Seminar on New Frontiers for Statistical Data Collection02.11.2012
Further challenges
Wider use of administrative and commercial data Do we need two data collection tools? Centralization of data processing function?
02.11.2012 Seminar on New Frontiers for Statistical Data Collection
Thank you for your [email protected]
02.11.2012 Seminar on New Frontiers for Statistical Data Collection