electronic data collection system in csb of latvia by karlis zeila, vice president, csb of latvia it...
TRANSCRIPT
Electronic data collection Electronic data collection
SystemSystem in CSB of Latvia in CSB of Latvia
By Karlis Zeila, Vice President, CSB of Latvia
IT DG meeting, October 24-25 2005,
Eurostat
The analysis of classical situation
ISDMS – Integrated statistical data management system
Post
Central Statistical Bureau Respondents
ISDMS
Operation with paper questionnaires consists of procedures listed below:
• Questionnaire design using MS Word;• Questionnaires printing - resources consuming procedure;• Ensuring of the pre-printing process of selected
guestionnaires;• Sending questionnaires to the respondents by post;• All incoming questionnaires have to be registered and
reminder letters prepared and sent out to non responded units. • Collected data from paper questionnaires has to be retyped
into CSB Data Management System, data checking procedures have to be done;
• More than 120 persons from total of 540 CSB employees ensure timely execution of the processes listed above.
Problems
• Expensiveness of the postal services
• Data has to be retyped from paper questionnaires
• The received questionnaires have to be checked and analysed
• Rather unefficient control and tracing
• Time and resources consuming process
Electronic Data Collection aproaches
Data carrier
Communication channel
Data security options
Process control & management options
Floppy disks or
CD
Ordinary mail service in the Post office
Use of encryption software on both CSB and respondent sides
Not available
e-mail message
with attachme
nts
e-mail system via internet connection
Use of encryption software on both
CSB and respondent sides
Low
www Internet connection, with specific software
installed on the PC work station
www provided security
(SSL)
High
General requirements EDC System
•EDC system should be integrated in the Integrated Statistical Data Management System;
•Layout of web forms has to be as much similar to paper questionnaires as it is possible, to ensure simple transition to web based data submission for the respondents; •Functionality of the EDC system particularly on the respondents side has to be very advanced to raise up interest to use it instead of classic paper forms;
•All processes have to be metadata driven from common DMS metadata base and automated as much as possible;•The same design tool MS Word for both paper and web forms has to be used
Solution
e-SurveyISDMS
internet
Central Statistical Bureau Respondents
Web data collection:advantages:
for respondent:
the possibility to see, analyze and edit the provided data for the previous periods,
it is possible to enter the data gradually,
it is possible to run validation procedures,
no postal expenses are required.
for CSB:
less postal expenses,
no manual data entry required,
increased data quality, because primary data control has already been done by respondent,
flexible system of automatic reminders sending to respondent gives possibility to rise response rate.
ISDMS architecture
Integrated statistical data management system
Corporative data Warehouse CSB Web Site
Macrodata base
Metadata base
Microdata base
Registers base
OLAP data base
User adminis-
tration data base
Dissemi-nation data
base
Windows 2000 Server Advanced MS Internet Information
Server SQL server 2000,
PC-AxisISDMS Business application Software Modules
Core metadata base modulerelated with DB:
Registers module
related with DB:
Data entry and validation module
related with DB:
Data aggregation module
related with DB:
Data analysis module
related with DB:
FIR
EW
AL
L
METADATA
USER ADMINISTRATION
REGISTERS
USER ADMINISTRATION
METADATA MICRODATA REGISTERS
USER ADMINISTRATION
METADATA MICRODATA REGISTERS
USER ADMINISTRATION
OLAP
METADATA
MACRODATA
Raw data base
Data dissemination
modulerelated with DB:
Data WEB entry module
related with DB:
Data mass entry module
related with DB:
Missed data imputation module
related with DB:
METADATA MACRODATA REGISTERS
USER ADMINISTRATION
METADATA MICRODATA REGISTERS
USER ADMINISTRATION
METADATA MICRODATA REGISTERS
RAW DATABASEUSER
ADMINISTRATION
METADATA MICRODATA REGISTERS
DATA IMPUTATION SOFTWARE
User administration module
related with DB:
METADATA MICRODATA MACRODATA
USER ADMINISTRATION
INTEGRATED, METADATA DRIVEN STATISTICAL DATA MANAGEMENT SYSTEM (IMD-SDMS)
Statistical metadata(structured and unstructured)
Dataentryfrompaperform
MICRODATABASE
Web rawdata base
ACTIVE structured statistical metadata for ISDMS1.VARIABLE=INDICATOR + ATTRIBUTE (CLASSIFICATIONS)2.QUESTIONNAIRE,TABLE,RowColumn Code3. METHODOLOGICAL MATERIALSETC
Structured and Unstructured Statistical metadata for Dissemination1.STATISTICAL DOMAINS2.BREAKDOWNS, CLASSIFICATIONS3.INDICATORS (basic and derived)
Datavalidation
Microdataanalysis
Dataagregation
MACRODATABASE
Macrodataanalysis OUTPUT
DATA
PC AXISWEB
modules
Statisticaldata from
othersources
Datagathering
Post
WWW
Data entry management Data aggregation and analysemanagement
Data dissemination management
Search
e-Clients
Publication inpaper form
e-Publication
Met
adat
a fo
r d
ata
dis
sem
inat
ion
Mea
tdat
a fo
r d
ata
pro
cess
ing
RESPONDENTS
CSB PersonnelCSB Clients - Respondents
CSB clients - data users
OLAP
Respondents relatedmetadata:1. Description of surveys2. Explanatory notes etc
METADATABASE
BusinessRegister
IntrastatRegister
Refers to selection property of the Objects
Summarized values of variables Crosclassififiying variables Time parameters
System architecture
e-Survey
meta data base
raw data base micro data base
1MS WordTemplate
2
4
5
6
ASP
ASP
XML
HTML
XML
ISDMS3
1. Guestionnaire design(MS Word)
2. Transfer to HTML
3. Linking to Metadata base
4. Filling in the data on respondents site(ASP)
5. Data transfer to CSB Raw data base
( XML)
6. Data checks and transfer to Microdata base(XML)
Technical platform and software:
Servers with Windows 2000 Advanced Server operating system
Microsoft products:
MS SQL Server2000 Enterprise Edition
MS Access2000
MS Word 2000
Active Server Pages WEB solution
Electronic data collection system is developed using ISDMS metadata base.
Electronic data collection system consists of the following modules:
1. HTML forms generation from Word documents and their publication to a WEB server,
2. Module, which provides with the data exchange between respondent and CSB (including the data entry and validation),
3. Respondents, questionnaires and data administration module.
LINKIG e SURVEY in live
http://www.csb.gov.lv/
Data entry and validation
Data transfer to Microdata
Base
Description of data entry
forms
Description of validation
rules
Standard data entry and validation
Creating list of Respon-
dents
MICRO DATA BASE
RAW Web
DATA BASE
META DATA BASE
BUSINESS REGISTER
Mass data entry
Web data entry and validation
RAW DATA BASE
Data validation
Web Data validation
F i r e w a l l
Data import from files
Full data validation
CONCLUSIONS
User interface is very user friendly and does not require special training neither on respondents’ side nor on CSB side,
Being developed as a part of Data Management System it can operate only with surveys described in the DMS common metadata base. In DMS there are 67 business statistics surveys at the time being,
Use of MS Internet Explorer is a restriction, a lot of enterprises are moving to open source software usage (Mozilla for instance),
CONCLUSIONS
Sometimes respondents are suffering from unstable work of the communication channels or Internet services providers,
Hot telephone help desk had to be established in CSB and system administrator takes over the technical assistance functionsIt is not possible to fill in the same web form from several workstations or by several persons on respondent’ s side simultaneously.
