s t a t i s t i c s a u s t r i a e-quest "new" 1 frederick rennert june 2008 ©...

S T A T I S T I C S A U S T R I Ae-Quest "new" 1www.statistik.at

Frederick Rennert

June 2008© STATISTICS AUSTRIA

i n f o r m a t i o n

Moving

“e-Quest”the next generation

S T A T I S T I C S A U S T R I Ae-Quest "new" 2

Overview

e-Quest concepts e-Quest „classic“ – the well-loved original eQuest/Web – spreading the word e-Quest „new“ – plans for the future e-Quest „new“ – prototype demo


Survey

•periodicity•communication information•observation unit types•hierarchies•questionnaire types

question 1

question 2

question 3

Economic Survey

question 1

question 2

question 3

Economic Survey

question 1

question 2

question 3

Economic Survey

Concepts

Collector of Statistical Information

CSI

Observation Unit(enterprise)

Observation Units

(establishments)

Respondent

question 1

question 2

question 3

Economic Survey

Questionnaires

question 1

question 2

question 3

Economic Survey

question 1

question 2

question 3

Economic Survey

question 1

question 2

question 3

Economic Survey


Metadata

Survey

•periodicity•communication information•observation unit types•hierarchies•questionnaire types

question 1

question 2

question 3

Economic Survey

question 1

question 2

question 3

Economic Survey

question 1

question 2

question 3

Economic Survey

Collector of Statistical Information

CSI

Observation Unit(enterprise)

Observation Units

(establishments)

Respondent

question 1

question 2

question 3

Economic Survey

Questionnaires

question 1

question 2

question 3

Economic Survey

question 1

question 2

question 3

Economic Survey

question 1

question 2

question 3

Economic Survey

Structuralmetadata

XML

Questionnairedefinition

XML

Respondentstructure

XML

Respondentinitialisation

XML


Survey preparation & definition

e.g.Survey, VersionPeriodicity, Observation unit types Questionnaire:• content, • representation,• layout,• control flow,• validation

Resp. specificinformationResp.-structureInitialisation data

Respondent

Incoming inspection

Further processing

Complete questionnaire

ST.AT

Phases of data collection


Questionnaire Manager

e-Quest

The WindowsTM Solution: e‑Quest

CSI

CD-ROM

structuralmetadata

(XML)

Responsepackage

(XML) export(XML)

import(XML)

Deploy

Respondent-specific

(meta)data(XML)

[mainframe]

Receive DB

Package Storage

Respondent

MetadataManager

e-Quest follow-up processing systems

Receive

e-Quest

Distributor

Hoover

Package Mgr

Questionnaire Manager


e-Quest „classic“ facts and statistics

created 1999-2001, with later additions Microsoft™ Visual Basic 6 code, some C++ about 125.000 lines of code for e-Quest proper, another

150.000 lines in related Statistics Austria internal applications about 1500 installations with 1700 users used mainly for the larger enterprise surveys (Short Term,

Structural Business, Production Input and Cross-Border Services surveys)

between 2% and 15% (Short Term survey) of all reported questionnaires


e-Quest advantages

rich interactive functionality, high performance control flow and validation by freely programmable (VB) scripts full interpretation of questionnaire XML definition, no specific code per

questionnaire (except for the scripts embedded in the XML) many optional functions (local user management, client/server installation,

display options, etc.) satisfied respondents (67% of e-Quest users who have tried both systems

want to stay with e-Quest)

Q33: Do you plan to move to eQuest/Web in the near future?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

1

yes

no

w e have alreadymoved to eQuest/Web


e-Quest drawbacks

Visual Basic 6 is nearing the end of its lifecycle VB .net is not compatible with VB 6 only one person can give technical support for the full system e-Quest relies on a relational database (MSDE) – most of the

technical problems have to do with DB installation distribution offline by CD-ROM to every prospective respondent

(up to 60.000 per year): costly and makes updates difficult e-Quest only runs on Microsoft™ systems (most on Win XP)


Respondent-specific

(meta)data(XML)

[mainframe]

Respondent

Browser

follow-up processing systems

eQuest/WebGenerator

HTML, JSP JavaScript, Java Web-Services

ST.AT

structuralmetadata

(XML)

e-QuestMetadata Manager

Framework

Questionnaire Logic

Metadata

e-Quest / WebData

RDBMS

SQL

Intra

net

Internet

Browsere-QuestReceive

Subsystem

op

tion

al

Web server

The Web Solution: e-Quest/Web


eQuest/Web facts and statistics

Web-based forms in XHTML created 2003-2004, continuous further development Java code, JSP pages about 250.000 lines of manual Java code, 900.000 of generated

code central database (IBM DB2 V8) about 500.000 user IDs, of which about 30.000 are used

monthly or yearly, another 100.000 at least once used for an increasing number of surveys which conform to the

standard questionnaire model; several special cases between 25% and 75% of all reported enterprise questionnaires,

up to 100% in agricultural surveys (with no paper or e-Quest alternative)


eQuest/Web advantages

can be used on any PC with a Web browser standard XHTML with no browser-specific code good performance central server makes updates (software and data) simple based directly on the e-Quest XML questionnaire definitions Web form generator generates database definitions, Java code for standard

control flow and single-field validations, and JSP pages JSP pages are „prettified“ by hand; non-standard control flow and multi-field

validations are hand-coded in Java adheres to Austrian e-government standards, relatively barrier-free Web

pages reported data is processed further by the standard internal e-Quest

applications setting up a one-time Web questionnaire with eQuest/Web often costs less

than doing the same with e-Quest „classic“


XML

XML

Receive („Pot“) appl.

Respondent

eQWebData

eQWebGenerator

Metadata

eQWebData

e-Quest CD

Integrated data flows

MetadataManager

XML

paper

formRespondent

filled-inform

MainframeDB

Design


eQuest/Web drawbacks

reliance on standard browser functionality implies less interactivity

e-government standard prohibits reliance on Javascript, thus no Ajax etc.

surveys with large numbers of respondents or a short reporting period put strain on Web server and database

Web session timeout bothers many respondents, but filling in the forms offline is impossible.


Conclusions

e-Quest is most useful for large periodic surveys and for complicated questionnaires,

but installing it is a hassle, so it is only used for the large enterprise surveys,

and it is nearing the end of its lifecycle. eQuest/Web is most useful for one-time (or long-period)

surveys, but creating richly interactive questionnaires is difficult. Also, the „offline“ variant has a certain fan club. So, how can we combine the best of both worlds?


Proposal for e-Quest „new“

When creating the „new“ e-Quest, it must– integrate even better with eQuest/Web

• using the same programming language (Java)• using the same generator• using the same questionnaire database• using the same authorization database• using identical XML schemata

– be even simpler to use• without needing a local database• with all unused or little-used functions stripped away

– obtain software, metadata and data online• only what is needed when it is needed

but must offer comparable questionnaire function and flexibility to e-Quest „classic“

– plus the goodies which eQuest/Web offers (PDF conversion, XML schema-based import/export, etc.)

and the whole system must be migrated step by step, in an organic manner.


Repository (file or database based)

structural metadata ClassificationsQuestionnaire data

Application framework (RCP)

generalfunctions

XML-processing

PersistencyFramework

baseGUIBase

PDF-funct.

Classificationhandling

Application coordination,logging,

authorization,navigation

Questionnaire framework

Questtype

A

Questtype

B Commonfunctions for

Survey C

QuestC1

QuestC2

QuestC3

Com

mun

icat

ion*

…

* Software download/update, download of structural metadata, classifications and questionnaires, questionnaire upload (report or import into the open Web questionnaire)

e-Quest „new“: Architecture based on eclipse RCP (schematic)


Repository (file or database based)

structural metadata ClassificationsQuestionnaire data

Application framework (RCP)

generalfunctions

XML-processing

PersistencyFramework

baseGUIBase

PDF-funct.

Classificationhandling

Applicationcoordination,

logging,authorization,

navigation

Questionnaire frameworkparadigm 1

Questtype

A Commonfunctions for

Survey C

QuestC1

QuestC2

QuestC3

Com

mun

icat

ion*

* Software download/update, download of structural metadata, classifications and questionnaires, questionnaire upload (report or import into the open Web questionnaire)

e-Quest „new“: extensible function through plugins

Questionnaire frameworkparadigm 2

Questtype

X

Questtype

Y…


e-Quest „new“ - activities

Investigation of technical options Creation of a prototype on the basis of those findings Survey of e-Quest „classic“ users

– what functions are used

– what preferences they have

– how they compare e-Quest and eQuest/Web

Tentative project plans – in progress Preparation of the decision – in progress


e-Quest „new“ - scenario

Login with eQuest/Web user ID and password– check the central DB for questionnaires for this user

– download questionnaires with initial data as XML

– download all plugins required for these questionnaires: questionnaire definition XML, help system, classifications (as XML), control and validation software for the questionnaire types

Break online connection Present questionnaires in tree When questionnaire is opened, interpret the XML definition and

create questionnaire dynamically. During filling in, control and validate. Open online connection for user to send the filled in

questionnaire as XML to the eQuest/Web database.


e-Quest „new“ prototype – technical ideas

based on eclipse rcp („rich client platform“) all software created as plugins central Maven repository for version and dependency control several open source tools and packages

questionnaire XML (contents, representation and layout) are interpreted dynamically, just as in e-Quest „classic“

control and validation modules generated by automatically translating the original Visual Basic scripts into Java code

the latter can then be customized by hand if necessary (or while the translator is still incomplete)


Wir bewegen I n f o r m a t i o n e n

Moving

i n f o r m a t i o n

www.statistik.at

s t a t i s t i c s a u s t r i a e-quest "new" 1 frederick rennert june 2008 ©...

Documents