copyright 2010, the world bank group. all rights reserved. processing, part 1 data capture, editing,...

Copyright 2010, The World Bank Group. All Rights Reserved.

PROCESSING, Part 1

Data capture, editing, imputation and tabulation

Quality assurance for census

1


COMPONENTS OF PROCESSING

Data CaptureEditingImputationTabulation

2


DATA CAPTURE

Key EntryScanningDirect Entry

3


KEY ENTRY

Key entry as a data capture techniques has advantages and disadvantages:

Pro• Relatively inexpensive• Skills readily available• Employment

4


KEY ENTRY (cont’d)

Con• Time consuming• Requires many workstations• Error prone

5


SCANNING

Scanning is a process similar to photocopying, but the current technology has advanced far beyond simple photocopying.

There are three levels of scanning:1. OMR: Optical Mark Recognition2. OCR: Optical Character Recognition3. ICR: Intelligent Character Recognition

6


SCANNING (cont’d)

Pro• Fast processing

• Reliable

7


Con• Expensive upfront costs for equipment ,

software, & training• Very precise requirements for paper,

printing and processing

8

SCANNING (cont’d)


DIRECT ENTRY

• Most Recent Innovations

• Enumerators use hand-held computers,

or where internet is common

• Self enumeration through the internet

9


DIRECT ENTRY

Pro• More efficient– Saves a step (immediate data capture)

• Improves data quality– Editing at respondent level– Timeliness

• Reduces some costs– Printing questionnaires

10


DIRECT ENTRY

Con• Requires better trained enumerators• Riskier– Hardware failure– PDA loss– Requires electricity

• Expensive

11


EDITING

The Editing Process• Identifies errors• Identifies non-response• Identifies logical inconsistencies

12


DEALING WITH ERRORS

• Replacement (Imputation)

• Weighting

13


IMPUTATION

Two classes of imputation

• Deterministic

• Stochastic

14


DETERMINISTIC IMPUTATION

• Will yield the same answer each time– Missing data may be calculable from other values

- e.g. Citizenship can be calculated from Place of Birth

– Missing data is imputed by a sequential donor technique or any other method that will yield identical results

15


DETERMINISTIC IMPUTATION

• Six main types– Deductive or Logical– Mean Value– Ratio/Regression– Sequential Hot Deck– Sequential Cold Deck– Nearest-Neighbor

16


STOCHASTIC IMPUTATION

• Can yield different results if the process is rerun– Use of a random donor or other randomized

approach– Use of randomized residuals to create realistic

data

• Most deterministic methods have a stochastic counterpart

17


IMPUTATIONWhich inconsistency to change?• The general rule is to change as few values as

possible• Which values to impute?– For a 10 year old married university graduate, do

we change the age or the education and marital status?

– changing only the age can make the record consistent, therefore change age

18


VALIDATION

After imputation data are consistent.May or may not be correctValidation is the final step before certification

and release

19


VALIDATION

May be subject to bias: • Design• Training• Enumerator• Respondent• Processing

20


CERTIFICATION

Final step before data release

NSO’s expression of confidence in the data

21

copyright 2010, the world bank group. all rights reserved. processing, part 1 data capture, editing,...

Documents

world bank group

scanning contd slide

data capture key entry

key entry key entry

scanning scanning

deterministic imputation

data quality editing

direct entry pro