1 census data processing in bps – statistics indonesia

1

Census Data Processing

In BPS – Statistics Indonesia

Vision: BPS is the provider of quality statistics.

A non-departmental government institution.

Directly responsible to the President of the Republic of Indonesia.

Headed by a director general.

- Provide data to the government, and the public.

- Assist statistics divisions of government ministries and other institutions, in developing the statistical system they need for setting up work programs and periodic reporting.

Develop and promote standards to be incorporated in the implementation of statistical techniques and methods.

Provide necessary services in the field of education and training in statistics.

Establish cooperation with international institutions and other countries for the benefit of Indonesia’s statistical development.

About BPS – Statistics Indonesia

Director General

Bureauof Program

Management

Bureauof Personnel

& Legal Affairsl

Bureauof Finance

Bureauof General

Affairs

Principal Secretary

Deputy DGfor SocialStatistics

Deputy DGfor Methodo

logy and StatisticalInformation

Deputy DGfor Natio-

nal Account andStatistical Analysis

Deputy DG for Distribution and Services

Statistics

Directorate forDev of Census &

SurveyMethodology

Directorateof Statistical

Dissemination


Information System

Directorateof Population

Statistics

Directorateof Social Welfare

Statistics

Directorateof Social Resilience

Statistics

Directorateof Distribution

Statistics

Directorate of PriceStatistics

Directorate of Financial, IT, andTourism Statistics

Directorateof Production

Account

Directorateof Consumption

Account


Analysis

Education andTraining Center

Deputy DGfor

Production Statistics

Institute ofStatistics

BPS RegionalStatistics Offices

MainInspectorate

InspectorateArea I

InspectorateArea II

InspectorateArea III

Directorate of Animal

Husbandry, Fishery,

And Forestry Statistics

Directorate of Indutrial Statistics

Directorate ofFood Crops,

Horticultural,and Estate

Crops Statistics

Organizational Structure

There are

33 BPS provincial offices

344 BPS regency offices and

96 BPS city offices.

they are the vertical representative offices of BPS.

At sub-district level, BPS employs a statistical staff (sub-district level Statistical Coordinator).

Technically and administratively, BPS regional offices are directly under the control of BPS-Statistics Indonesia.

The number of BPS staffs is 13,465 (as July 2008)

1,466 persons in BPS Head Quarter (11%)

11,999 persons in BPS-Regional Offices (89 %).

Provincial and Regional Offices

ADMINISTRATIONDIVISION

INTERNAL AFFAIRSSUB DIVISION

PERSONNELAND LEGAL AFFAIRS

SUB DIVISION

FINANCESUB DIVISION

PROGRAM MANAGEMENTSUB DIVISION

ASSETS

MANAGEMENT

SUB DIVISION

PRODUCTIONSTATISTICS

DIVISION

DISTRIBUTIONSTATISTICS

DIVISION

REGIONAL ACCOUNTAND STATISTICAL

ANALYSIS DIVISION

SOCIALSTATISTICS

DIVISION

DATA PROCESSINGAND

DISSEMINATIONDIVISION

AGRICULTURALSTATISTICS

SUB DIVISION

CONSUMERAND WHOLESALEPRICE STATISTICS

SUB DIVICION

PRODUCTION ACCOUNTS

SUB DIVISION

DEMOGRAPHICAND HOUSEHOLD

STATISTICSSUB DIVISION

NETWORKINGAND STATISTICAL

REFERENCESUB DIVISION

INDUSTRIALSTATISTICS

SUB DIVISION

FINANCIALAND PRODUCER

PRICE STATISTICSSUB DIVICION

CONSUMPTIONACCOUNTS

SUB DIVISION

MAN POWERSTATISTICS

SUB DIVISION

DATA PROCESSING

INTEGRATION

SUB DIVISION

MINING, ENERGY& CONSTRUCTION

STATISTICSSUB DIVISION

COMMERCE& SERVICESSTATISTICS

SUB DIVICION

STATISTICALANALYSIS

AND DEVELOPMENTSUB DIVISION

SOCIAL WELFARESTATISTICS

SUB DIVISION

DISSEMINATION& STATISTICAL

SERVICESUB DIVISION

CHIEF OF BPSPROVINCIAL OFFICE

Provincial Level Organization

Population census: in the years ending with 0. previous population census: in 1961, 1971,

1980, 1990, 2000. The next Population Census would be in 2010

Agricultural census: in the years ending with 3. previous agricultural census: in 1983, 1993,

2003. Economic census:

in the years ending with 6. previous economic census: in 1986, 1996,

2006.

Census in Indonesia

• Using Mainframe:

• (NEC ACOS 500 in 1980)

• (NEC ACOS 1520 in 1990)

• Batch Processing (not interactive)

Data Processing Support for Population Census 1980, 1990

1. Decentralized & Distributed Data Processing:

• Encourage End-users Computing: subject matter units are entitled to process their own survey under coordination of data processing division.

• Develop application in-house by HQ & decentralize operation to Regional offices.

• Implement Distributed / Client-Server Data Processing Model.

2. IT Department (Directorate of SIS) focuses on IT infrastructures & complex DP Systems such as Censuses.

8

Data Processing Policy(after elimination of Mainframe)

• PC based Client-Server Data Processing

• Using Scanner: Kodak DS3500• Speed: 3500 forms / hour• Use of OMR, ICR: Nestor

Reader• ICR Ascent-Capture: started in

2003 (Census of Agriculture).

Data Processing Support for Population Census 2000

PC 2000 Data Capture Stages

Scan documents (produce image file)

Recognize image (produce ascii characters)

Verification & Correction (operators decide characters ambiguous to recognize engine)

Completion completeness check

Release save into database

Experience from PC 2000• Unexpected low quality of enumerator’s

handwriting to recognize by ICR• Unable to recognize characters correctly

– Rejected too many values (too high confidence level)

– Accepted incorrect values (too low confidence level) realized from tabulation output

– Should check-digit help to increase accuracy ?• Ineffective learning engine• In a LAN environment using MS-SQl Server, 4

concurrent scanning processes slow down the whole DP activities consisting of 3 recognize stations, 10 correction stations,and 25 computer editing stations.

Experience from PC 2000• Extra work to erase & rewrite Census Form• Powder from rubber eraser stick on Census

forms & affected Scanner performance• Costly maintenance effort of Scanners

which spread over all provinces• Relatively high cost (investment) for Data

Capture H/W& S/W• However, Data Capture using scanner was

able to shorten Data Processing Completion Time.

• It was not easy to find out well motivated enumerators & Data Capture operators.

PC2010 Data Processing Plan

Main Documents:• Short Form : L for Listing & basis for 10%

sample, using data entry (DE), : C using scanner,

complete enumeration (100%) of households & members

• Long Form : S using data entry / scanner, 10% sample.

Estimation:• No of Districts / Regency = 456

• No of Sub-districts = 5,901

• No of villages = 72,084

• No of Census Blocks = 658,308 (in year 2000)

• No of smallest Administrative Areas (SLS) = 900,000

• No of households = 64 million

• No of population = 236 million

Document: Batch Control

Document C

PC2010 Data Processing Plan (cont)

Issues on existing Scanners:

Kodak DS3500 procured for PC 2000: 79 units

About 60 % of them need spare-parts replacement

Spare-parts are discontinued. (Dicom Singapore eager to reserve all available spare-parts to BPS)

Running out of Ascent Capture Counter (dongle) limit

Operable uncertainty for PC 2010

1. Utilize scanners with character recognition technology (OMR, OCR, ICR), focused on OMR.

2. Expect ICR with better Learning Engine capability.

3. Procurement of Scanning H/W and S/W in 2009.

4. Formation of 8 Data Processing Centers.

5. Utilize SMS facility for transmitting and accessing data, as well as monitoring data collection activities.

6. Utilize web-based application for monitoring data processing activities.

7. In 2008, all of 33 province level offices and some (34) district / city level offices are connected using Virtual Private Network (VPN).

17

PC 2010 Data Processing Strategy

PC2010 Data Processing Plan (cont)

Formation of 8 Processing CentersMedan 11 NAD Surabaya 35 Jawa Timur

12 Sumatera Utara 51 Bali13 Sumatera Barat 52 N T B14 Riau 53 N T T21 Kepulauan Riau Banjarmasin 62 Kalimantan Tengah

Palembang 15 Jambi 63 Kalimantan Selatan16 Sumatera Selatan 64 Kalimantan Timur17 Bengkulu Makasar 71 Sulawesi Utara18 Lampung 72 Sulawesi Tengah19 Kep. Bangka Belitung 73 Sulawesi Selatan

DKI Jakarta 31 DKI Jakarta 74 Sulawesi Tenggara36 Banten 75 Gorontalo61 Kalimantan Barat 76 Sulawesi Barat

Bandung 32 Jawa Barat 81 MalukuSemarang 33 Jawa Tengah 82 Maluku Utara

34 DI Yogyakarta 91 Irian Jaya Barat92 Papua

8 Data Processing Centers

SMS / Web Based Progress Reporting & Monitoring

GSM modem

Database Server

Regional Offices

Internet / VPN

Headquarters

Regional Offices

Progress Reports Monitoring

Monitoring

Cellular Operator

Headquarters

Progress Reports

SMS ApplicationServer

Thank you for your attention

September 2008