the use of ict in data collection, analysis, management and … · 2016-02-25 · the use of ict in...

The Use of ICT in Data Collection, Analysis,

Management and Dissemination

By

Biyi Fafunmi Head, Information & Communication Technology Department

National Bureau of Statistics, Abuja

The use of ICT in Data Collection, Analysis, Management and Dissemination, Feb. 24-25, 2016

Biyi Fafunmi Head, Info & Comm. Tech Dept, NBS, Nigeria

Introduction What is ICT?

Data Collection: Processes Standards and

Practice ICT base Data Collection

Data Analysis, Management & Dissemination

Outline

Still …

Beyond

Reach

Back

gro

un

d /

Con

text



Introduction….

• The called for “Data Revolution” to underpin

the five transformational shifts recommended

– Echoed by others

• What does the “Data Revolution” mean for Nigeria?

• How do we ensure that the Nigerian statistics community are involved?–

Actually take the lead



Introduction...

Over the years, available data is in adequate to meet the challenge, and is not sufficient to effectively inform evidence-based policies. Then the need for a new innovation

The world’s capacity to collect such data is reported to double every 40 months since the 1980s, with about 2.5 billion gigabytes (GB) of data being created per day in 2012.

The advent of the Web, mobile devices and other technologies such as sensor networks has caused a fundamental change to the nature of data that is been generated every minute.



Why Data Revolution?

• To harnessing big data and new technologies.

• To close the data gaps

•

• To strengthen national statistical capacities,

• To assess the new opportunities linked to innovation, technical progress , that makes public and private data providers to support and complement the conventional national countries statistical systems ,



Introduction :

Check &

coding by

manual

Data

input

Host computer

(key punchers) (Check & coding staff)

Questionnaire



Introduction: Problems and Needs application

of new tech

• Taking a long period for data collection and

capture

Needs for Earlier publishing

• Using a lot of personnel for data collection

and capture

Needs for Personnel reduction

Efficient use of new technology is crucial



Information and Communication Technology (ICT), is a composite term, which embodies three important concepts. To understand ICT, one must understand all three concepts.

"Information" means many things to many people, depending

on the context. ◦ A processed data. Or ◦ Any potentially useful fact, quantity or value that can be use to aids decision

making.

Communication refers to the transfer or exchange of information from person to person or from one place to another. ◦ When action produces a reaction, whether positive or negative,

communication has taken place. ◦ a process of transfer of information, ideas, thoughts and messages. It

involves a sender, a receiver, a code and, a language that is understood by both the sender and the receiver.

What is ICT?



Technology refers to the use of scientific knowledge to invent tools that assist human beings in their efforts to overcome environmental hazards and impediments to comfort. ◦ In this regard, technology refers to the things like the computer, telephone,

cell phone, GSM handsets, television, radio, etc.

Put together, therefore, ICT has been defined as:

The acquisition, analysis, manipulation, storage and distribution

of information; and the design and provision of equipment and software for these purposes. (deWatteville and Gilbert 2000)

ICT and Information Technology (IT) are similar concepts that can be used interchangeably. IT implies communication and therefore it becomes obvious that the two terms are synonymous.

What is ICT?



Why new Technology (ICT)?

Faster

Smaller

Cheaper

Handier

Better

Accurate



..the impact on data collection

ICT Trend



Data collection is a systematic approach to gathering information from a variety of sources to get a complete and accurate picture of an area of interest.

There are various methods of data collection,

such as personal interviewing, telephone, mail and the internet. Depending on the survey design, these methods can be used separately or combined.

There are also three sources of data which are:

What is data collection?



Data Sources

Censuses

Surveys

Administrative Statistics



Process of Data Collection for Production

Dissemination

Hard Copies



Data Production - Need Assessment Matrix

Step Key Activity Current Practices &

Tools

Best Practices &

Tools

Planning Interactions between Users,

Suppliers, Producers to define

Concepts, Indicators, Scope,

Timelines, etc

Focused face-to-face

planning sessions

User group meetings, etc

Use of interactive

collaboration tools – e.g.

email, intranet, chat engines,

video conferencing, etc

Development of

Instrument

Develop survey instruments like

Questionnaire, Manual, Templates,

Labor, etc

Static questionnaire

Static Templates

Digital forms

Online questionnaire

Reusable electronic templates

Training Training of Trainers

Training of Enumerators

Instructor-led classroom

sessions

Computer Based Training

(CBT) packages for standard /

generic trainings

Pilot (Test) Run Simulate the data gathering process

under in controlled environment

Manual simulation of

operating procedure

Automated workflow systems

where applicable

Finalise

Operational

Modalities

Review and agree key operating

elements

Incorporate lessons / experiences

of the pilot run

Manual review Automated workflow systems

where applicable

Real Data

Collation

Identify Target population

Canvass

Conduct data collation, compilation

and batching

Manual questionnaire GPS-GIS systems

Electronic handheld devices

Digital forms

Relational Database

management systems



Data Production - Need Assessment Matrix

Step Key Activity Current Practices

& Tools

Best Practices & Tools

Quality

Checks

Spot check in the field

Checking, Editing, scrutiny,

etc

Manual spot checks and

editing

Workflow systems

Reusable electronic templates, etc

Data

Processing /

Analysis

Data entry

Quality control, Editing, etc

Analyze data using Tables,

charts, graphs

Largely manual

Use of simple productivity

tools like Excel

Computer systems

Specialized data analysis tools

Business intelligence systems

Centralized data processing

Relational Database Management Systems

(RDMS)

Report Writing Develop draft copy of the

report

Largely manual – using

Microsoft Word

Electronic

Report

Validation

Stakeholders to review and

validate draft report

Develop Zero Draft

Manual Workflow systems

Reusable templates

Provide for

Dissemination

Present report to users in

all possible formats –

hardcopies, soft copies,

web

Produced in hardcopies

and softcopies

Portal Technology with XML capabilities for

multi-channel delivery,

Email and other collaboration tools

Integration with all data producing agencies

Data

Archiving

Data banking and storage Manual record keeping

in Hardcopies

Electronic data storage and archiving

Disaster recovery systems



Survey Planning

Use of digital (imagery and sketch) EA maps for: ◦ Scientific sample selection

◦ Monitoring of Enumerator

◦ Data validation and spot checking

24 February 2016 19

Planning : Conversion from Paper to Digital EA

0437

0438

0440

0439

0441

192

169

G.M

109107

U/C

U/C

U/C

U/C

Shop

004P/R/98

004P/R/180

004P/R/120

Kem Clinic

Coop Building

N.T.C Leaf Barn

N.T.C Leaf Barn

Bapt Gramm Schl

N.T.C Leaf barn

St. Thomas Church

Texaco Filling Station

24 February 2016 20

by State and Local Government in Nigeria

NATIONAL BUREAU OF STATISTICS,

Plot 762, Independence Avenue, CBD,ABUJA, http://www.nigerianstat.gov.ng

Niger

Borno

Yobe

Taraba

Bauchi

Oyo

Kogi

Kebbi

Kaduna

Kwara

Edo

Benue

Sokoto

ZamfaraKano

AdamawaPlateau

Jigawa

Delta

Katsina

Ogun Ondo

Nasarawa

Gombe

FCT

Rivers

Osun

Imo

Ekiti

Cross River

Bayelsa

Enugu

Abia

Ebonyi

Akwa Ibom

LagosAnambra

Abia Cross River Imo Lagos

Adamawa Delta Jigawa Nasarawa Rivers

Akwa Ibom Ebonyi Kaduna Niger Sokoto

Anambra Edo Kano Ogun Taraba

Bauchi Ekiti Katsina Ondo Yobe

Bayelsa Enugu Kebbi Osun Zamfara

Benue FCT Kogi Oyo

Borno Gombe Kwara Plateau



Forms

24 February 2016 22

Product

24 February 2016 23

Mobile Data Collection

Handheld - Price Statistics GPS - Agric Census

Micro data Census & Survey GPS & Handheld

Computer

Macro data MDAs Administrative

Statistics

Automated Sector and State Statistical

databases

CAPI - Household Statistics



Mobile data collection

Tablet

Other Smart

Phones

Overview CAPI Devices

A system of using a Computer Assisted Personal Interviewer Device to conduct survey.

http://en.wikipedia.org/wiki/File:Tablet.jpg



How it works

• A developer creates an application to be used for data entry in the PC.

• The application is then deployed to a hand-held device.

• The device displays the questions on screen,

• The interviewer reads questions to the respondent, and then enters the respondent's answers.

• After completion the interviewer send the responses



• Data validation and accuracy;-

– Data validation is built into the software, which does not allow invalid values to be entered.

– Back-checks can be performed to ensure data accuracy.

• Data collection and Data Entry happen at the same time, and no separate Data Entry staff needs to be hired.

• Eliminates missing information

• Data can be scrutinized real time as the data is getting uploaded to the server.

• Shortens interviews (safe lots of good time).

• High data quality is guaranteed if managed properly

Why Digital Survey (CAPI)



Costs balanced out by • Printing of only 10% of questionnaires • No field editors • No data entry • No computers required for data entry

Cost savings • If Tablets are re-used • Savings on secondary processing, data

editing

Why Digital Survey (CAPI) Contd.



Laptops Mini Dedicated Server Tablet (Windows 7 or 8)

Accessories ◦ SD card ◦ Spare battery ◦ Case/Covers ◦ Screen protector ◦ Spare stylus ◦ Power Banks(1 per team) ◦ Mobile Sim Cards

Hardware Requirement.



CSPro (Census and Survey processing System):- For data capturing CSEntry

Licensed SPSS(Statistical Products and Services Solutions):- For Table generation and analysis

Licensed Dropbox:- For data transmission to Head office from the field

Licensed Antivirus:-For virus protection

IHSN (International Household Survey Network’s) Microdata Management Toolkit:- For data archiving.

Software requirement.



Communication Methodology

Interviewer’sTablets

Dropbox

Interviewers

Cloud Assigned Interviews

Central Office PC

Supervisor’s Tablet

Wednesday, February 24, 2016

www.nigerianstat.gov.ng

Enumerators repeat

interviews if errors

are detected

Interviewers

synchronize their

devices and upload

completed

questionnaires

Supervisors monitor

the submissions

Questionnaires with

no errors are

uploaded to the

central server

Supervisors assign

households to

individual

interviewers

WiFi

Programmerrs design

questionnaires using

visual tools and

upload them to the

central server

Interviewers visit

households and

collect data

HQ distributes the

sample lists across

teams of enumerators

Internet

WiFi

Internet

PROCESS FLOW

HQ

Supervisor

Field



Data Management Development of a reliable database to

document and store all data for future access, analysis and mining .

Data Transmission an automated system that lets you manage, synchronize and share unlimited files and folders with Cloud storage with your trusted devices (i. e Laptops and tablets etc).

Data Management and Transmission

24 February 2016 34

Data Collection Using GPS/Hand Held Device : Agric Survey

Data Collection using Hand Held Device : Prices Survey



Data Integration: Develop a System of Data Integrating

36



MDAs Data: Automated Sector and State Statistical Databases

a compendium of statistical terms, concepts and definitions for harmonizing statistical production in all MDAs.

a Template to capture administrative data in all states of the federation for the production of National, Zonal and State Statistical Yearbooks (NSYB, ZSYB and SSYB).



MDAs On-line Data Capture Template

Sector Statistics Database Management System



24 February 2016 39

National Statistical Year Book Template

NSYB

NC

Zone

NW

Zone

SW

Zone SE

Zone

SS

Zone

NE

Zone

States



NSS connectivity

Nigerian Data Nervous System (NDNS)

NBS under NSDS has created

By bringing all data producer and user Agencies together through a Virtual Private Network (VPN) cloud

a



Internet

Cloud Covering all Data

Producers, Data

Suppliers & Data Users

in Nigeria

The VPN Cloud Of the Nigerian Data Nervous Systems (NDNS)



NBS & 36 SSAs Plus FCT VPN Design

NBS HQ Abuja

AbiaA

dam

awaAkwa-Ib

omAnambra

Bauchi

Bayelsa

Benue

Borno

Cross - River

Delta

Ebonyi

Edo

Ekiti

Enugu

Gom

be Imo

Jigaw

a

Kaduna

Ka

no

Ka

tsin

a

Ke

bb

i

Ko

gi

Kw

ara

Lagos

Nasa

raw

aN

iger

Ogu

n

Ond

o

Osu

n

Oyo

Plateau

Rivers

Sokoto

Taraba

YobeZamfaraFCT Abuja

Active Sites In Progress To be Activated in 2014



NBS Federal MDAs VPN Design



.. an e-Statistics grid : MDA connection to NBS

SSA &

other

Connectio

ns

CPTC

CPTC

CPTC

CPTC

CPTC

CPTC

Codex

Modem

Codex

Modem

Cisco

2620

Router

Cisco

3640

Router

64kbps Fibre

Digital Subscriber

Line DSL

LAN

25

1

NBS Data

Center

MDAs

Remote Connectivity between NBS and SSAs

V-SAT connecting SSA in Niger State to NBS

V-SAT connecting NBS Hqrt to all SSAs



The Link between Data and Understanding

Tell how to go from data to assessment

Transform data into Information

Developing Tools for Data Analysis depends on Size of the data, mode of storage, the users need, level of details, and mode of dissemination

Data Analysis



DATA PROCESSING

CODING & EDITING

DATA ENTRY COMPUTER

EDITING & DATA VALIDATION

FREQ. TABLES & SAMPLE

ESTIMATES

STANDARD ERROR OF ESTIMATES

SYSTEM DESIGN & DEVELOPMENT

STATISTICAL TABLES

GENERATION

Stages before Data Analysis

Data Quality Checks



Levels of analysis

48

Descriptive

simplest; present facts; summarization

Quantitative

Measures of central tendency ( Mean, Median, Mode)

Measures of dispersion (Range, Standard deviation)

Measures of association

Projection and forecasting

Spatial



Some Visual Presentation of Analyzed Data

49

Types:

Tables

Charts

Plots

Maps

etc



Some Available Data Analysis Software

There are many Statistical Analysis Packages available today,

From simple to complex and robust

analysis

SPSS, SAS, STATA, Statgpahics, E-view, Shazam etc.

ArcGIS, Geomedia, etc for raster / vector data and Spatial Analysis



Data storage is the holding of data in an electromagnetic form for access by a computer processor.

Data Storage - consideration for ICT data collection



There are three main levels

of storage: ◦ Primary storage -RAM and

other memory devices that are built into computers.

◦ Secondary storage - external storage devices such as hard disks, tapes, CD's.

◦ Tertiary Storage – removable Cloud

Data Storage



Flat file

Simple database

Relational database

Object oriented

Hierarchical

Network

Data bank/warehouse

More than 100 top DBM software

Database for data Management

Dabases

SQL

Oracle

CSV/xls

XML

etc



Example of Existing databases

DHIS 2: Health Information System DHIS 2 is National District Health Information systems for data collection, management and analysis purposes, for health program monitoring and evaluation .. Education Management Information System is an information system with the main purpose of providing information which is used for improving the management of education at all levels. The value of information for management purposes is increasingly recognized in the business environment and by government bodies. What is a Human Resources Information System (HRIS)? A HRIS, or Human Resource Information System, is a software solution for small to mid-sized businesses to help automate and manage their HR, payroll, management and accounting activities. SSYB – State Statistical Year Book a Template to capture administrative data in all states of the federation for the production of National, Zonal and State Statistical Yearbooks (NSYB, ZSYB and SSYB DevInfo – A database system developed by UNICEF for capturing information on children and women at various level of government.

Biyi Fafunmi Head, Info & Comm. Tech Dept, , NBS, Nigeria


Humanware



Users Consideration: Develop Strategy for Dissemination to meet users’ need



To meet different user’s need, we disseminate in various forms

Hard Copy – Publications (Annual,

Quarterly, Monthly reports, Flyers, Fact sheets, Brochures etc)

Offline Access – Data shop

Soft copy ◦ Drives ◦ Tapes ◦ Databases

Online (web 2.0 – XML, SDMX) ◦ Email: [email protected] ◦ web : http://www.yourdomain.gov.ng ◦ Social Network & RSS feed

Major Platform for Dissemination

Media

DataBases online

Printed Publications

mailto:[email protected]

http://www.nigerianstat.gov.ng/

http://www.google.jo/url?sa=i&rct=j&q=tv&source=images&cd=&cad=rja&docid=7NH_7sfXxI4RZM&tbnid=9hu6sYhr3_4aPM:&ved=0CAUQjRw&url=http://gwinnettmedical.blogspot.com/2012/06/get-real-reality-tv-habits-that-can.html&ei=fsIuUsidBsHcswa13IHYBQ&psig=AFQjCNExqPb1m2M2APZ_mUoKTIKQWEcPqA&ust=1378882456448862



Use of Web Portal for Data Dissemination

NBS www.nigerianstat.gov.ng



Example of Data Portal

NigeriaInfo

NigeriaInfo



Microdata Access Dissemination



Conclusion: A Typical 21st Century DCMS

Database1

Database2

Database3

Database MIS

Data Supplier

Data Supplier Data Supplier

Data Supplier

Reports

Data Collectors



Thank you for your attention

the use of ict in data collection, analysis, management and … · 2016-02-25 · the use of ict in...

Documents