getting ict data through surveys -...

InternationalTelecommunicationUnion

Committed to Connecting the World

Getting ICT data through surveys

ICT Data and Statistics DivisionTelecommunication Development BureauInternational Telecommunication Union


Outline

Collaborating and coordinating for household ICT statistics

Getting ICT data through surveys: good practices

The ITU Manual

Data disaggregations


Collaborating and coordinating for household ICT statistics

3


4 4

National coordination of ICT statistics 10th WTIM (2012) and 11th WTIS (2013)

High-level panel debate triggered numerous interventions from participants – issue is of concernto most involved in production of ICT statistics

Recommendations:

Countries should put in place coordination mechanisms

NSOs should play an active role

Countries should include ICT statistics in NSDS

ITU should develop guidelines and models for coordination mechanisms to assist countries

Topic needs further discussion in international and regional forums


5

Figure 1. Stakeholders in the ICT statistics system

Information providers

Households, individuals, businesses, schools, government institutions, etc.

National statistical offices

National regulatory agencies

Sector ministries

Private users

ICT industry

Other business

Media, journalists General public

Academia Consultants

Statistics producers Statistics users

Policy-makers

NRAs

International organizations

Stakeholders in the ICT statistics system


Why national coordination on ICT statistics?

Objective: to produce high-quality official statistics

ICT statistics are cross-cutting and therefore involve many stakeholders with different competencies and skills

Statistical data collection and dissemination is often fragmented - data quality suffers, duplication of effort

6


Why national coordination on ICT statistics?

Close data gaps

Eliminate duplication of work

Avoid conflicting data and statistics

Promotes comparability

Not to burden and confuse data providers and users

Promote effectiveness

7


Fundamental Principle of Official Statistics - Principle 8

Coordination among statistical agencies within countries is essential to achieve consistency and efficiency in the statistical system.

8


9 9

National coordination of ICT statistics

Different coordination models exist

National statistical coordination bodies

Formal inter-institutional committees and working groups (involving different Ministries)

Multi-year planning

National information society observatories


National statistical coordination bodies

National Statistical Commissions or Committees (established by statistical law)

Usually coordinated by NSOs, which may have satellite units in sector Ministries (eg. health, education, agriculture)

Can establish subject-matter working groups to discuss methodologies etc. (e.g. on ICT)

Example: Inter Agency Committee on ICT Statistics

10


Inter-institutional committees and working groups

Less institutionalized forms of collaboration among data-producing agencies

Inter-institutional working groups with clearly defined responsibilities for establishing technical standards (e.g. for data collection and analysis, dissemination of findings)

Bring together representatives from Ministries, NSOs, NRAs, etc.

11


Multiyear plans

Most national statistical systems are governed by a multiyear program for the production of official statistics

Multiyear plans should specify which institutions are responsible for each statistical operation, the timeframe and frequency

Covering different domains including ICT

12


National information society observatories

Objective: to centralize all ICT indicators and disseminate them through one national web portal

Requires close cooperation with all data producers in the country

Example: Spain – National Observatory for Telecommunications and the Information Society (ONTSI)

13


14

Spain

•National Observatory of the Telecommunications and the Information Society (ONTSI)

•Ministry of Industry, Tourism and Trade

•Covers many areas including ICT from private and public sources

•Surveys on ICT use in Households and Businesses carried out by the National Statistical Institute (INE)


15

Spain•Economic data from telecommunication operators collected by said ministry

•Price information collected by National Telecommunication Commission (regulator)

•Telephone network and broadband coverage data from the State Secretary for Telecommunications and the Information society

•Ad-hoc studies by ONTSI itself


16


17

BrazilThe Brazilian Network Information Centre (NIC.br), is a private non-profit organization, created to implement decisions from the Brazilian Internet Steering Committee (CGI.br). Comprised of members from the government, the corporate sector, non-profit organizations and the academic community, the CGI.br represents a unique Internet governance model for the effective participation of society in decisions involving network implementation, management and use.

The Brazilian Center of Studies on Information and Communication Technologies (CETIC.br) is a department of NIC.br, created in 2005 to tackle the challenge of periodically producing, organizing, analyzing and publishing data on the access and use of the Internet in Brazil.


18

BrazilThe Brazilian Internet Steering Committee (CGI.br) comprises members from the government, the corporate sector, non-profit organizations and the academic community, it represents a unique Internet governance model for the effective participation of society in decisions involving network implementation, management and use.

The Brazilian Network Information Centre (NIC.br), is a private non-profit organization, created to implement decisions from the CGI.br.

The Brazilian Center of Studies on Information and Communication Technologies (CETIC.br) is a department of NIC.br, created in 2005 to periodically produce, organize, analyze and publish data on the access and use of the Internet in Brazil.


19

BrazilCETIC.br conducts regular national surveys on the following topics: ICT Households; ICT Enterprises, ICT Kids Online Brazil, ICT in Education, ICT in Health, ICT e-Government, ICT Providers and ICT Non-profit Organizations.

In the process of collecting, organizing and disseminating information about ICTs, CETIC.br follows the standards and recommendations from:

• Partnership on Measuring ICT for Development, • ITU, • Eurostat, • OECD and • UNCTAD.


20

BrazilThe ICT Survey process at CETIC.br follows the principles of multilateralism, participation and transparency by fostering the participation and collaboration of an extensive network of academics and experts from government, the corporate sector and non-profit organizations who are renowned not only for their excellence in research methodology, but also for their expertise regarding the study of the use and impacts of ICTs.

Each ICT survey relies on a specific group of experts from different segments of society who specialize in the theme of the survey. This group usually meets twice during the survey process, once during the planning phase, in order to validate methodology, indicators and questionnaires; and later again during the stage of data analysis to provide input for interpreting results.


21

BrazilThe results of the surveys and their microdata base are made available to the expert groups under a confidentiality and ethical use agreement for data use.

Engaging different stakeholders in the surveys conducted by CETIC.br reflects the multistakeholder model for Internet Governance in Brazil at CGI.br and is considered key to granting legitimacy of the survey process.

For more information on CETIC.br surveys and publications, see http://www.cetic.br/english/.


22


23

OmanStakeholders•Information Technology Authority (ITA)

•National Centre for Statistics and Information (NCSI)

Collaboration: •Surveys on access to and Use of ICT by Businesses in 2011

•Surveys on access to and Use of ICT by Households in 2013

Frames: NCSIQuestionnaire: ITAData collection: NCSI / contractorData processing: NCSI / ITADissemination: ITAUsed international standards in questionnaire design and compilation of indicators including partnership’s core list


Getting ICT data through surveys: good practices

24


Some preliminaries

Primary (1o) data vs secondary (2o)data / admin records

Census vs sample survey


Some preliminaries

Target population / survey population

Complex Survey Data Analysis with SAS, Lewis


Some preliminaries

Sampling Simple Random Sampling (SRS) Each statistical unit has the same

chance of selection

Complex samplingPrimary sampling UnitsStrata (eg urban /rural)Cluster

Sample size and allocation


Some preliminaries

Estimation (sample statistic and population parameter)

PopulationSample

Sample statisticPopulation parameter


Some preliminaries

Sampling variation / Sampling errors

Dedicated surveys vs ridersWe’d like you to carry these 4

questions in your survey, tabulate them according to our

specifications, estimate the population totals, means and proportions and calculate the

standard errors for us.


Some preliminaries

Data collection methodsDirect observation

Postal

F2F

Telephone

CATI

CAPI

CAWI

Online


Surveys usually done by the NSO and assumed as such in this presentation.

Legal backing to collect data, usually the Statistics Law or Act or some other law that provides for compliance and confidentiality.

Survey methodology must be scientific and based on probabilistic sampling. The skill to do this is with the statisticians in the NSO.

Good practices


Good Practices: Planning of surveys

ObjectivesA clear statistical statement on the desired

information, giving a clear description of the population and geographical coverage. How the results are going to be used. Budget. Stakeholders.

Survey universeGeographical areas ,Target population,

exact population sampled to be identified, first stage units, second stage units, comprehensive and mutually exclusive frames for every stage of selection.


Good Practices: Planning of surveys

Information to be collected List of questions requiring statistical answers,

availability of some required data in existing sources, include supplementary items that are correlated with main items, tabulation plan, tabulation plan to be circulated for comments and improvement

Survey budget and survey timelines Cost estimates as detailed as possible, every

survey step exacts a cost, Survey budget will depend largely on survey design, precision required, geographical coverage, judicious cost control, accountability enhances credibility.


SURVEY COSTING


Timeframe for survey steps

1 Meeting with stakeholders (users and producers)

1 month

2 Preparatory activities 3 months

3 Initial questionnaire design 2 months

4 Send questionnaires to user committee members

1 month

5 Include in questionnaire agreed suggestions

1 month

6 Draft interviewer’s and supervisor’s manuals

2 months

7 Print questionnaires and manuals (coding questionnaire into CAPI)

1 month



8 Make plans for pretest 1 month

9 Train interviewers and supervisors 2 months

10 Conduct the pretest 1 month

11 Revise questionnaire (if need be) 1 month

12 Revise manuals (if need be) 1 month

13 Sample design 1 month

14 Design and test data entry programme (s)

1 month

15 Design and test data cleaning programme (s)

1 month

16 Data collection 3 months



17 Data entry 4 months

18 Design and test tabulation plan 2 months

19 Data cleaning (e.g. range and consistency checks)

2 months

20 Assignment of chapters to authors 2 months

21 Carry out the processing according to plan

1 month

22 Calculate sampling errors among other estimates

1 month



23 Distribution of tabulation to authors 1 month

24 Analysis and report writing 2 months

25 Compilation and tabulation of final report

2 months


Good Practices: Execution of surveys

Data collection methods Direct observation and measurement, mail, personal

interview (incl telephone, CATI, CAPI ) online including web and CAWI

Response rate

Questionnaire design Size and format, suited to data collection mode,

questions grouped in relevant sections, proper sequencing, easy reading, clear instructions, definitions, operational equivalents pre-testing (pilot)

For ICT access and Use by Households and Individuals, use globally accepted standard such as the model questions contained in Manual for Measuring ICT Access and Use by Households and Individuals.

Translation into local language must be done by an expert and tested on the ground


Good Practices: Execution of surveys

Tabulation and analysis plan Tabulation plans, dummy tables, titles, stubs and captions,

substantive variables, background variables, population groups, categories of classification

Implementation of fieldwork Need for a well-organised and effective field organisation Equipment and materials Management of survey operations, clear well defined line of

command Publicity Selection of interviewers Training of interviewers Field supervisors Follow-up of non-respondents Reducing non-response


Documentation

“Documentation and evaluation of sample designs in particular and survey methodology in general are too often neglected in the rush to release survey findings. This is especially true in countries with little prior experience in conducting household surveys…”

-UNSD


Documentation

A record of how it went Keep careful records of the survey and

sampling procedures as they are being carried out operationally in the survey process

Sample plan, adaptations at various stages of field work

To make sure the implementation is faithful to the design / record all departures

For adjustments to be made in analysis Indispensable for planning future surveys


Technical reports

Fairly brief , user-friendly description of survey methodology, sample plan and implementation

Limitations Comprise the technical section of the various substantive

reports on findings

Stand alone more detailed description of survey methodology Intended for professional researchers, social scientists and

statisticians rather than policy maker / public Publish in statistical journal / Present at WTIS

Special office to handle documentation

Documentation


References

Designing Household Survey Samples: Practical Guidelines, UNSD, 2005

Manual for Measuring ICT Access and Use by Houeholds and Individuals, ITU, 2014

Complex Survey Data Analysis with SAS, Lewis, 2017


2 mainstays


The ITU Manual

5

3


ITU Manual (2014)

Chapter 1. Introduction

Chapter 2. Coordination among national stakeholders in ICT measurement

Chapter 3. Planning and preparation for ICT household surveys

Chapter 4. Statistical standards and measurement topics for ICT household statistics

Chapter 5. Data sources and collection techniques for ICT household statistics

Chapter 6. Question and questionnaire design for ICT household surveys

Chapter 7. Designing ICT household surveys

Chapter 8. Data processing for ICT household statistics

Chapter 9. Data quality and evaluation for ICT household statistics

Chapter 10. Dissemination of ICT household data and metadata


Preparation and revision process

First release in 2009

2012-13: two rounds of complete revisions

Comments from Expert Group on HouseholdIndicators (EGH) forum

Version 2 launched at WTIS 2013 (December2013, Mexico)

Revision of indicators in 2014-2015: added HH16

HH17, HH18, HH19 on the website, but not yet in the Manual


ITU statistical standards: ICT household statistics

Statistical standards associated with the core ICT indicators for household accessto, and individual use of, ICT: concepts definitions of termsmodel questions classificatory variables (breakdowns) scope units (households and individuals)

Formula of calculation Use (policy relevance)


Core household indicators, main concepts

The indicators consist of those:

Referring to household access to ICT equipment and services

Referring to individuals’ use/ownership of ICT equipment and services


Concept of access

ICT device/service should be available for use of any member of the household at any time

Device can be owned or not by the household

Applies to all indicators referring to household ICT access

Device should be in a working condition


Age scope

Countries should report ICT usage information for the three main core indicators on individuals’ use of ICTs (computer, mobile phone and Internet) for the entire population of the country, i.e. there is no minimum age scope any more for these indicators


Reference period

Information on ICT usage should be collected and reported with a reference period of the last 3 months


Data disaggregations

6

1


Disaggregating the data by socio-demographics: why and how

Important to policy-makers

Disaggregation shows socio-economic problems that create barriers to use of ICT by individuals. These problems are diverse and broadly cover lack of opportunity and lack of ability. They include illiteracy and other linguistic limitations, socio-cultural barriers, lack of ICT and other skills, lack of confidence or awareness and low income.

Gives more information i.e. who is using the ICTs i.e. male/ female, age, location (urban/ rural) etc


International Standard Classificationsused in Core Questionnaire

COICOP- Classification of Individual Consumption According to Purpose by UNSD

ISCED - International Standard Classification of Education by UNESCO

ICSE-93 – International Classification of Status in Employment by ILO

ISCO – International Standard Classification of Occupations by ILO


Individual characteristics

Sex:

Sex disaggregation of data is a fundamental requirement for gender statistics and in particular for the analysis of the gender gap in the use of ICT. A MUST HAVE FOR ALL CORE INDICATORS

Age:

Age is a strong determinant of ICT use so a common age cut-off and categories are important

Recommended ranges: under 5; 5–9; 10–14; 15–24; 25–34; 35–44; 45–54; 55–64; 65–74 and 75 and over

Committed to Connecting the WorldEducation levels:

For international comparisons, countries required to classify education as International Standards Classification of Education follows:

primary education or lower (ISCED levels 0, 1),

lower secondary education (ISCED level 2),

upper secondary education or post-secondary non-tertiary education (ISCED

levels 3,4),

tertiary education (ISCED levels 5, 6), and

post-tertiary education (ISCED levels 7, 8).

Labour Force:

Based on the International Labour Organization (ILO) International Classification of Status in Employment (ICSE-93), with additional categories for those who are unemployed or outside the labour force.

Employee;

Self-employed (includes the four categories: employers, own-account workers, members of producers' cooperatives, and contributing family workers);

Workers not classifiable by status (for whom insufficient relevant information is available, and/or who cannot be included in the preceding categories);

Unemployed; and

Outside the labour force. i.e student, retired.

Further classification may be given as per occupation.


Disability status:

Because of a Health problem:

1) Do you have difficulty seeing even if wearing glasses?

2) Do you have difficulty hearing even if using a hearing aid?

3) Do you have difficulty walking or climbing stairs?

4) Do you have difficulty remembering or concentrating?

5) Do you have difficulty with (self-care such as) washing all over or dressing?

6) Using your usual language, do you have difficulty communicating (for example understanding or being understood by others)?

Response categories:

No difficulty; Some difficulty; A lot of difficulty; Cannot do at all

Other classifications at individual level are:

level of literacy, ethnicity, languages spoken, language skills.

The revised ICT household indicators include HH15, Individuals with ICT skills, by type of skills. It will therefore be possible to cross-classify the individual use indicators by ICT skill level.

Committed to Connecting the WorldHousehold Characteristics

Household composition (households with children under 15 and households without children under 15). Household composition is relevant to measuring the digital divide in households with children

Household size (number of household members, including those outside any age scope imposed).

Geographical disaggregation such as urban/ rural. Countries use their own definition for the urban/ rural and include it in the metadata. Countries can disaggregate this to towns, districts, counties to match their local needs.

Household with electricity can be used especially for the household ICT access indicators

Household income

Committed to Connecting the WorldCross-classification of data

Can produce information that is very useful for analytical purposes as is more detailed

example: Internet use by young women (data are cross-classified by age and gender).

ITU proposes the following cross-classification:

household composition by rural/urban,

rural/urban by sex,

age by sex,

educational attainment by sex,

status in the labour force by sex, and

occupation by sex.

Committed to Connecting the WorldICT Access by urban/rural and household composition


Thank you

For more information

https://www.itu.int/en/ITU-D/Statistics/

and

[email protected]

https://www.itu.int/en/ITU-D/Statistics/

mailto:[email protected]

getting ict data through surveys -...

Documents