rapid mobile phone-based (ramp) survey

www.ifrc.orgSaving lives, changing minds.

RAMPRAMP

Rapid Mobile Phone-based (RAMP) survey

Changing the way we plan and manage health and development surveys

Version September 10th 2012


RAMP

Presentation overview

1) Introducing RAMP: an overview

2) How does RAMP work?

3) Stakeholder benefits

4) RAMP survey sampling approach

5) How much does a RAMP survey cost?

6) Costs of the RAMP pilot surveys

7) RAMP malaria survey

8) Reporting and dissemination of results

9) RAMP survey toolkit


RAMP

Introducing RAMP: an overview


RAMP

Purpose of RAMP

To provide a survey methodology with which Red Cross Red Crescent National Societies, governments, NGOs and other partners can conduct health and development surveys: at reduced cost with limited technical assistance to achieve high standards of survey design and quality

To decrease dramatically the time and effort needed to complete data collection and have results available for decision-making


RAMP

Background

IFRC and partners (WHO, CDC) developed and refined the RAMP methodology and tools over several years

Based on: improvements to the EPI survey method (WHO EPI) improvements based on scientific article published by

Donna Brogan, Emory professor of biostatistics, Emory U, Atlanta, USA, specialist in survey methods and analysis, and Emory and CDC colleagues in 1994

public-domain mobile phone software Pilot tested in four surveys in Africa 2011-2012

(Kenya, Namibia and Nigeria)


RAMP

Improvement over old EPI methodSurvey activity Old EPI method Improved

RAMP method

Choosing start point (first household) in the cluster

Biased: “Spin the bottle method” random direction

Simple random sampling

Choosing subsequent households (2nd, 3rd, etc.)

Often biased in practice: next nearest door

Simple random sampling

Fixed number of households visited per cluster

Biased: quota sampling without weight adjustment

Yes

Attempt to re-interview missing households

No Yes

Adjusts for non-response, household and individual

No Yes

Adjusts for inaccurate population estimate of clusters

No Yes (as option)

Analysis based on weight adjustments

No Yes


RAMP

Key features of RAMP

Simplicity of sampling design to facilitate survey implementation and reduce field costs

Use of standard survey sampling methods Use of mobile phones and internet for data collection

that enables: web-based questionnaire design downloading of questionnaires onto standard mobile

phones data collection using low-cost mobile phones

Real-time, web-based dataset that can be easily accessed and exported for rapid analysis and reporting

Availability of many tools to guide survey planning and implementation


RAMP

Changing the way we work

Paper questionnaires filled out in the field

Data entered into a computer at a central location

Data analysis and reporting often takes months to complete

Local capacity is often under-utilized and there is a dependence on external experts

Mobile and internet-based technologies are used to reduce the time taken from data collection to reporting

RAMP changes the way that data are collected and enables rapid reporting of results

RAMP empowers local ownership of evaluation and research

The “old” The “new”


RAMP

When might a RAMP survey be suitable?

Surveys where rapid results are key Surveys where cost is a significant issue Sub-district surveys involving multiple villages that are

typically carried out by non-governmental organizations (NGOs)

National, provincial or district-level surveys carried out by Ministry of Health (MoH) or government departments (immunization, maternal and child health, malaria, HIV/AIDS, nutrition, water and sanitation, etc.) to collect intervention coverage data quickly and at low cost


RAMP When might a RAMP survey be suitable? (cont’d)

District-level surveys conducted by the district health management team

Baseline and endline surveys for donor-funded projects/programmes

Repeated surveys to track time trends for key indicators


RAMP When might a RAMP survey be suitable? Flexibility

Items that can be adjusted Comments

Adjust precision ±10%, 5%, 3%, etc.

Adjust indicator type (denominator of indicator) including mixtures of indicator types

- Person all ages- Children <5 years old- Pregnant women- Households- Schools

Adjust number of domains - 1 domain with 30 clusters- 2 domains with 30 clusters to

compare statistically - 10 domains with 30 clusters each to

compareAdjust overall sample size - 200 to 5000 households


RAMP When might a RAMP survey not be suitable: limitations?

Very long questionnaires with a large quantity of skip patterns Paper method or commercial mobile phone software

might work better


RAMP

How does RAMP work?

(Short version)


RAMP

With RAMP you can transform a standard mobile phone into a cutting-edge evaluation and research tool

Conduct surveys and capture data

froma standard mobile

phone

Manage surveys, people and data from your web-based server


RAMP

How does it work?


RAMP

Web-based server

Create a free account using Datadyne’s Episurveyor software

Access EpiSurveyor’s web-based server from a web browser anywhere in the world

Design your questionnaire with embedded logic and in multiple languages

Monitor, manage and communicate with your team

Export data and analyse results in real-time

www.episurveyor.org/user/index

http://www.episurveyor.org/user/index


RAMP

Why use mobile phones to collect data?

Real-time data entry on mobile phones Daily upload of data from mobile phone over 2G mobile

network to internet database Real-time data monitoring and data quality checks Real-time data cleaning Real-time data analysis Rapid production of survey results within hours or days

of last interview


RAMPMobile phone applicationTransforms complex logic into an effortless step-by-step process

• Install Java application onto a standard mobile phone

• Log in to EpiSurveyor using user name and password, and download form

• Capture data in the field, even without network coverage

• Enter data using the interface of the mobile phone

• Automated submission of data when 2G reception is available


RAMP

How does RAMP work?

(Longer version)


RAMP RAMP takes advantage of two technologies

Low-cost, standard mobile phones

Web-based, easily accessible software application (EpiSurveyor) that enables mobile phones to become a data collection platform


RAMP

How does it work?


RAMP Web-based server

• Get started and set up your team to work with the EpiSurveyor software

• Create a free account using EpiSurveyor

• Assign privileges• Access the web-

based server from anywhere in the world


RAMP Design your questionnaire

•

Collaborate with others to design the web-based questionnaire• Easy to master• Supports multiple languages • Embedded logic and skip patterns


RAMP


RAMP Store questionnaire on phone

• Survey questionnaire is downloaded onto the mobile phone

• Install Java application onto a standard mobile phone

• Log in to EpiSurveyor using a user name and password and download the forms


RAMP Collect data in the field

Fieldworker retrieves the questionnaires on the mobile phone Enters data directly into the mobile phone during interviews


RAMP Data sent to web-based server

…

Fieldworker submits data when 2G reception is available Data are available immediately for processing and

analysis Can transfer data from phone to computer if 2G cell

network is not available


RAMP Data monitoring and analysis

Real-time data monitoring Real-time data cleaning Real-time analysis Allows for preliminary

analysis even before the data collection is complete


RAMP Produce survey results, rapidly

RAMP deceases the time between data collection and the production of the survey results

The results can be available within days of the last interviews


RAMP So, why use mobile phones to collect data?

Real-time data entry on mobile phones Daily upload of data from mobile phone over

2G cell network to internet database (eliminates paper-to- electronic database transfer)

Real-time data monitoring and data quality checks

Real-time data cleaning Real-time data analysis Rapid production of survey results within hours

or days of last interview


RAMP

Stakeholder benefits


RAMP

Stakeholders

<Insert a photo collage illustrating the various stakeholders (decision-makers; programme managers; evaluators/researchers; fieldworkers)>


RAMP

Stakeholder benefits: decision-makers

• Scalable solution for studies of varying sizes

• Optimizes resource usage and reduces environmental impact

• No software licensing or subscriptions

• No consultants needed• Maintain control of data; data

not analysed by third party• Data rapidly available for

dissemination and decision-making


RAMP Stakeholder benefits: programme managers

• Do not have to reinvent the wheel. The RAMP toolkit offers a variety of knowledge and tools to use and adapt for needs and context of programme managers

• Enables monitoring of survey team work rate, productivity and quality


RAMP Stakeholder benefits: evaluators and researchers

• Incorporate a multitude of question types with custom logic and validation

• Manage and upload surveys in multiple languages

• Export data for custom analysis using any statistical analysis package


RAMP

Stakeholder benefits: fieldworkers

• Builds local capacity for monitoring and evaluation

• Use standard and familiar mobile phones

• No more paper to collect, transport or return

• Automated submission of data when network reception is available


RAMP

RAMP survey sampling approach


RAMP

RAMP survey sampling approach

Simplicity of sampling design to facilitate survey implementation and reduce field costs

Uses standard survey sampling methodologies Uses segmentation to break clusters into small

pieces makes fieldwork more manageable lowers the cost of field surveys

Households are selected by simple random sampling at the last stage

Described in detail in the RAMP toolkit (Volumes 1 and 2)


RAMP

RAMP: a two-stage cluster survey

First stage: selection of clusters using probability proportionate to estimated size (PPES)

Second stage: selection of households segmentation of clusters into smaller segments using

PPES (if there are too many households for simple random sampling [SRS])

SRS of a fixed number of households in a selected segment


RAMP How is the RAMP method different from MIS/DHS?MIS/DHS RAMP

Complex design, uses external consultants to design survey

Simple design, external consultants not needed

Listing of all households is done in all clusters; expensive, often taking several days in each cluster

Divides clusters into manageable-sized segments using standard survey methods; takes <1/2 day

Simple random sampling of households (from the cluster list)

Simple random sampling of households (from the final segment list)

Real-time data cleaning not possible Real-time data cleaning during the survey

Real-time data analysis not possible; results take several months

Real-time data analysis and results/draft report finished within 3 days of last interview

Data analysis done by third-party consultants

Organization performs analysis, building capacity and maintaining control of data


RAMP

Accuracy of the estimates

Sampling bias Very, very small theoretical bias

related to potential errors in the estimated sizes of clusters (there is an option showing how to eliminate this potential bias)

related to potential errors in the estimated sizes of segments (bias likely to be nearly non-existent)

Potential non-sampling bias generally more important than sampling issues Examples including non-response, information bias

including social desirability, faulty questionnaires, interviewer recording errors, data processing errors and others


RAMP

The RAMP survey team


RAMP

RAMP survey team

Programme manager Survey coordinator Local data manager/data analyst Field survey team

field supervisor, interviewers, drivers, local guide Supervisory support and monitoring team


RAMP

How much does a RAMP survey cost?


RAMP

Cost of a RAMP survey (average)Description Cost (US $)

Training (4 or 5 days) including two facilitators 10,623

Field survey, including transportation, daily allowances and accommodation 12,415

Mobile phones, accessories and air time 3,806

Survey administration 2,243

Total in-country expenditure (US $) US $ 29,087


RAMP

Costs of the RAMP pilot surveys


RAMP

Cost of the Kenya RAMP surveyDescription Cost (US $)

Training (4 days) including two facilitators 8,792

Number of trainees 20

Field survey, including transportation, daily allowances and accommodation (vehicles provided free of charge by KRC)

6,767

Number of survey teams 6


Survey administration 3,355



RAMP

Cost of the Namibia RAMP survey

Description Cost (US $)

Training (5 days) including two facilitators 12,425Number of trainees 18Field survey, including transportation, daily allowances and accommodation (vehicles rented) 17,134

Number of survey teams 6Mobile phones, accessories and air time 3,433Survey administration 1,782Total in-country expenditure (US $) US $ 34,774


RAMP

Cost of the Namibia II RAMP survey


Training (4 days) including two facilitators 10,912Number of trainees 18Field survey, including transportation, daily allowances and accommodation (vehicles rented) 14,531

Number of survey teams 6Mobile phones, accessories and air time 6,203Survey administration 3,247Total in-country expenditure (US $) US $ 34,893


RAMP

Cost of the Nigeria RAMP survey


Training (5 days) including two facilitators 10,362

Number of trainees 20

Field survey, including transportation, daily allowances and accommodation (vehicles rented) 11,229

Number of survey teams 6


Survey administration (includes locally hired data manager) 3,943



RAMP

RAMP malaria survey


RAMP Initial focus of the RAMP survey tool: malaria

First RAMP surveys were done to evaluate malaria bed net mass distribution programmes

Sample size was calculated to provide results to guide management decision-making

Questionnaires created based on alignment with MIS/RBM; core RBM/WHO indicators related to nets are measured

Training curriculum and fieldwork procedures have been developed and can be adapted to the local needs and situation

Rapid reporting of results (a survey bulletin and a survey report are produced)


RAMP

Survey methods

Cross-sectional, population-based survey Sampling frame:

First stage: selection of 30 clusters/PSUs using probability proportional to estimated size (PPES) sampling

Second stage: segmentation of the cluster/PSU; one segment chosen using PPES

Simple random sampling (SRS) to choose 10 households in the segment

Sample size: 300 households Precision:

+/- 7-8% for each key net indicator


RAMP

Questionnaires

Survey questionnaires (modelled and adapted from MIS/RBM) Household questionnaire

number of sleeping spaces IRS household characteristics (wealth asset questions, etc.)

Person roster number and age of persons slept in household last

night diagnosis of malaria/fever in children under five and

its treatment Net roster

type of nets, source of nets who slept under each net last night


RAMP

Recruitment and training

Interviewers and field supervisors recruited and trained Training: 4 to 5 days RAMP toolkit: Volume 3, Guide for trainers, provides

sample agenda, curriculum, tools and forms


RAMP

Training content and methods

Content survey background and purpose questionnaires informed consent mobile phone basics interview techniques field procedures field logistics/reporting supervisor training

Methods presentations, role play, group discussion, demonstrations,

field practice (2 outings), review of data collected


RAMP Household questionnaire

v


RAMP

Persons roster


RAMP

Bed net roster


RAMP

RAMP malaria questionnaires: job aid


RAMP RAMP malaria questionnaires: job aid (cont’d)


RAMP

Fieldwork

Most surveys can be finished in one week (5 days)!

6 teams, 1 cluster per day per team, 30 clusters in 5 days


RAMP

Fieldwork

Survey teams: 6 teams of 1 team supervisor and 2

interviewers per team

Survey supervisory support and monitoring team (eg., RCRC, IFRC, MoH): Planning, logistic and financial

responsibilities, field support, daily “quality” rounds, and remote monitoring of data quality


RAMP

A typical day’s schedule

Morning briefing and travel to cluster

Meeting with community leader and guide, preparation

of sketch maps, segmentation, selection of households

to be interviewed

Conduct household interviews

Fieldworker sends data to server

Debriefing at day’s end with supervisory support and

monitoring team

Data monitoring, cleaning and preliminary analysis


RAMP

Survey results bulletin and report


RAMP

Precision <slide to be updated>

Key indicators Point estimate Width of confidence interval, ±%

ITN use, all ages 60 6

ITN use, children <5 years old

71 6

Percentage of ITNs hanging

79 4

% households with > 1 ITN 82 6


RAMP

Reporting and dissemination of results


RAMP

Survey bulletin

A brief, spreadsheet-based results bulletin can be produced 24 hours from last interview

Visually displays the data from analysis tables contained in the analysis plan

Shows main indicators and the results


RAMP

Survey report

Full survey report provides a detailed description of background, methods, results, and discussion of results

Disseminate report to stakeholders and others to prompt action on the survey findings


RAMP

RAMP survey toolkit


RAMP

The RAMP survey toolkit

Three linked publications plus files on the website (www.ifrc.org/ramp) make up the RAMP toolkit Volume 1: Designing a RAMP survey: technical considerations Volume 2: Implementing a RAMP survey: practical field guide Volume 3: Training a RAMP survey team: guide for trainers Files on the website: dynamic and kept up to date

Example database and STATA files for data cleaning and analysis of a sample malaria survey

Latest up-to-date malaria questionnaires and STATA files for data cleaning and analysis

Country reports and results bulletins, information, useful links

Together these provide a methodology, operations protocol and numerous tools to carry out a RAMP survey


RAMP

RAMP website: a dynamic resource

<Insert images of the RAMP website, with “www. ifrc.org/ramp superimposed in large font over

the images”>• See key documents from RAMP surveys being

done around the world. Survey questionnaires, training curricula, fieldwork tools and forms, survey reports—and more!

• Become part of a network of those using RAMP for development work

FOR FURTHER INFORMATION ON RAMP, PLEASE CONTACT:

IFRC HEALTH DEPARTMENT

JASON PEAT, SENIOR OFFICER, MALARIA

TEL. : +41 022 730 4419

EMAIL: [email protected]

THIS PRESENTATION IS PUBLISHED BY

INTERNATIONAL FEDERATION OF

RED CROSS AND RED CRESCENT SOCIETIES

P.O. BOX 372

CH-1211 GENEVA 19

SWITZERLAND

TEL.: +41 22 730 42 22

FAX.: +41 22 733 03 95

rapid mobile phone-based (ramp) survey

Documents

ramp survey methodology

pilot ramp surveys

ramp methodology

ramp tools

ramp survey cost

ramp toolkit

ramp work

rampintroducing ramp