Nicholas Minot (IFPRI/Uganda)Atsuko Toda (IFAD/Vietnam)

Nguyen Ngoc Ahn (DEPOCEN)

Presented to IFAD/Rome From Hanoi, 29 March 2012

RIMS+ surveys: A tool for project design and evaluation

Outline of presentation

Background on RIMSChanges in RIMS+ Cost and implementation issuesBenefits: Results from three RIMS+

surveys in VietnamSummary and conclusions

Background on RIMSResults and Information

Management System (RIMS)Provides information at three

levels1st -level results refer to project

activities and outputs2nd -level results relate to project

outcomes and reflect changes in beneficiaries behavior, improved performance and sustainability of groups, institutions and infrastructure

3rd -level results are associated with project impact on child malnutrition and household living standards.

We focus on the household survey used to collect third-level results

Background on RIMSRIMS survey guidelinesShould be implemented for

large, national IFAD projectsShould be done before,

during, and at end of projectSample size: 900 beneficiary

householdsReturning to same

households not recommendedConcern about concentration

of IFAD program effortsAdministrative complications

of finding old households

Background on RIMSRIMS questionnaireObjective is to measure

assets and child nutrition6 pages (plus cover)

divided into three sectionsSection 1 – Household

demographicsSection 2 – Housing,

assets, and food securitySection 3 –

Anthropometry

Background on RIMSStandardization of RIMS

questionnaireEnsures comparability across

countriesMakes analysis relatively quickAssures qualityBut little flexibility in

questionnaire design & analysis

Does not collect intermediary indicators

Changes in RIMS+Overview of changesChanges Rationale1. Expanded questionnaire

Collect additional information to diagnose farmer constraints, improve design of interventions, and measure impact on intermediate indicators

2. Use of control group

Better measurement of impact of project by controlling for broader changes in rural conditions

3. Additional training and supervision

Improve quality of data

4. GPS to geo-reference households

Facilitate return to same households (panel) and better supervision of enumerators

5. Flexible questionnaire & analysis

Address information needs of the IFAD project and IFAD planning in general

Changes in RIMS+1. Expanded questionnaire (cover + 14 pages)

RIMS+ RIMS New info in RIMS+A. Member characteristics

1. Household demographics

+ ethnicity, school attendance, & reasons for not attending

B. Housing 2. Survey questions

+ roof material, ownership status, location of toilet

C. Assets 2. Survey questions

+ agricultural equipment

D. Land (no info) Farm size, ownership, irrigation, distance

E. Crop production

(no info) Production, sales, & prices for 25 crops; cost of 6 inputs

F. Livestock & fisheries

(no info) Herd size, sales, & costs for 12 types of animals, use of vet services, type of feeding

Changes in RIMS+1. Expanded questionnaire (continued)

RIMS+ RIMS New info in RIMS+G. Extension & market access

(no info) Access to extension, who uses, cooperatives, details of sales, distance to markets

H. Non-farm activities

(no info) Income and business expenses for 11 non-farm income sources, gender roles

I. Food security 2. Survey questions

+ coping strategies and quality of diet

J. Credit & borrowing

(no info) Access to credit, info on loans received

K. Socio-Economic Development Plan

(no info) Knowledge of and participation in SEDP process

L. Risk & vulnerability

(no info) Perceived risk of six natural disasters

M. Anthropometry

3. Anthropometry

No new information

Changes in RIMS+2. Use of control groupControl group is 300 households that are similar to

beneficiaries but not in project areaUseful to control for changes in rural areas due to other

factorsBeneficiary households

Control households

Impact according to current before-after comparison

Actual impact using info from control group

Example 1

Income rises 8%

Income rises 4% due to economic growth

Suggests that project caused 8% increase in income

Actually, only a 4% increase due to project

Example 2

Income does not change

Income falls 4% due to drought

Suggests that project had no effect

Actually, 4% increase in income due to project

Changes in RIMS+2. Use of control group (continued)

TimeBefore project After project

Control group

Beneficiary householdsOutcome indicator

Actual effect of project Before-after

difference

is hypothetical path of beneficiary households without the project, based on growth in control group

Changes in RIMS+3. Additional training

and supervisionBecause questionnaire is

longer and somewhat more complicated, need for additional training & supervision of enumerators

IFPRI & DEPOCEN prepared detailed enumerator manual

DEPOCEN provided 5 days of training plus testing of questionnaire

DEPOCEN also provided additional supervision during data collection, particularly important in first week of data collection

Changes in RIMS+4. Use of GPS unitsGPS units are sometimes

used in RIMS surveysMain purpose is to make

it easier to find household to interview in later round of survey

Additional benefit of verifying that enumerators have visited households in village

Changes in RIMS+5. Flexible questionnaire & analysis of results

Original RIMS is analyzed in a “black box”Advantage is analysis is fast, reliable,

and comparableBut little opportunity to customize

results for project RIMS+ questionnaire can be

customized for project

Type of IFAD project

Possible customization of questionnaire

Farmer training & extension

Access to extension, sources of info, perception of usefulness, adoption of advice, yield

Linking farmers to market

Travel time to markets, types of buyers, degree of competition, prices received, share sold

Promotion of non-farm enterprises

Number & composition of NFEs, profitability, training needs, perceived constraints, factors affecting success

Improved access to credit

Sources of credit, interest rates paid, use of credit, reasons for use of informal credit, factors affecting repayment rate

Changes in RIMS+5. Flexible questionnaire & analysis of results

RIMS+ analysis can be customized to address questions relevant for project design & implementationIs access to extension services different

for female-headed farmers? Can pepper be successfully grown by

small-scale farmers with limited resources?

Is targeting landless households more (or less) pro-poor than targeting farmers with less than 0.5 hectares?

Is satisfaction with project services higher in one district than in another?

Expanded questionnaireMore information and more complicated

questionnaire Requires additional training and supervision Longer interview time (double at least)

Requires a new data entry programSeparate data entry in CSPro for 1200 questionnairesAt least 2 days in preparing CSpro entry data formAnother 2 days for training in data entry in CSPro in

addition to RIMS training.Increased complexity in analysis and reporting

Cost and implementation issues

Use of control groupIncreased workload with financial implication

(additional 300 non-project household)Implementing survey in non-project area is

more difficult due to logistics, cooperationData entry in both RIMS and CSPro

RIMS software to enter RIMS core questions for 900 beneficiary households

Data entry in CSPro for full questionnaire for1200 household sample

Additional training/supervisionProject managers do not see immediate benefit

Cost and implementation issues

Use of GPS

Cost and implementation issues

Increased training time (1/2 day) and additional time at household (10 minutes)

Not easy to use due to language barrierAdditional burden due to the fact that

interviewers already have to carry weight and scale

Cost and implementation issues

Component First-time costs Per survey costsExpanded questionnaire in data collection

Already carried out under IFAD-IFPRI Partnership

Interview time is approximately doubled

Use of control group

No fixed cost Increases field costs by 50-100%

Additional training & supervision

Enumerator manual prepared under Partnership

Approximately US$ 10-15k per survey

Use of GPS units Cost to purchase = US$ 100 x 20 units = US$ 2000

Modest - GPS units can be shared across projects or rented

Analysis of data Large initial cost of preparing analysis programs, already undertaken by Partnership

For standard analysis, negligible. For customized analysis, requires Stata skills

Cost estimates

Questions

Results of Vietnam RIMS+

Which crops are pro-poor?How does crop commercialization vary across farmers?Do female-headed farmers have equal access to

modern inputs? How important is income from non-farm activities?How to farmers perceive the risks of natural disasters? Is food security threatened by crop commercialization? How involved are farmers in the preparation of the

Socio-Economic Development plans? Will raising farmer income improve child nutrition?

Which crops are pro-poor? Results of Vietnam RIMS+

• Rice is grown by majority of the poor, but fewer high-income households

• Maize, groundnut, red onion, bananas, tea, and vegetables are grown by both poor and non-poor

• Avocado, mango, durian, pepper, sugarcane, coffee, and cashew are grown disproportionately by high-income farms

• This is not to say they can’t be grown by poor farmers, but any untargeted support to these crops will not be pro-poor

Is input use less among female-headed households? Results of Vietnam RIMS+

• Not much evidence that input use per hectare is lower• But smaller farm sizes lead to smaller crop production and lower

income

What is the importance of non-farm income? Results of Vietnam RIMS+

• Even the 20% of farms with the smallest area (less than 0.10 hectares) earns the bulk of their income from crop production

• 45% of smallest farms rent, sharecrop, borrow, or use illegally other land

How do farmers perceive the risk of different natural disasters?

Results of Vietnam RIMS+

• Perception of disaster risk varies by province• Also, perception of likely losses is greater for poor households

Is food security threatened by commercialization? Results of Vietnam RIMS+

• Commercialization is defined as the share of the value of crop production that is sold

• Relationship holds even after controlling for per capita income and farm size in regression analysis

Will raising farmer income improve child nutrition? Results of Vietnam RIMS+

• Yes, but effect is weak• Many other variables influence child nutrition: sanitation, health care,

education, child rearing practices, etc.

-50

5Z-

scor

es

10 12 14 16 18Log of per capita income

Length/height-for-age Z-score Weight-for-length/height Z-scorelowess haz06 lnpcinc lowess whz06 lnpcinc

Summary & conclusionsAdvantages of RIMS+ Additional costs

Expanded questionnaire gives much more information for diagnosis of problems and measuring project-specific indicators

Fixed costs of preparing questionnaire, manual, and data analysis programs have already been incurred by IFAD-IFPRI Partnership

Control group allows better measurement of impact of project, taking into account trends in rural areas

Per-survey cost is increased 3x by expanded questionnaire and control group. GPS and additional quality control also imply costs.

Additional training & supervision provides higher-quality data

Cost of analyzing basic results using existing programs, but customization of questionnaire or analysis implies additional costs.

Use of GPS units makes it easier to revisit same respondents in later rounds of surveyAbility to customize questionnaire & analysis to meet project needs

Summary & conclusions

RIMS+ surveys probably not suitable for all IFAD projects because of additional costs

Conditions under which it is most suitable: IFAD project design is flexible, can be revised

in light of new information from survey IFAD project focuses on a new topic or new

region, so there is a need for informationThere are gaps in knowledge about farm

household livelihoods and behavior relevant to project

IFAD project is relatively large, implying an adequate M&E budget

When is RIMS+ most suitable?

Additional issuesSize of control group

At the moment, 900 treatment to meet standard RIMS requirement and 300 control

But typically control group is similar sizeIt would reduce costs to develop a Core Module and

additional modules that are selected depending on project (e.g. agricultural marketing, credit, extension)

RIMS+ would require additional capacity building for IFAD project staff

Project has prepared an enumerator manual and data entry programs and could also prepare an implementation guidelines if needed

Summary & conclusions