Selecting indicators for monitoring and evaluation of agriculture and rural development in countries with weak statistical capacity

based on FAO/WB Sourcebook

Presentation by Naman Keita, FAO

ISI SATELLITE MEETING ON AGRICULTURAL STATISTICS Maputo, Mozambique 13-14 August 2009

SESSION 4: DETERMINING A MENU OF INDICATORS AND A CORE SET OF DATA

THREE PILLARS OF THE GLOBAL STRATEGYTHREE PILLARS OF THE GLOBAL STRATEGY

1.1. MENU OF INDICATORS AND SET OF CORE DATAMENU OF INDICATORS AND SET OF CORE DATA

2.2. INTEGRATION OF AGRICULTURAL STATISTICS INTO INTEGRATION OF AGRICULTURAL STATISTICS INTO NATIONAL STATISTICS SYSTEMNATIONAL STATISTICS SYSTEM

3.3. AGREED SUITE OF METHODOLOGIES FOR AGREED SUITE OF METHODOLOGIES FOR INTEGRATIONINTEGRATION

INDICATORS AND DATA/STATISTICS IN THE GLOBAL INDICATORS AND DATA/STATISTICS IN THE GLOBAL STRATEGYSTRATEGY

• An indicator provides a broad signal of change, direction, or state of being.

• The statistics are the numbers that represent each indicator for a point in time, scope, and coverage

• The data items are those needed to provide the statistics describing each indicator.

• The data include the crop area, yield entering into the estimate of production of each crop, livestock numbers for meat production, etc.

FAO/WB SOURCEBOOKFAO/WB SOURCEBOOK

FOCUSES ON:FOCUSES ON:

HOW TO MONITOR AND EVALUATE (MEASURE) THE HOW TO MONITOR AND EVALUATE (MEASURE) THE IMPACTIMPACT OF OF AGRICULTURE AND RURAL DEVELOPMENT POLICIES IN COUNTRIES AGRICULTURE AND RURAL DEVELOPMENT POLICIES IN COUNTRIES WITH WITH WEAK STATISTICAL CAPACITYWEAK STATISTICAL CAPACITY

DEFINE A FRAMEWORK OF DEFINE A FRAMEWORK OF STANDARDISED APPROACHES FOR STANDARDISED APPROACHES FOR SELECTING INDICATORSSELECTING INDICATORS

DEFINE A DEFINE A MENU OF 86 INDICATORSMENU OF 86 INDICATORS AND A LIST OF AND A LIST OF 19 PRIORITY 19 PRIORITY INDICATORSINDICATORS FOR MONITORING AGRICULTURE AND RURAL FOR MONITORING AGRICULTURE AND RURAL DEVELOPMENT INITIATIVES AT PROJECT, NATIONAL, REGIONAL DEVELOPMENT INITIATIVES AT PROJECT, NATIONAL, REGIONAL GLOBAL LEVELSGLOBAL LEVELS

DATADATA REQUIREMENT, REQUIREMENT, SOURCESSOURCES AND STATISTICAL CAPACITY AND STATISTICAL CAPACITY

::

LOGFRAME AND FREQUENCY OF MONITORING LOGFRAME AND FREQUENCY OF MONITORING VARIOUS INDICATORSVARIOUS INDICATORS

Impact Indicators(Ultimate goal)

Outcome Indicators(behavioral change)

Output Indicators(Goods and Services)

Input Indicators (Material, financial, human)

Medium to Long Run (maybe 5 years by the time surveys are carried out)

Medium Term - Ideally annually – maybe every 2 – 3 years

Short – Medium Term - Ideally more than once a year or annually

Short Term - Ideally every three months or annually

Outcome and impact: RESULT INDICATORS

SELECTION OF INDICATORSSELECTION OF INDICATORS

What makes a Good IndicatorWhat makes a Good Indicator

SS SPECIFIC and SENSITIVE to the changes induced as a result of actions taken

MM MEASURABLE progress can be shown and is not easily manipulated

AA ATTAINABLE and APPLICABLE to the policy action taken

RR RELEVANT to the areas in question

TT TIME BOUND and TRACKABLE by showing changes over time

MAIN CRITERIA USED TO SELECT MAIN CRITERIA USED TO SELECT 19 PRIORITY INDICATORS19 PRIORITY INDICATORS: :

RELEVANCE (FOR SUB SECTORS ONLY ONE WAS SELECTED) RELEVANCE (FOR SUB SECTORS ONLY ONE WAS SELECTED)

COMPARABILITY COMPARABILITY

AVAILABILITY (BASED ON COUNTRY STUDIES AND VALIDATIONS)AVAILABILITY (BASED ON COUNTRY STUDIES AND VALIDATIONS)

NINETEEN PRIORITY INDICATORS (SELECTED FROM MENU OF 86)NINETEEN PRIORITY INDICATORS (SELECTED FROM MENU OF 86)

1 Public spending on agriculture as a % of GDP from agriculture sector

11 % of the rural population using financial services of formal banking institutions

2 Public spending on agricultural input subsidies as a percentage of total public

spending on agriculture

12 Public investment in agricultural research as a % of GDP from agriculture sector

3 Prevalence (%) of underweight children under five years of age in rural areas

13 Irrigated land as a % of crop land

4 Food production index 14 % change in sales/ turnovers of agro-enterprises

5 % annual growth in agricultural value added 15 % of farmers who are members of producer organizations

6 Rural poor as a proportion of total poor population

16 Agricultural withdrawal as % of total freshwater withdrawal

7 % change in yields of major crops of the country

17 Proportion (%) of land area formally established as protected area

8 % annual growth in value added in the livestock sub-sector

18 % change in soil loss from

watersheds

9 Capture fish production as % of existing stock ( or a rating of state of major capture fish stocks relevant to

exports and local food) REPLACED BY% change in capture and aquaculture fish

production used for human food

19 % land area for which there exists a legally recognized form of land tenure

10 Proportion of land area covered by forest (%)

INDICATORS, DATA REQUIREMENTS, DATA SOURCES (EXAMPLE 6 SECTOR WIDE INDICATORS)

Indicator data requirements Data Sources

Sector Wide indicators for agriculture and rural development

Public spending on agriculture, subsidies, and infrastructure

Government budget allocations, and spending related to agriculture. Agriculture includes forestry and fisheries

Ministry of Finance, National Accounts, Planning commissions, Donor reports

Public spending on rural infrastructure including health and education

Government budget allocations, and spending related rural areas

Ministry of Finance, National Accounts, Planning commissions, Donor reports

Percent of rural children that are underweight compared to national level

Anthropometric data Anthropometric Surveys

Food production index, food security Area, production and yield for food crops, livestock numbers and production of meat, milk, eggs, fish captured and cultured, and other food products, non-food use of food products, food imports and exports

Agricultural Census, surveys of agricultural enterprises, processors, fish landings, administrative data such as imports, exports. Food Balances and Household consumption surveys

GDP growth from Agriculture value added. Estimates of total production and value for all commodities produced in the country; including that from small holders/household plots minus estimates of the cost of inputs such as seed, feed, energy, fertilizer, labor, etc. Agriculture includes forestry and fisheries

Censuses and surveys agricultural enterprises, farm and rural households, administrative and processor. data

Rural poor as a percent of total poor population

Household income and consumption estimates for national poverty lines. Purchasing Power Parities for comparisons across countries

Household Surveys. International Comparison Program for comparisons across countries

CORE SET OF DATA RECOMMENDED IN THE GLOBAL STRATEGY

Participantobservation

Sentinel sitesurveillance

Beneficiary assessment

LSMS

Household budgetsurveyFish landing Surveys

Censuses

CWIQ

Community Surveys

Windscreensurvey

P.P.A

Casestudy

Purposive selection

Quotasampling

Small prob.sample

Large prob.sample

Census

Direct measurement

Questionnaire(quantitative)Questionnaire(Qualitative)

Structuredinterview

Openmeetings

Subjective assessments

Conversations

MAIN DATA SOURCESMAIN DATA SOURCESSurveys vs. non-formal appraisal methods

1 2 3 4 5

  Sample size DurationVisits to household

Question-naire size Cost ($m)

Population census Full coverage 3-6 months 1 4-8 15-25

Agricultural census/survey 20 000-50 000 1-1.5 years 2-4 8-12 5-10

LSMS/integrated survey 5 000-10 000 1-1.5 years 2 40+ 1-2

Household budget survey 4 000-10 000 1-1.5 years 15-25 15-20 1-2

Community survey 100-500 4-6 months 1 4-6 0.2-0.4

Service delivery survey (CWIQ) 10 000-15 000 2-3 months 1 8 0.2-0.4

Focus group interviews 40-50 2-3 months 1-3 - 0.05-0.1

Windscreen survey 10-20 2-3 weeks 0   0.01

KEY FEATURES OF DIFFERENT SURVEYS AND INDICATIVE COST

A TEN YEAR CYCLE OF DATA COLLECTION ACTIVITIES

Instruments and Sources YEAR

1 2 3 4 5 6 7 8 9 10

1 Census of agriculture

2 Periodic agricultural production survey

3 Farm management survey

4 Rural household income—expenditure survey

5 Food consumption survey

6 Survey of post—harvest losses

7 Survey of agric. service establishments

S Rural labour force survey

9 Soil survey

10 Livestock census

1! Special agricultural censuses and surveys

12 Census of population and housing

13 Demographic survey

14 Census of industrial establishments

15 Survey of rural household industries

16 Urban household income—expenditure survey

1. IMPORTANCE OF LINKING DATA USERS AND PRODUCERS FOR:- relevance- avoiding duplication FAO/WB STUDY FOUND THAT M&E AND STATISTICS SYSTEMS ARE DISCONNECTED IN MOST OF THE 5 PILOT COUNTRIES

2. CHALLENGES OF DECENTRALISATION AND CORRESPONDING DATA REQUIREMENT AND NEED FOR METHOGOLOGICAL RESPONSES (Small area estimation, better use of administrative data etc..)

3. SELECTION OF INDICATORS: DOES THE STATISTICS SYSTEM HAS THE CAPACITY? NEED FOR INTEGRATED STATISTICAL PROGRAMME WITH CORRESPONDING STATISTICAL CAPACITY BUILDING COMPONENT

4. NINETEEN PRIORITY INDICATORS OF FAO/WB SOURCEBOOK PROPOSED AS STARTING POINT FOR MENU OF INDICATORS IN GLOBAL STRATEGY

CONCLUDING REMARKSCONCLUDING REMARKS

THANK YOUOBRIGADO