![Page 1: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/1.jpg)
6/25/2010
Impact Evaluation of the
National Agricultural Advisory Services
(NAADS)
Funding support: World Bank and NAADS Secretariat
Sam Benin (IFPRI)
Ephraim Nkonya (IFPRI)
Geresom Okecho (NAADS)
Josee Randriamamonjy (IFPRI)
Edward Kato (IFPRI)
Geofrey Lubade (NARO)
Miriam Kyotalimye (ASARECA)
Francis Byekwaso (NAADS)
![Page 2: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/2.jpg)
What is NAADS impact?
6/25/2010 – Page 2
Tuesday January 23, 2007
What's Behind NAADS Sign Post?
The measurement of NAADS impact is not by the signposts, which only serve to guide and inform the public about the presence of NAADS Technology Development Sites (TDS) or demonstration sites within the vicinity…
NAADS impact is actually measured by [examples]:
Increase in farmers' awareness of new technologies and practices
Availability of improved seeds/breeds to farmers
Utilization of technologies by farmers
Increase in farmers' incomes
…
![Page 3: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/3.jpg)
6/25/2010 – Page 3
NAADS program: goals and objectives
Goal: increasing market-oriented production by empowering
farmers to demand and control agricultural advisory and
information services
Objectives
» Increasing effectiveness, efficiency and sustainability of the
extension delivery service
» Increasing farmers’ access to and sustaining knowledge,
information and communication to the farmers
» Increasing access to and sustaining effective and efficient
productivity-enhancing technologies to farmers
» Creating and strengthening linkages and co-ordination within
the overall extension services
» Aligning extension to government policy, particularly
privatization, liberalization, decentralization and
democratization
![Page 4: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/4.jpg)
6/25/2010 – Page 4
Objectives of impact evaluation study
Assess the incidence of rural public services among farming households
Estimate the impacts of the program on various indicators associated with the objectives of the program, including:
» Empowerment to organize and demand and manage advisory
services;
» Perception of the availability and delivery of advisory services;
» Awareness and incidence and intensity of adoption of improved
technologies and practices;
» Agricultural productivity;
» Market participation; and
» Income, assets, food and nutrition security, and welfare
Analyze and quantify the contribution of other factors that
influence participation in the program and realization of the
outcomes
Assess the return on investments made so far in the program
![Page 5: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/5.jpg)
6/25/2010 – Page 5
Outline of remainder of presentation
Key NAADS program principles and design
features
Conceptual framework for assessing
NAADS impacts
Methods, data, analysis
Results
Summary of key findings and implications
![Page 6: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/6.jpg)
6/25/2010 – Page 6
Key NAADS principles and design features
Demand and control of agricultural technologies and
extension advice economically active poor
farmers through farmer groups
Service provision private sector via technology
demonstration sites (TDS) and trained community
based facilitators (CBFs)
Phased roll out allow for capacity building and
lesson learning
![Page 7: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/7.jpg)
6/25/2010 – Page 7
Feed-back effect / dynamic perspective
NAADS
Farmers empowered to
organize, demand and
manage advisory services
Improvedadvisory services
Increased incidence or intensity of
adoption of info, techs, and practices
Increased awareness of improved or profitable production, NRM, post-harvest, and marketing
information, technologies and practices
ImprovedNRM
Improved food and nutrition security
Increased income
and assets
Increased market participation
Impact pathway
Increased productivity
Reduced unit cost of production
NAADS Impact Pathways (Hypotheses)
Influencing factors
Policy and national level
(P)
Service providers
(S)
Community level(C)
Household level (H)
Farm level(F)
Influencing Factors
Group level(G)
![Page 8: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/8.jpg)
6/25/2010 – Page 8
Main challenge in impact evaluation
Attributing change in the outcome indicator of interest to the program or intervention
Notion is to establish a reliable control group for the participants of the program (or treatment group) and then compare changes in the value of the indicator associated with the two groups
Let y represent the set of outcome indicators of
interest, then the impact of the NAADS program can
be measured by … Average Treatment effect of the
Treated:
![Page 9: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/9.jpg)
6/25/2010 – Page 9
Cohort
(sub-county)
Year
joined
NAADS
Sample size (HHs)
2004 2007 Panel
Early NAADS 2001/02 402 402 323
Intermediate NAADS 2002/03 300 300 202
Late NAADS 2005/06 0 81 81
Non-NAADS 200 417 113
Conducted household (HH) surveys in 2004 and 2007 (719
panel HHs). Categorized NAADS presence into three cohorts
according to year NAADS program introduced into sub-county
Data: stratified panel data
![Page 10: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/10.jpg)
6/25/2010 – Page 10
Who is a NAADS participant?
Direct participant (72 HHs) if farmer:
» member of a NAADS-participating farmer group
Indirect participant (354 HHs) if:
» not a member of a NAADS-participating farmer group, but
» received visit from a NAADS service provider
or
» received visit from a community based facilitator
Non-participant otherwise (293 HHs)
![Page 11: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/11.jpg)
6/25/2010 – Page 11
Analytical methods and estimation procedure
Panel data of 719 households
Ej [ y2007, j – y2004, j | NAADSj=1] – Ei [ y2007, i – y2004, i | NAADSi=0]
Direct and indirect effects estimated by combining
matching and regression methods
» Propensity Scores (probability of participation) used to
match participants and non-participants
» Simple double-difference (DD) based on matched sample
» Two-stage weighted regression (2SWR) using based on
matched sample and using propensity scores as weights
![Page 12: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/12.jpg)
6/25/2010 – Page 12
Selected Results
![Page 13: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/13.jpg)
6/25/2010 – Page 13
Adoption of improved technologies%
of
ho
use
ho
lds
Positive effect of NAADS on adoption of improved technologies
and practices
Direct participants caught up with indirect participants in 2007
0
20
40
60
80
NA
AD
S-D
irec
t
NA
AD
S-I
nd
irec
t
no
n-N
AA
DS
NA
AD
S-D
irec
t
NA
AD
S-I
nd
irec
t
no
n-N
AA
DS
NA
AD
S-D
irec
t
NA
AD
S-I
nd
irec
t
no
n-N
AA
DS
NA
AD
S-D
irec
t
NA
AD
S-I
nd
irec
t
no
n-N
AA
DS
NA
AD
S-D
irec
t
NA
AD
S-I
nd
irec
t
no
n-N
AA
DS
Crop improved
varieties
Recommended
spacing and planting
Pesticides Inorganic fertilizers Livestock improved
breeds
2004 2007
![Page 14: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/14.jpg)
Crop and livestock productivity
6/25/2010 – Page 14
% c
ha
ng
e
Positive and significant impact on crop and livestock productivity
Greater direct effect on livestock productivity, but greater indirect
effect on crop productivity
Relatively smaller total effect on livestock productivity is unclear
0
10
20
30
40
50
Direct effect Indirect effect Total effect
Crop Livestock
![Page 15: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/15.jpg)
Commercialization (sales of outputs)
6/25/2010 – Page 15
% c
ha
ng
e
Small positive impact on commercialization of agriculture
Positive direct impact on sale of crop and livestock output
Negative but insignificant indirect impact on sale of livestock
output
-2
0
2
4
6
Direct effect Indirect effect Total effect
Crop Livestock
![Page 16: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/16.jpg)
Gross agricultural income
6/25/2010 – Page 16
% c
ha
ng
e
Significant positive impact on overall agricultural income
Direct effect is about 40% greater increase in the per capita
agricultural income of participants compared to non-participants
0
10
20
30
40
Direct effect Indirect effect Total effect
![Page 17: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/17.jpg)
Perception of change in welfare
6/25/2010 – Page 17
More participants than non-participants perceived that their
standard of living had improved between 2000-07
More non-participants than participants perceived that their
situation had not changed or it had worsened between 2000-07
Pro
po
rtio
n o
f h
ou
seh
old
s
0
0.2
0.4
0.6
00-04 04-07 00-04 04-07 00-04 04-07 00-04 04-07 00-04 04-07 00-04 04-07
NAADS non-NAADS NAADS non-NAADS NAADS non-NAADS
Average wealth level Access to adequate food Nutritional quality of food
Improved No change Worsened
![Page 18: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/18.jpg)
Distribution of income effects (direct)
6/25/2010 – Page 18
% c
ha
nge
Several factors have enhanced the program’s effects
For example, the program has been more effective among men,
the younger generation, those with primary education, the asset
poor, and those living in the Eastern and Northern Regions
-60
-40
-20
0
20
40
60
80
Mal
e
Fem
ale
<4
0
>4
0
no
ne
pri
m
po
st-p
rim
Po
or
Mid
dle
Ric
h
<5
km
>5
km
Cen
tral
Eas
t
No
rth
Wes
t
Gender Age Education Assets Dist to
mkt
Region
![Page 19: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/19.jpg)
6/25/2010 – Page 19
Returns to Investments
![Page 20: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/20.jpg)
6/25/2010 – Page 20
0%
20%
40%
60%
80%
100%
2001-02 2002-03 2003-04 2004-05 2005-06 2006-07
Market
Dev't
Technology
Dev't
Advisory/Inf
ormation
Planning,
Monitoring, Admin
Institutional
Dev't
Perc
en
t
• In surveyed sub-counties: about UGX 7 billion spent, shifted
over time from farmer institutional development to
Advisory/Information services to TDS (figure)
NAADS Expenditures• National total: about UGX 110 billion (2000 value) spent between
2001/02-2006/07
![Page 21: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/21.jpg)
6/25/2010 – Page 21
Returns to NAADS Investment
Estimated benefit-cost ratio is 1.6-3.5
24.3
52.0
21.6
46.2
18.2
38.8
-14.7 -14.7 -13.2 -13.2 -11.2 -11.2
0
1
2
3
4
-20
0
20
40
60
Low ATT High ATT Low ATT High ATT Low ATT High ATT
Discount Rate=5% Discount Rate=8.5% Discount Rate=14%
Rat
io
UG
X B
illi
on
Discounted Costs (UGX) Discounted Benefits (UGX) Benefit-Cost Ratio
Cost adjustments
Imputed cost district governments and opportunity cost of time of CBFs
Cost of related farm inputs and operations: 35% of gross income
Cost of raising public funds: 0.125% interest on loan
![Page 22: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/22.jpg)
6/25/2010 – Page 22
Summary of key findings and implications (1)
NAADS program has had a positive impact on overall
welfare between 2000 and 2007
» More NAADS participants than non-participants
perceived that their standard of living had improved
» More non-participants than participants perceived that
their situation had not changed or it had worsened
Positive impact on crop and livestock productivity and
overall agricultural income
» 30-40% greater increase in the per capita agricultural
income of participants compared to non-participants
» The program has been more effective among men, the
younger generation, the asset poor, and in the worse-off
regions
![Page 23: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/23.jpg)
6/25/2010 – Page 23
Participation in program increased the capacity of
farmers to demand production and post-production
advisory services
Higher physical capital endowment and level of
education are major factors associated with higher
demand for advisory services
» Efforts to build capacity of farmers to demand advisory
services should be directed more to resource-poorer
farmers
» Building capacity of farmers should be supported by
other programs that help farmers to acquire productive
assets and improve their education
Summary of key findings and implications (2)
![Page 24: Impact Evaluation of the National Agricultural Advisory ...ussp.ifpri.info/files/2010/06/naads_benin_6-24-08.pdffarmers to demand and control agricultural advisory and information](https://reader033.vdocument.in/reader033/viewer/2022050206/5f596032723f283a765a0cf7/html5/thumbnails/24.jpg)
6/25/2010 – Page 24
Thank you