time matters: ‘short’ term vs. ‘medium’ term impacts
DESCRIPTION
Soumya Balasubramanya Making Impact Evaluation Matter: Better Evidence for Effective Policies and Programs September 3-5, 2014 Manila, PhilippinesTRANSCRIPT
Phot
o: D
avid
Bra
zier/
IWM
I Ph
oto
:Tom
van
Cak
enbe
rghe
/IW
MI
www.iwmi.org Water for a food-secure world
Phot
o: A
ndre
a Si
lver
man
/IW
MI
Name Date Event Location
Time matters: ‘short’ term vs. ‘medium’ term impacts
1
Soumya Balasubramanya Making Impact Evaluation Matter: Better Evidence for Effective Policies and Programs
September 3-5, 2014 Manila, Philippines
Balasubramanya, S., Pfaff, A., Bennear, L., Tarozzi, A., Ahmed, K.M., van Geen, A. (2014). Evolution of households’ responses to the groundwater arsenic crisis in Bangladesh: information on environmental health risks can have increasing behavioral impact over time. Environment and Development Economics, 19(05), pp. 631-647.
www.iwmi.org Water for a food-secure world
Summary • Impact of an arsenic information and testing campaign on
adoption of risk-avertive behaviors over time. • Followed households over two time periods and tracked
behaviors and perceptions over 2003-05 and 2005-08. • Impact of information on risk-avertive behaviors depends
on ability to recall information – Households that recalled: behaviors were persistent and double
the # of households took up behaviors over time – Households that didn't’ recall: observed perverse behaviors
2
www.iwmi.org Water for a food-secure world
Background
• GW is primary drinking source; accessed through private tubewells (shallow, < 30 m; 90% of wells are shallow )
• GW contaminated with naturally occurring arsenic; unpredictable in shallow aquifer
• WB- funded countrywide groundwater testing program
b/w 1999-2003 • Tested households’ tubewells for arsenic; painted spouts
red/green • Encouraged households to share green wells • Village level awareness campaigns • Media campaigns
3
www.iwmi.org Water for a food-secure world
What happens within two years of testing?
• Schoenfeld, 2005 – Census of all tubewells tested by WB in 2003; in 62
villages of Araihazar Upazila – Interviewed the wives of the well owners – Recorded color of well (observed by enumerator) – Elicited safety perception (asked respondent if well was
safe or unsafe) – Elicited switching behavior b/w 2003-05
• Switching behaviors from unsafe wells ~20%
4
www.iwmi.org Water for a food-secure world
What happens over time?
• Household values both convenience (well nearby) and safe drinking water (green well) – Switch if cost (switching) < benefit (switching)
• Can the costs and benefits change over time? – ‘Social learning’ by observing peers (Miguel and
Kremer, 2004; Munshi and Myaux, 2006) – Household ‘loses’ information – Concern about health risk reduces (Karlan et al., 2010) – Safe well owners reduce access (Fehr and Fischbacher,
2003; Hanchett et al., 2002)
5
www.iwmi.org Water for a food-secure world
Two Questions
• Is well-switching a persistent behavior
change?
• How does adoption of well-switching evolve over time?
6
www.iwmi.org Water for a food-secure world
Methods and Data
• Follow-up study on households whose switching behaviors had been previously studied in Schoenfeld, 2005 – Randomly selected subset of households in 59 villages – Interviewed same person as in previous survey – 1557 households/wells (1038 unsafe; 519 safe) – Use sampling weights
7
www.iwmi.org Water for a food-secure world
Data
8
www.iwmi.org Water for a food-secure world
Consistency in recall
9
• 78% recalled switching behavior of 2003-2005 consistently • Always consider households with consistent recall
of 2003-2005 switching behavior
• By 2008, no color left on any well spout • 78% recalled well safety consistently (well color = safety perception in 2005 = safety perception in 2008)
www.iwmi.org Water for a food-secure world
Did impact of testing increase or erode over time?
10
• Consider households with consistent recall of well tests • If switched in 2003-2005, check if switches were
persistent – 16% of households at unsafe wells switched during 2003-2005
• 14% stayed put • 2% switched to a second well they thought safe
• If did not switch during 2003-2005, check if switched
during 2005-2008 – 17% of households at unsafe-tested wells had switched – Switching over five years doubles that over two years
www.iwmi.org Water for a food-secure world
11
www.iwmi.org Water for a food-secure world
What about those that ‘forget’ information?
12
• Consider households with inconsistent recall of well tests (remembered correctly in 2005, but not in 2008)
• In 2005
– Paint visible on all wells (1039 red/ 519 green) – Almost all households recalled well safety correctly (97%
unsafe/ 95% safe)
• In 2008 – Paint washed off – 77% unsafe tested households recalled safety accurately – 81% safe tested households recalled safety accurately
www.iwmi.org Water for a food-secure world
Would it help to reinforce provision of arsenic information?
13
• Safe-tested hhs that changed safety perceptions after
2005: 26% switched in 2005-2008
• Unsafe-tested hhs that changed safety perceptions after 2005: 11% switched in 2005-2008
www.iwmi.org Water for a food-secure world
14
www.iwmi.org Water for a food-secure world
Discussion • Behavioral:
• Recall of arsenic information changes over time • Recall is critical for behavior
• Evaluation: • Short-term evaluations present a snap shot • Impacts are heterogeneous
• Policy perspective:
• Cheap, responsible for most reduction in arsenic exposure (World Bank, 2005; Johnston et al., 2010; Ahmed et al., 2006)
• Supporting continued well testing: new well constantly installed; 1/3 wells untested
15
www.iwmi.org Water for a food-secure world
17
www.iwmi.org Water for a food-secure world
18