www.3ieimpact.org hugh waddington what works in wash? evidence from systematic reviews hugh...
Post on 21-Jan-2016
216 Views
Preview:
TRANSCRIPT
www.3ieimpact.orgHugh Waddington
What works in WASH? Evidence from systematic
reviews
Hugh Waddington
Geneva Evaluation Week
7 May 2015
International Initiative for Impact Evaluation
www.3ieimpact.orgHugh Waddington
3ie is managing 3 linked work streams
Impact evaluations•Funding available for 2 studies•RFQ (2-stage awards)•Review of proposals from June 7th
•Presentation of baseline findings in Kathmandu (Nov-Dec 2015)•Draft report in December 2016
Mid-term review•WSSCC medium-term strategy•IFMR contracted•Draft report by January 2016
Systematic reviews•2 questions•Call closes May 12•Contracting in June•Draft reports in 2016
www.3ieimpact.orgHugh Waddington
Call for SR proposals: deadline 12 May
2 questions developed with WSSCC and 3ie-WSSCC programme advisory group
•What is the effectiveness of interventions aiming to promote sanitation and hygiene behaviour change in communities?
•To what extent has the sanitation and hygiene sub-sector taken into account the life-cycle approach in the design, implementation, maintenance and use of programmes during the MDG period?
www.3ieimpact.orgHugh Waddington
“Literature reviews are like
sausages... I don’t eat
sausages as I don’t know
what goes into them”
Dean Karlan
www.3ieimpact.orgHugh Waddington
3ie Evidence gap map
gapmaps.3ieimpact.org
www.3ieimpact.orgHugh Waddington
A sector-wide logic model
Inputs Activities Outputs Outcomes Impacts
WASH sector programmes:
Improving operator performance
Private sector delivery, contracting out
Pricing reforms e.g. subsidies
1.1 1.2
1.3 Decentralized delivery (e.g. CDD)
Water supply services
Water supply facilities
Better, more reliable access
Time use, reduced
travel time
Improved child nutritional
status, safety Reduced
mortality
Better educational attainment
Higher income and consumption
Reduced costs of health care
Poverty reduction
Happiness
1.4 Information and education communic-ation (IEC) services
Water treatment
Better quality water: less fecal contamin-ation in water source
1.5 Improved water use practices: less fecal contamin-
ation in drinking water
1.6 Reduced morbidity, and better maternal
and child health
outcomes
Behavior change campaigns (BCC) about water use
BCC about sanitation practices
1.7 1.8
1.9 Access to safe sanitation
Improved sanitation practices: less fecal contamin-
ation in environment
1.10
Sanitation services Sanitation
facilities User
satisfaction Empowerment:
dignity, safety
www.3ieimpact.orgHugh Waddington
Key evidence gaps• 138 Impact and 26 SRs
of WASH evidence in low and middle income countries.
• Mostly measure diarrhoeal health outcomes (not very rigorously)
• Very few studies examining demand-side approaches (eg CLTS, san marketing)
• Few estimate impacts for sanitation and hygiene programmes as part of scale up
• No rigorous prospective studies of sanitation programmes in SSA
• Few on governance
www.3ieimpact.orgHugh Waddington
3ie systematic reviews
BEHAVIOURAL CONTEXT (Pre-project period)
Individual characteristics and abilities: Age, gender, pre-existing WASH practices, knowledge, skills, self-efficacyHousehold demographics: Socioeconomic status, household structure, religionCommunity/socio-political structure: Social norms, access to WASH technologies and materials, health system structure, governmental policies and financial support
Early introduction
Communities and individuals are first introduced to WASH technologies and promotion messages. • May have previously been exposed, or may be new to campaigns.• WASH technologies or messaging may be new or exciting• Promoter may be viewed as an influential person
Maintenance during project period
Promotion continues to end of intervention period. • Health promoters assist with problems• Participants able to practice behaviours independently
Continued use (short-term) after end of project period
Intervention period ends. Participants practice WASH behaviours without study support. • Messaging and education still “fresh” in participants’ minds• Lack of reminders from promoter visits
Sustained adoption (long-term) after end of project period
Ranges from 6 months to years after project period ends. Captures true behaviour practice without influence of intervention promotion.• Habitual behaviour established• New families unfamiliar with WASH interventions may not practice
behaviour• Cost may be unsustainable
PRO
JECT
PER
IOD
POST
PRO
JECT
PER
IOD
WASH Technology Knowledge/skills of WASH behaviour
www.3ieimpact.orgHugh Waddington9
Methodology
• Extensive search of published and unpublished sources (updated in 2012-13)– PubMed, Embase, LILACs, Web of Science – JOLIS, IDEAS, British Library for Development Studies (BLDS),
Cochrane Library, scholar.google– Personal communication with leading researchers– Hand-search, back-referencing and citation tracking
• Inclusion criteria: – IEs measuring impact of intervention on diarrhoea morbidity
using experimental (RCTs) and quasi-experimental methods– reported specific water, sanitation, and/or hygiene
intervention(s); – were conducted in low- or middle-income countries; – use an infant or child as the unit of observation; and – estimate impact on diarrhoea morbidity, measured under non-
outbreak conditions.
www.3ieimpact.orgHugh Waddington
WASH systematic reviews search process74,181 records identified through database search
Scre
enin
gIn
clud
edEl
igib
ility
Iden
tifica
tion
1,024 records identified through other sources
49,472 records after duplicates removed
1,869 records screened 47,603 records irrelevant
225 full-text articles assessed
120 full-text articles excluded
137 studies included in quantitative
synthesis of effects
44 studies included in qualitative synthesis
‘Factual’ evaluation studies (eg qualitiative,
ethnographic etc)
‘Counterfactual’ impact evaluation studies
www.3ieimpact.orgHugh Waddington11
Diarrhoea impacts: evidence from 71 studies update ongoing)
NOTE: Weights are from random effects analysis
Water supply interventions
Subtotal
Water quality interventions
Subtotal
Sanitation interventions
Subtotal
Hygiene interventions
Subtotal
Multiple interventions
Subtotal
ID
Study
0.98 (0.89, 1.06)
0.58 (0.50, 0.67)
0.63 (0.43, 0.93)
0.69 (0.61, 0.77)
0.62 (0.46, 0.83)
ES (95% CI)
0.98 (0.89, 1.06)
0.58 (0.50, 0.67)
0.63 (0.43, 0.93)
0.69 (0.61, 0.77)
0.62 (0.46, 0.83)
ES (95% CI)
Ratio favours intervention 1.1 .5 .75 1 2
www.3ieimpact.orgHugh Waddington
Sustainability 1: less impact over longer periods
NOTE: Weights are from random effects analysis
Water supply (12 months or more)
Subtotal
Water quality (under 12 months)
Subtotal
Water quality (12 months or more)
Subtotal
Sanitation (12 months or more)
Subtotal
Hygiene (under 12 months)
Subtotal
Hygiene (12 months or more)
Subtotal
Multiple (under 12 months)
Subtotal
Multiple (12 months or more)
Subtotal
ID
Study
0.82 (0.71, 0.96)
0.56 (0.47, 0.66)
0.81 (0.67, 0.97)
0.64 (0.37, 1.10)
0.72 (0.60, 0.86)
0.67 (0.49, 0.91)
0.41 (0.23, 0.74)
0.77 (0.70, 0.85)
ES (95% CI)
0.82 (0.71, 0.96)
0.56 (0.47, 0.66)
0.81 (0.67, 0.97)
0.64 (0.37, 1.10)
0.72 (0.60, 0.86)
0.67 (0.49, 0.91)
0.41 (0.23, 0.74)
0.77 (0.70, 0.85)
ES (95% CI)
Ratio favours intervention 1.1 .5 .75 1 2
www.3ieimpact.orgHugh Waddington13
Sustainability 2: compliance falls over time
• Ceramic filter provision in Cambodia; 3 years later only 31% households were still using the filters (Brown et al, 2007)
• Pasteurisation in Kenya; 4 years later only 30% continued to pasteurise their water (Iijima et al, 2001)
• Programme promoting POU water disinfectant in Guatemala 1 year later; repeated use among only 5% of households from original trials (Luby et al, 2008).
• Water filters in Bolivia; compliance 67%; assessment made 4 months after trial ended (Clasen et al, 2006)
www.3ieimpact.orgAuthor name
Sustainability 3: lack of WTP (Kremer et al. 2012)
In Kenya, access to free chlorine increased uptake to over 60 percent, whereas coupons for even a 50 percent discount had a minimal effect
www.3ieimpact.orgHugh Waddington
Sustainability of hygiene impacts
NOTE: Weights are from random effects analysis
Water supply (12 months or more)
Subtotal
Water quality (under 12 months)
Subtotal
Water quality (12 months or more)
Subtotal
Sanitation (12 months or more)
Subtotal
Hygiene (under 12 months)
Subtotal
Hygiene (12 months or more)
Subtotal
Multiple (under 12 months)
Subtotal
Multiple (12 months or more)
Subtotal
ID
Study
0.82 (0.71, 0.96)
0.56 (0.47, 0.66)
0.81 (0.67, 0.97)
0.64 (0.37, 1.10)
0.72 (0.60, 0.86)
0.67 (0.49, 0.91)
0.41 (0.23, 0.74)
0.77 (0.70, 0.85)
ES (95% CI)
0.82 (0.71, 0.96)
0.56 (0.47, 0.66)
0.81 (0.67, 0.97)
0.64 (0.37, 1.10)
0.72 (0.60, 0.86)
0.67 (0.49, 0.91)
0.41 (0.23, 0.74)
0.77 (0.70, 0.85)
ES (95% CI)
Ratio favours intervention 1.1 .5 .75 1 2
Source: Waddington et al 2009 3ie Systematic Review
www.3ieimpact.orgHugh Waddington
Scalability of hygiene promotion
• Vietnam (Chase & Do, 2012):– Handwashing BCC (not soap) scale-up
through 1) mass media & 2) inter-personal communication – No health or productivity effects
• Peru (Galiani & Gertler):– Hygiene promotion through 1) mass media & 2) community
level (health facilities, schools)– No impacts health or nutrition (compliance measured)
• Bangladesh (Huda & Luby 2012):– SHEWA-B local community hygiene promoters– Low compliance rates & no impacts on diarrhoea or ARIs
www.3ieimpact.orgHugh Waddington
Supply doesn’t create its own demand
• Reduction in child disease rates not observed by carers or seen as substantial enough benefit to warrant costs (money/time) – what role for health education?
• Adoption of innovations (social change) is a slow process (early adopters vs. laggards) – what role for ‘triggering’ (CLTS)?
• Perceived benefits important (e.g. user satisfaction, time-use, safety) – role for appropriate technology/maintenance?
www.3ieimpact.orgHugh Waddington18
Thank you
• Systematic review evidence library:http://www.3ieimpact.org/evidence/systematic-reviews/
top related