monitoring, evaluation, and learning (mel) framework for the millennium water program, kenya
DESCRIPTION
Monitoring, evaluation, and learning (MEL) framework for the Millennium Water Program, Kenya. May 8, 2012 MWP-K Learning Event. Background. Collaboration between: Millennium Water Alliance MWP-Kenya partners (CARE, CRS, FH, Living Water, Lifewater, World Vision) - PowerPoint PPT PresentationTRANSCRIPT
May 8, 2012MWP-K Learning Event
Monitoring, evaluation, and learning (MEL) framework for the Millennium Water Program, Kenya
Background
Collaboration between: Millennium Water Alliance MWP-Kenya partners (CARE, CRS, FH, Living Water,
Lifewater, World Vision) Center for Global Safe Water, Emory University
Objectives: Improve effectiveness and increase sustainability
through more data-driven decision making Build foundation of learning and collaboration through
MEL to support future continuation of MWP-Kenya
Challenges Addressed by MEL Framework
Monitoring usually limited to project activities and outputs (e.g. # latrines built, # trainings completed)
Final evaluation is only time seized to assess achievement of outcomes (e.g. behavior change)
Want to use monitoring to track progress toward intended behavior change
outcomes to enable a change of course if needed
Challenges Addressed by MEL Framework
Minimal information sharing & reflection on monitoring data or other lessons throughout project life cycle.
Want to intentionally budget time and persons responsible for guiding data sharing and reflection to maximize
lessons learned to improve practice
Challenges Addressed by MEL Framework
Sustainability seldom evaluated after project life cycle due to close of project, budget limitations, etc.
Want to incorporate M&E indicators into tools used during project life cycle to
assess likelihood of future sustainability (are factors in place to improve longevity
of benefits after project is finished?)
Components of MEL for MWP Kenya
1. Key program indicators2. Evaluation: baseline and follow-up3. Monitoring: systematic, regular data
collection Includes process and tools to guide “Data for decision-
making”: Sharing, feedback, and action; Monitoring tools to assess progress towards outcomes
4. Focused topical learning Includes partners’ leadership in special learning
topics; Emory support for some sub-studies (e.g. sanitation
assessment)
1. Key Program Indicators
Emory assisted partners in developing logical framework in May 2010
Indicators include: Outputs (activities completed by partners) Outcomes = central focus (changes that beneficiaries
adopt in response to program) Includes indicators that suggest future sustainability
E.g. School has specific WASH budget; school management committee is involved in WASH program
2. Evaluation
Emory facilitated baseline assessment planning and implementation: Led development of evaluation tools and protocols
Household, school, and health facility surveys Analyzed data
Each organization collected data prior to implementation
3. Monitoring
Output-level indicators Responsibility of each partner Tracked through internal monitoring system
Outcome-level indicators Measured throughout course of project
To engage implementers to critically reflect on and address programmatic challenges in real-time
Monitoring activities conducted quarterly or once per school term
Data for Decision-Making
Sharing of experiences and lessons learned between partners
Data Reflection Tool (DaRT) Summarizes findings from each monitoring and
learning activity Project expectations Progress toward achieving indicators Plans to address lack of progress Potential threats to sustainability
Shared and discussed within partnership on a quarterly basis before Program Management Group meeting
Data for Decision-Making Quarterly Work Flow
Complete monitoring activities quarterly
Summarize findings into
Data Reflection
Tool (DART)
MWP Coordinator reviews DARTs
Meeting:Internal
Learning/
Reflection
Meeting:Pre-PMG Learning/ Reflectio
n
Meeting:
PMG
1. Flag issues for follow-up
2. Use to guide agenda for Pre-PMG meeting
3. Distribute copies at pre-PMG meeting
Discussion of:
1.What’s going well/ not as well as expected2.Possible course changes3.Additional info needs
Discussion of:1.Highlights from DARTs2.Lessons learned, best practices3.Group feedback4.Issues to raise at PMG
Discussion of:
1.Issues flagged at Pre-PMG meeting2.Lessons learned & best practices
Send
Decide:
Needed course
changes
Additional learning
needs
= Responsibility of each organization= Responsibility of Coordinator= Responsibility of all partners together
Summary of Monitoring Tools
Tool Objective
Rapid household observationsObserve and inquire about WASH practices in households in the intervention area to track changes toward achievement of intended outcomes and understand the trends in behavior change.
Water point observations and management committee key informants interview
Observe repair or maintenance needs; observe practice of integrated water resources management (IWRM); understand patterns of use for the water point, financial management practices, maintenance practices, and management challenges.
Water point user interviewsDetermine functionality, use patterns, and perceptions of water point and management committee from community perspective. Provides a double check on some information gathered in the management committee interview.
School monitoring visit Determine whether and how the school enacts the intervention on a day-to-day basis and assess threats to sustainability that might need to be addressed.
School management committee key informant interview
Understand the degree to which the SMC is involved in supporting school WASH activities, and encourage their support by inviting them to join you in your monitoring visit observations.
Pupil interviews Gain pupil perspective on WASH provision and education at school and preferences. Provides a double check on some information gathered in the school assessment.
4. Focused Topical Learning
Proposed Learning Leader
Topic
CARE Improving pupil handwashing behavior change in schools
CRS Improving utilization of household latrines
FH Improving community handwashing behavior in ASAL, nomadic areas
LWI Improving financial management of water sources
WVI Improving water supply sustainability
CARE Improving pupil handwashing behavior change in schools
Partners expressed interest in specific topics for further investigation
As learning leader, each org. should organize one meeting with partners to discuss lessons on topic
Summary
Thoughtful, timely collection of relevant dataData used to drive decision-making:
Guide targeting, technology choice, and program approaches to improve effectiveness and sustainability
Feed back findings into continued program activities in a timely and meaningful way
Engage donors, beneficiaries, and government stakeholders in the water, sanitation, and hygiene (WASH) sector
Embrace benefits of working in partnership Collective learning and reflection on best practices &
lessons learned