using the outcome mapping concept of outcomes in evaluations to enrich theories of change and to...
TRANSCRIPT
![Page 1: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/1.jpg)
Using the Outcome Mapping concept of outcomes in evaluations
To enrich theories of change and to foster an evaluative culture
John MauremootooRichard Smith
Presentation given at the American Evaluation Association on 14th November 2015 in the session entitled: Applying Outcome Mapping to the evaluation of socially
transformative projects, programs, and policies
Photo: Public Domain
![Page 2: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/2.jpg)
Project locationsMongolia
World Bank & SDC Funded
DEFRA & DFID (UK) funded
India
Indian OceanEU funded
![Page 3: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/3.jpg)
Mongolia
Strengthen the capacity of Civil Society Organisations to undertake social accountability monitoring
![Page 4: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/4.jpg)
People and Pollinators initiative: Enhance pollination services for sustainable agriculture and biodiversity conservation
India
All photos of India from the Centre for Pollination Studies (CC-BY-NC)
![Page 5: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/5.jpg)
Manage coastal zones for environmental sustainability and livelihood generation
Indian Ocean
All Indian Ocean photos by John Mauremootoo (CC-BY)
![Page 6: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/6.jpg)
Theories of Change
Mongolia: Results chain from proposal
India: Logframe from proposal
Indian Ocean: Logframe from proposal
![Page 7: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/7.jpg)
The Use of Outcome Mapping Concepts
Development
Implementation
Closure
Mongolia
Indian Ocean
India
Summative evaluation 3 years after project
closure
Summative evaluation 3 years after project
closure
Introduced 8 months into implementation for
internal planning, monitoring & evaluation
All projects planned and implem
ented using donor-m
andated logframes
![Page 8: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/8.jpg)
Value of using OM Conceptsto enrich Theories of Change
Theory of change – the preconceived map of the journey
Theory in use – what actually happens
![Page 9: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/9.jpg)
Value of using OM Conceptsto enrich Theories of Change
Outcome mapping makes step two explicit by using concepts such as outcomes as behavioural change, boundary partners and contribution not attribution
![Page 10: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/10.jpg)
Outcomes as Behavioural Change
Mongolia
Activities
OutcomesAs indicators of effectiveness & sustainability
An effectiveness and sustainability evaluation framework
![Page 11: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/11.jpg)
Outputs
Outcome Challenge
BoundaryPartner
ProgressMarkers
Project Purpose
Cont
rol
Influ
ence
Inte
rest
Outcomes as Behavioural Change
India
Activities
Inputs
![Page 12: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/12.jpg)
Outcomes as Behavioural Change
![Page 13: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/13.jpg)
Boundary Partners
From Tableatny on Flickr
Individuals, groups or organisations with which the program interacts directly and which the program hopes to influence
![Page 14: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/14.jpg)
Boundary Partners
Mongolia
Limited scope of intervention meant the selection of participants / boundary partners by the implementing body was central to any achievements.
Evaluation data collection was directly from those who participated as trainees / mentees in the intervention.
![Page 15: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/15.jpg)
Boundary Partners
India
The Boundary Partner concept highlighted the importance of site-specific political and administrative structures e.g. very different intervention models in the two project sites (not explicitly accounted for in the project design)
![Page 16: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/16.jpg)
Boundary Partners
Indian Ocean Given the short duration of the projects (<18 months) most of the successful projects were those which built on existing partnerships.
![Page 17: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/17.jpg)
Contribution not Attribution
The intervention may be only one among many factors that contributes to an outcome
![Page 18: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/18.jpg)
Contribution not Attribution
Mongolia
The intervention introduced tools that were previously unused in Mongolia to CSOs with little or no experience of social accountability work therefore the contribution of the intervention could be easily identified.Several direct outcomes led to further outcomes and even impact. In such cases other factors and actors contributed.
![Page 19: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/19.jpg)
Contribution not Attribution
India
Appreciation of the value of partnerships to produce sustainable outcomesValuable for team spirit as everybody understood that they were contributing to results
![Page 20: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/20.jpg)
Contribution not Attribution.
Some projects failed to sustain outcomes nor contribute to impact because of failure to address actors and factors in the enabling environment (outside the control of the project) e.g. policy makers, legislation and the provision of infrastructure.
Indian Ocean
![Page 21: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/21.jpg)
Outcome Mapping Concepts Strengthen the Design or Management of Interventions
![Page 22: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/22.jpg)
OM Concepts Inform Further Support
Mongolia
Evaluation informed what to focus on for a follow up project and made a case for further support
Recommendations for discussion contributed to new project design
![Page 23: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/23.jpg)
India
OM Concepts Adopted
Planning, monitoring and evaluation is now seen as integral to project implementation by project leaders who are looking to incorporate OM concepts into new project development.
![Page 24: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/24.jpg)
Donor Interest Recommended
The project was rated A++ (outcome substantially exceeded) in its final external review and it was recommended that the PME manual (based on an OM-LFA fusion) should be publicised by defra as ‘best practice’
India
![Page 25: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/25.jpg)
Evaluation results inform programme development.
Indian Ocean
Incorporation of lessons learned from evaluation into guidance documents for follow up EU programme – emphasis on partnerships, participation and building upon existing actions.
![Page 26: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/26.jpg)
OM concepts helping us to become better Cartographers
OM helps us to gain a better understanding of the mental maps of those in the project’s ecosystem
Gervase R. Bushe ~ Professor of Leadership and Organization Development. A quote on leadership which could equally be applied to project implementation.
Leadership is a lot about influencing people and you cannot do that if you don't understand their maps.
![Page 27: Using the Outcome Mapping concept of outcomes in evaluations to enrich theories of change and to foster an evaluative culture](https://reader031.vdocument.in/reader031/viewer/2022011721/589f86971a28abf0638b65e7/html5/thumbnails/27.jpg)
Acknowledgements
Mongolia: Jeremy Gross (Co-evaluator) Amarbayasgalan Dorj (Co-evaluator)
India: Parthib Basu (Project Leader India & evaluation support)Barbara Smith (Project Leader UK & evaluation support)Soumik Chatterjee (Project Manager & evaluation support)
Indian Ocean: Dunstan Kishekya (Co-evaluator)