an introduction to monitoring and evaluation in outcome mapping applied and adapted use bern, 15...
TRANSCRIPT
An introduction to Monitoring An introduction to Monitoring and Evaluation in Outcome and Evaluation in Outcome
MappingMapping
Applied and adapted useApplied and adapted use
Bern, 15 April 2010
ACTIVITIES
OUTPUTS
OUTCOMES
IMPACT
INPUTS
Applied Monitoring
Inspired by Jeff Conklin, cognexus.org
Time
ACTIVITIES
OUTPUTS
OUTCOMES
IMPACT
INPUTS
Applied Monitoring
Inspired by Jeff Conklin, cognexus.org
Time
ACTIVITIES
OUTPUTS
OUTCOMES
IMPACT
INPUTS
Applied Monitoring
Inspired by Jeff Conklin, cognexus.org
Time
Performance
Your programme’s functioning as an organizational unit
ACTIVITIES
OUTPUTS
OUTCOMES
IMPACT
INPUTS
Applied Monitoring
Inspired by Jeff Conklin, cognexus.org
Time
Strategy
The effectiveness of how you are attempting to influence boundary partners
ACTIVITIES
OUTPUTS
OUTCOMES
IMPACT
INPUTS
Applied Monitoring
Inspired by Jeff Conklin, cognexus.org
Time
Outcomes
Change you influence in the boundary
partners
Adapted Monitoring
For example…You have no baselineAre in complex situations where relations of cause and effect are unknownAre a non-hierarchal organisation
ACTIVITIES
OUTPUTS
OUTCOMES
IMPACT
INPUTS
Inspired by Jeff Conklin, cognexus.org
Time
Situations in which this…
Looks like this
OUTPUT
OUTCOME
INPUTS
ACTIVITY
INPUTSACTIVITY
INPUTS
ACTIVITY
INPUTS
OUTPUT
OUTPUT
ACTIVITY
OUTPUT
OUTPUT
OUTCOME
OUTCOME
OUTCOME
OUTCOME
OUTCOME
OUTCOME
You monitor by
Watching social actors for changes in their behaviour, relationships, actions, policies or practices
Focusing on the outcomes you are achieving
Working backwards from outcomes to determine how you contributed to that change, rather than attempt to attribute the change to your activities
And then deciding how to improve strategy and organisational performance in order to achieve outcomes
Evaluation is the least developed of the three stages of Outcome Mapping, formally at least.
Limited to designing the Evaluation Plan.
Elements of an Outcome Mapping Evaluation Plan
EXAMPLE EVALUATION PLAN
Evaluation Issue: Results Achieved by Recipient Research Institutions
Who Will Use the Evaluation?
How? When?Questions
Information Sources
Evaluation Methods
Who Will Conduct and Manage the Evaluation?
Date (Start
& Finish)
Cost
Executive director to use findings to inform program proposal
What influence has the institution had on research users and other researchers?
Progress Markers & Outcome JournalTrip reportsStrategy Journal
Key informant interviews with program staff
Senior Consultant with regional and sectoral expertise
Six months
35K
www.idrc.ca/evaluation
But as with monitoring, the outcome concept is wonderfully adaptable.
It can potentially serve the three principal evaluation modes.
Formative evaluation
18
Summative evaluation
19
Developmental evaluation
Examples of formal evaluationsTreatment response for
problematic use of ecstasy, ketamine and gamma-hydroxybutyrate in Australia
Six civil society projects in Bosnia and Herzegovina
Community development program with Roma and non-Roma communities in Bosnia and Herzegovina
OM applied as a planning, monitoring and evaluation tool to the Ceja Andina project in Carchi, Ecuador
A working group on 'values’
Examples of adaption of Outcome Mapping in evaluation
7 international networks – 2 based in Latin America, 2 in Europe, 2 in Asia, and I in the USA.
8 programmes of five grant makers – Oxfam Novib, IDRC, Ford Foundation, Hivos, and the Open Society Institute
Their 130+ social change and development grantees
Outcomes not predefined
Evaluation
■ Internal outcomes
■ External outcomes
■ Their significance
■ The organisation’s contribution
In these evaluations we “harvested” descriptions of who changed what, when and where, its significance and the contribution of the organisation.
In addition, we substantiate the outcomes with independent third parties who have a working knowledge of the change and how it came about.
We mapped processes of change to understand how outcomes reinforce or undermine each other.
Thank you! For more information, examples of use and to share your experiences
www.outcomemapping.cawww.idrc.ca/evaluation