Date: 15 Nov 2010

What difference did it make?

Penelope Beynon

Evaluating an Open API

Who should benefit from an Open API?

How will they benefit? Ie what will be different if the Open API is successful?

How will you know if those benefits are realised?

How are we evaluating MK4D?

Monitoring outputs – what did we do?– webstats, finance, publications, reach figures– Event reports

Evaluating outcomes – what difference did it make?– Principles of outcome mapping– Interviews, Surveys, ‘tests’, data analysis

Improvement - what lessons did we learn?– Event reports– Learning labs

Principles of outcome mapping

OM was developed by International Development Research Centre (IDRC) in the late 1990s – an online user group is run by Overseas Development Institute (ODI) http://www.outcomemapping.ca/

OM is a methodology for planning, monitoring and evaluation. – OM focuses on outcomes rather than outputs– OM acknowledges the limit to your influence– OM is people-focussed – OM looks for contribution rather than attribution

In IDS we are exploring use of OM for understanding the research uptake process, and evaluating research uptake work

Intentional Design

1. What are you trying to achieve? (vision)2. How are you going to do it? (mission)3. Who do you need to influence? (boundary partners)4. How will other people behave if you have been successful?

(outcome challenge)5. What changes do you hope to see in other people along

the way? (progress markers)6. What will you do to achieve the mission? (strategy maps)7. How will you behave if you are working well?

(organisational practices)

Outcome and Performance Monitoring

8. How will you make monitoring manageable? (monitoring priorities)

9. How will you record changes in other people? (outcome journals)

10. How will you record changes in what you are doing? (strategy journal)

11. How will you record changes in your own behaviour? (performance journal)

Evaluation Planning

12. What will you explore in-depth? (evaluation plan)

(adapted from Earl et. al. 2001)

Three stages of outcome mapping

Project team

Outputs:Inputs, activities

Partners/actors

Outcomes: Changes in attitudes/awareness/behavior

Beneficiaries

Impact: Changes in economic, physical and social wellbeing

Sphere of control

Sphere of influence Sphere of interest

Context Context Context

Spheres of control, influence, interest

Evaluating an Open API

Who should benefit from an Open API?

How will they benefit? Ie what will be different if the Open API is successful?

How will you know if those benefits are realised?