outcomemapping.ca

Introduction to Outcome Mapping

Ziad Moussa, LebanonJan Van Ongevalle, Belgium

Heidi Schaefer, Canada

22 September 2014 – Dar Es Salaam – OM Lab

Workshop Objectives

Clarify what Outcome Mapping is and doesIntroduce and apply Outcome Mapping

concepts and toolsConsider if and how Outcome Mapping could

be useful in our work

Agenda Overview

Key concepts in Outcome Mapping

Acknowledgements

This presentation makes use of various materials that were shared by members of the global OM community. Without being exhaustive, special thanks goes to Terry Smutylo, Steff Deprez, Jan Van Ongevalle, Robert Chipimbi, Daniel Roduner, Kaia Ambrose and many others.

Source: A guide for project M&E: IFAD

TimeSource: Ricardo Wilson-Grau (inspired by Jeff Conklin)

Conventional thinking…

ACTIVITIES

OUTPUTS

OUTCOMES

IMPACT

INPUTS

Time

… clashes with relationships of cause and effect that are unknown

Source: Ricardo Wilson-Grau (inspired by Jeff Conklin)

ACTIVITIES

OUTPUTS

OUTCOMES

IMPACT

INPUTS

Social change can be…

• Complex: involve a confluence of actors and factors

• Unstable: independent of project duration

• Non-linear: unexpected, emergent, discontinuous

• Two-way: intervention may change

• Beyond control: but subject to influence

• Incremental, cumulative: watersheds & tipping points

Source: Terry Smutylo

OUTPUT

OUTCOME

ACTIVITY

INPUTSACTIVITY

INPUTS

ACTIVITY

INPUTS

OUTPUT

OUTPUT

ACTIVITY

OUTPUT

OUTPUT

OUTCOME

OUTCOME

OUTCOME

OUTCOME

OUTCOME

OUTCOME

Source: Ricardo Wilson-GrauTime

INPUTS

Evaluation challenges1. Establishing cause & effect in open systems

2. Measuring what did not happen

3. Reporting on emerging objectives

4. Justify continuing “successful” interventions

5. Timing – when to evaluate

6. Encouraging iterative learning among partners

7. Clarifying values

8. Working in ‘insecure’ situations

Source: Terry Smutylo

Brief definition of OM

• A participatory method for planning, monitoring and evaluation

• Focused on changes in behaviour of those with whom the project or program works

• Oriented towards social & organizational learning

OM’s answer

Source: Terry Smutylo / OM Lab 2012

Support people to build their own well-being

Enable interventions to adapt as they engage

Apply a systems understanding

Start from observable behaviour change

Recognise that all interventions have limited influence

Embrace different perspectives

“Being attentive along he journey is as important as, and critical to, ariving at the destination ”

Michael Quinn Patton

Three key concepts in OM:

1. Sphere of influence

2. Boundary Partners

3. Outcomes understood as changes in behaviour

There is a limit to our influence

ProjectPeople the project works with/through

People who benefit from the project

Sphere of control Sphere of concern

There is a limit to our influence

Inputs, activities, outputs

Outcomes: Changes in behavior

Impact: Changes in state

Sphere of control Sphere of concern

Participatory research on demonstration farms to develop approaches to drip irrigation

Farmers participate in field trials

Participating farmers learn how to use drip irrigation equipment

Extension workers visit demonstration farms

Training of extension workers

Publication of performance of different set-ups

Increased knowledge of techniques

Extension workers promoting drip irrigation

Farmers adopting drip irrigation methods

Reduced numbers of new wells

Greater quantities of groundwater available

Source: Terry Smutylo

Who are your boundary partners?

Programme

Beneficiaries Stakeholders Boundary Partners

Focus of Outcome Mapping

Outcome Mapping

Community ownership

increases

Program influence decreases

Inputs Activities Outputs Outcomes Impacts

✓OM is a guide to the journey we take with our partners. We co-create the map.

✓It focuses on the intention and what happens along the way

✓The map is not the territory, it shows the route taken

✓“The only real voyage of discovery exists, not in seeing new landscapes, but in having new eyes” (Marcel Proust)

Where is the map?

Step 1: Vision

improved human, social, & environmental wellbeing

I have a dream!

Martin Luther King, Jr.August 28, 1963

A vision statement..• Guides• Motivates and inspires• Is an ‘accountability-free zone’

37

Vision in graphic form, Nagaland (India)

Vision statement in narrative form

Local authorities, communities, and international organizations in developing

countries in Africa recognize the value of HIV/AIDS intervention as an integral part of

social & economic development. Municipal, regional, and national governments

actively support HIV/AIDS prevention activities by formulating and implementing

effective public health policies. Using research findings, they have developed a

comprehensive public health strategy to slow down the infection rate. Formerly

marginalized groups (e.g. women and youth) are organized into advocacy groups that

can effectively formulate their needs to policy makers. All groups have access to

reliable and relevant technical information about HIV/AIDS prevention and are able to

make informed choices. In essence, there are healthier, happier, and wealthier

communities.

Vision facilitation questionImagine that, 5-10 years from now, the program has been extremely successful. Things have improved beyond your most ambitious dreams.

• What changes have occurred? • What (& how) are your intended beneficiaries doing?• What are your partners doing? • Describe the better world you are seeking.

Step 2: Mission

The mission is that “bite” of the vision statement on which the program is going to focus.

• How the program intends to apply its resources in support of the vision

• The areas in which the program intends to work

• How the program will support the achievement of outcomes by its direct partners

A mission statement describes:

Example Mission Statement

In support of this vision and on behalf of its donors, the program will work in the areas of

research, dissemination, capacity building, & coordination. It will contribute to the production,

synthesis, & dissemination of research data, position papers, & other information that will

sensitize local & international actors to HIV/AIDS prevention. The program will seek to expand the

range of disciplines involved in HIV/AIDS research. It will enhance HIV/AIDS research capacity in

order to produce credible information for local, national, & international policy-making &

program development. It will promote an interest in HIV/AIDS research among new researchers

by providing research fellowships, mentorship, & training opportunities. The program will

contribute to the development of linkages between Northern & Southern researchers &

encourage partnerships between research organizations, advocates, & decision makers. It will

increase its visibility & credibility among the donor community & will convince them of the utility

of supporting HIV/AIDS prevention.

Summary

Vision✓ About the future

✓ Observable

✓ Idealistic

✓ Not about the program

Mission✓ Feasible

✓ Identifies activities and relationships

✓ About the program

Exercise : Correcting Vision and Mission Statements

1. Across rural India, women and girls are contributing to and sharing equitably in the benefits of development.

2. They utilize and benefit from appropriate health care, education, food and water security.

3. They are free from violence in the home and in the community.

4. The project seeks to improve their well being through a wide range of interventions.

5. Women are able to earn their own livelihoods, accessing freely the markets, credit, banking and municipal services they need to pursue their economic goals.

6. Villages are fully served by public transport, are well lit at night and have police forces that enforce all laws fully and equitably.

7. The consortium of donors responds favorably to the project’s progress reports and decides to continue funding for a second phase

1. The Swayamsiddha Project works with government, non-government and community-based organizations to strengthen their individual and combined efforts towards improving women’s health and empowerment.

2. The development of women’s self help groups is facilitated by the project, providing women with moral, material and educational support in identifying and taking action to make community and government services more responsive to their health and livelihood needs.

3. Primary and reproductive health care at the local level is directly accountable for, and responsive to, the needs of women and girls.

4. Enrolment of girls in primary schools increases by 35% as a result of project interventions

5. Research is undertaken in the application of ecosystem approaches to human health in agriculture and in the provision of health and sanitation services.

Step 3: Boundary Partners

Step 3: Boundary partners

Outcome mapping focuses on outcomes as changes in behaviour or relationships of those with whom the program or project is working directly (= boundary partners)

Boundary Partners are...

Those individuals, groups, & organizations with whom a program:•interacts directly to encourage change as a contribution to the vision.•can anticipate some opportunities for influence•engages in mutual learning

Who are your boundary partners?

Programme

Beneficiaries Stakeholders Boundary PartnersStrategic partners

CIDA

IDRC

BAIF

StateNGO

StateNGO

StateNGO

StateNGO

StateNGO

StateNGO

SHG Police CommunityLeaders

Families Banks PHCs

Swayamsiddha

Boundary partners have their own boundary partners - Often many actors and results at different levels

Facilitation questions• In which individuals, groups, or organizations is

your program trying to encourage change as a contribution to the vision?

• With whom will you work directly? • Are you choosing boundary partners because

you want to influence the ways they help or influence others?

• On whose actions does your success depend?

Strategic partners• Selected for their potential to contribute to the

mission

• A person or group with whom the program works directly to achieve the mission, without necessarily wanting to change the partner’s behaviour as part of the mission

• E.g. Donor agency, contracted service providers, NGOs doing similar work, media agencies

Facilitation questions• In which individuals, groups, or organizations is

your program trying to encourage change as a contribution to the vision?

• With whom will you work directly? • Are you choosing boundary partners because

you want to influence the ways they help or influence others?

• On whose actions does your success depend?

Screening of boundary partnersA program has normally not more than 4 or 5 types of boundary partners

•What if too many BP’s? What if you are not sure whether it is really a BP?

– Some boundary partners might be boundary partners of boundary partners!

– Discuss how the boundary partners are possibly contributing to the vision

– Can we realistically influence the boundary partners?– Where will we put most efforts and resources?– Sometimes similar actors can be clustered as one type of boundary

partners– Who are your ultimate beneficiaries / target group? There are not your

boundary partners– Maybe some partners are actually strategic partners?

Exercise : Boundary partners

Boundary partners: exercise

• Read the handout with the case of the VVOB teacher education and vulnerability programme

• On the basis of the case of the VVOB programme (see handout), try to position the various actors in (or out) of the various influence spheres on the flip chart provided.

• Solution: see handout!

20-04-23 59

Developing an actor focused TOC (case of VVOB Zimbabwe)

Inputs, activities, outputs

Sphere of control Sphere of interest

Final beneficiaries

PupilsPupils

College adminsCollege admins

Student peer educator clubsStudent peer

educator clubs

Student support structures

Student support structures

Boundary partners = Direct target groupsBoundary partners = Direct target groups

Staff devpt committeesStaff devpt committees

Earch Childhood depts

Earch Childhood depts

VVOB support

team

VVOB support

team

College studentsCollege

students

College LecturersCollege

Lecturers

Indirect target groupsIndirect target groups

Outcomes Impact

UnicefUnicef

DGDDGD

PLAN International

PLAN International

Local resource persons

Local resource persons

Step 4: Outcome Challenge

Outcome Challenge• Describes behaviour of a single boundary

partner

• Sets out the ideal actions, relationships activities

• Describes the boundary partner’s contribution to the vision

Facilitation questions• Ideally, in order to contribute to the vision,

what would the boundary partner be doing?

• With whom would they be interacting?

• How could this boundary partner contribute maximally to the vision?

Women’s self help groups are taking action to make community and

government services more responsive to the health and livelihood needs of

women and girls.

They influence banks, police, health and social service providers, local

officials and state and national government agencies in relationships of

mutual respect and joint action to improve women’s well being.

Women’s self help groups arrange bank loans for members and for life skills

training for girls to be included in the school curriculum.

They influence local, state and national government policies and

expenditures on community improvement and transportation and support

women candidates to run for election to local government office.

Step 5: Progress Markers

(Deep transformation)

(Active engagement)

(Early positive responses)

Love to see

Like to see

Expect to see

Progress Markers

Love to see

Like to see

Like to see

Expect to see

Like to see

Like to see

Expect to see

Like to see

Like to see

Expect to see

Why Graduated Progress Markers?

• Articulate the complexity of the change process

• Allow negotiation of expectations between the program and its partners

• Permit early assessment of progress

• Encourage the program to seek the most profound transformation possible

• Help identify mid-course improvements

?

Greater awareness…Empowered women…Community ownership…Reduced conflict…Increased collaboration…Governmental commitment…Gender sensitivity…Equal access…Budgetary transparency…Active participation… Poverty alleviation…Strengthened capacity…

How can we measure:

Examples• Project expects to see Tanzania Forest Service

– Consult with communities on the new forest policy and the national REDD strategy;

– Develop management plans for all Nature Reserves;– Continue a policy that is supportive of participatory

forest management.

Examples• Project would like to see Tanzania Forest Service

– respond promptly to information from communities about illegal activities and other forest management issues and take action to resolve the issue.

– incorporate recommendations from communities and civil society organistions in the revised forest policy and the national REDD strategy;

– approve voluntary social and environmental timber standards;– implement and monitor management plans for nature reserves

and other high biodiversity forest reserves with the participation of communities;

– approve independent forest monitoring and cooperate fully in its implementation;

– incorporated Biodiversity indicators in NAFORMA

Examples• Project would love to see Tanzania Forest Service

– successfully advocate for wider application of good timber standards throughout Tz gov’t systems.

– consistently and effectively apply the Forest Act and regulations within the timber and charcoal trade thereby increasing the revenue that they capture and reducing illegal timber harvesting and charcoal production;

– incorporated social and environmental standards in revised harvesting regulations and other documents related to the national forest programme;

– Applying good social and environmental practices in respect of reserve management

Facilitation questions• How can the programme know the boundary

partner is moving toward the outcome?

• What would they be doing?

• What milestones would be reached as the boundary partner moves towards their intended role in contributing to the vision?

Progress Marker ChecklistEach Progress Marker:Describes a changed behaviour by the boundary

partnerCan be monitored & observed

As a set, Progress Markers:Are graduated from preliminary to more profound

changes in behaviourDescribe the change process of a single boundary

partner

Exercise : Arranging Progress Markers

Step 6: Strategy Maps

Strategy Maps

For each Boundary partner:• What will the Project do to SUPPORT desired changes in the

Partner towards the Outcome Challenge?

• Diverse range (an array) of ACTIVITIES:

- Aimed at Partner

- Aimed at Partner’s environment

- Cause, Persuade, Support

Strategy Map

causalpersuasiv

esupportiv

e

Partner

Environment

6 types of support strategies

causal persuasive supportive

Iaimed at individual boundary

partner

directInfluence

(e.g. funding, prepare a report, …)

arouse new thinking;

build skills, capacity

(e.g. skill enhancement,

methodological workshops, training)

on-going support

supporter/mentor who guides change over

time,involvement is more frequent and sustained,

nurturing for self-sufficiency

(e.g. program member who provides regular guidance and input, coaching, expert, …)

Eaimed at boundary partner’s

environment

alter the physical or regulatory

environment(e.g. technical transfer, policy change, internet access, terms of reference, rules,

guidelines)

broad information

dissemination/access to new

info(e.g. Radio, TV,

website, publications, conferences, workshops)

create / strengthen

peer networkspartners working

together and collectively

supporting each other)

(e.g. research network, …)

Example strategy map: for extension agents

80

Causal Persuasive Supportive

Directed to the boundary partner

Provide research and training funds

Run technical training workshops for extension agencies and activities (e.g. ISFM)

Coordinate training on farmer participatory action research approaches to extension agencies, service providers and research teams

Conduct training on protocols development and project monitoring

Contract a farmer-training specialist to work with extension on farmer mobilization, action research, data collection and documentation

Conduct quarterly monitoring and review workshops

Directed to the environment of the boundary partner

• Building model storage facility for grain and backstopping

Develop extension materials and policy brief on adaptation to climate change

Facilitate establishment of exchange forums for sharing of information, knowledge and experience among partners

Establish a network of extension agents and researchers that facilitate knowledge and information sharing

Engage researchers and policy makers as conveners for discussions on climate change impact and alternative measures

Exercise : Strategy Map

Exercise on strategy mapsInstructions

•Your table will be given a number of cards with activities written on them. Discuss which type of strategy each of the activities represents. The Outcome Challenge has been provided so that you are able to think about what the program is contributing to.

•Select someone from your table to plot the activities in the appropriate section of the strategy map on the wall. Be prepared to explain your reasoning for plotting the different strategies as I-1, I-2, I-3, E-1, E-2, or E-3.

•!Remember, a strategy map describes what the program will do to try to encourage change(s) in a boundary partner. It breaks down depending on whether they are aimed at the individual boundary partner or the environment and whether they are causal, persuasive, or supportive. 

Time: 30 minutes

mission vision

Girls & Women

Community Leaders

Women’s Self Help Groups

Families

PoliceState NGOs

Banks

Public Health Clinics

Strategic Partners

Strategies Project’s OutcomesBoundary

PartnersBP’s outcomes

BAIF

Step 7: Organisational Practices

Organizational Practices

How does your team or organization stay relevant, viable and effective?

8 practices1. Prospecting for new ideas, opportunities, and

resources

2. Seeking feedback from key informants

3. Obtaining the support of your next highest power

4. Assessing and (re)designing products, services, systems, and procedures

8 practices5. Checking up on those already served to add

value

6. Sharing your best wisdom with the world

7. Experimenting to remain innovative

8. Engaging in organizational reflection

M&E planning and journals

5 kinds of M&E Information

Program Partner

outcomes(behaviour changes in the partners)

implementation(interventions by the program)

relevance & viability(actions of the program)

C o n t e x t u a l I n f o r m a t I o n

State, status or situational data

Strategies

M&E Planning Worksheet

What information will be collected?

How will it be collected, from what sources and when?

Who will collect it?

Who will make sense of the info & how?

Who will manage the collection-to- usage processes?

What resources are needed & who contributes?

Outcome journal• What, when, who, how much• Contributing factors• Sources of evidence• Unanticipated change• Lessons, required changes, action

Strategy journal• What did you do, with whom, when• (How) did it influence change• Outputs• Suggested changes/follow up• Lessons

Performance journal• Progress against internal commitments – e.g.

organisational practices• Reviewing the logic of the programme

Journal for Monitoring OutcomesOutcome Monitoring Journal

Period of monitoring and data collection: Contributors to Monitoring Update: Name of the person(s) who compiled the journal: Outcome Challenge:

Based on your analysis of each progress marker, please indicate the number on the scale that best represents the boundary partner's progress in achieving the progress marker (1 indicates no activity towards the progress marker while 7 represents full achievement of the progress marker) Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted Progress markers What happened (who, how,

what, where) Date (when)

Strategy Used and its Efectiveness

Follow up/corrective measures Evidence (documents etc.)

1)

Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted

2)

Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted

3)

Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted

4)

Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted

5)

Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted

Name of the boundary partner

Work dating from/to:

Name(s) of the person(s) who compiled the journal:

Outcome Challenge:.

Progress Markers Remarkable facts, what happened

Follow up / corrective measures

Narrative of a success story:

Unanticipated changes :

Which support strategies from the ICCO alliance where helpful or require further follow-up or action during the next term?

Contributing or limiting factors and actors towards achievement of progress markers: :

Summary of lessons learned/recommendations :

Outcome Journal

Thank you!For more info on OM:

•Visit the OM learning community:www.outcomemapping.ca

•Contact:Simon Hearns.hearn@odi.org.uk

Sharing information to improve evaluation

outcomemapping.ca