es gfw evaluation sept 07
TRANSCRIPT
![Page 1: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/1.jpg)
by Ellen Sprenger for GFW staff, Sept 07
1
Measuring Success
What’s New, What’s Next?
![Page 2: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/2.jpg)
2
Success is…
Contributing to social justice…Contributing to transformation of power…
![Page 3: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/3.jpg)
Gaventa, Just Associates: Making Change Happen: Power (2006)
3
We are dealing with power…
• Visible: – Rules, structures and policies that serve certain
people over others, decision making processes where certain groups are excluded
• Hidden:– Who sets the agenda, is being heard?
• Invisible:– Shaping meaning, sense of self and what is normal
![Page 4: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/4.jpg)
4
Basic assumptions and dilemma’s around
measuring success
![Page 5: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/5.jpg)
5
The world is logical– “A group of smart people can figure it out”– Change is about finding the right technical
fixes
The world is political– “Transformation of power relations”– Change is about conflict, struggle, processes
Basic assumptions & dilemma’s (1)
![Page 6: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/6.jpg)
6
Measuring is about proving, providing evidence– Focus is on the past– Value free, ‘perfect information’– Emphasis on quantitative information– The measuring process is driven by outsiders
Measuring is about improving– Focus is on the future– Learning, building knowledge– Emphasis on qualitative information– The measuring process is driven by insiders
Basic assumptions & dilemma’s (2)
![Page 7: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/7.jpg)
7
Simple systems– Change is linear: cause and effect
Complex systems– Change is multi-dimensional, the result of:
• Many different actions and circumstances• Intentional and unintentional actions
Basic assumptions & dilemma’s (3)
![Page 8: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/8.jpg)
8
Basic assumptions & dilemma’s (4)
Attribution of success (claiming credit)
Contribution to success (sharing credit)
![Page 9: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/9.jpg)
9
What’s Next, What’s New?
![Page 10: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/10.jpg)
International Development Research Center (IDRC)
10
Outcome Mapping• Deals with the question: how to attribute impact
(and probability of impact)• Focuses on changes within the programs sphere
of influence (outcomes)• Defines outcomes as changes in relationships,
activities or behaviors of direct partners• Focuses on how programs facilitate change
rather than control or cause change• Focuses on learning and increased
effectiveness
![Page 11: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/11.jpg)
11
Outcome mapping (summary)
The world is logical The world is political
Measuring for the purpose of proving
Measuring for the purpose of improving
Simple system(cause and effect)
Complex systems(non linear, multi stakeholder)
Attribution of success Contribution to success
![Page 12: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/12.jpg)
Caren Grown, 2006, Quick Impact Initiatives for Gender Equality
12
Quick Impact?
Quick Impact Initiatives (QIIs) in the context of the MDGs. Some examples:– Strengthen opportunities for post-primary
education for girls– Scholarships for girls– Elimination of school user fees– Making schools girl-friendly by improving their
safety, design of facilities (such as latrines for girls) and promoting girls attendance (such as permitting married adolescents to attend)
![Page 13: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/13.jpg)
13
Quick Impact (summary)
The world is logical The world is political
Measuring for the purpose of proving
Measuring for the purpose of improving
Simple system(cause and effect)
Complex systems(non linear, multi stakeholder)
Attribution of success Contribution to success
![Page 14: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/14.jpg)
Women's Funding Network www.wfnet.org
14
“Making the Case”Developed by women’s funds (WFN, Mama Cash)
for women’s funds, their grantees (and NGOs in general)
Purpose is threefold, namely to:• Provide evidence of results• Communicate results• Strengthen organizational learning and planning
An online tool, aggregated data on the results of the work of women’s funds as a community
![Page 15: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/15.jpg)
15
“MtC”: Five dimensions of change
1. Shifts in definitions / reframing – The issue is viewed differently in the community or larger
society2. Individual and community behavior
– People are behaving differently in the community or larger society
3. Critical Mass and engagement– People are more engaged
4. Institutional and policy changes– An institutional policy or practice has been changed
5. Maintaining / holding the line– Earlier progress has been maintained in the face of opposition
![Page 16: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/16.jpg)
16
“MtC”: How it worksIdentify relevant dimensions of change, e.g. To
what extend is the issue viewed differently because of your work?
• Baseline• Goals• Strategies• Evidence (indicators, quantitative and qualitative)• External Accelerators• External Inhibitors• Internal Accelerators• Internal Inhibitors• Unexpected results
“A success story”
![Page 17: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/17.jpg)
17
MtC: How it supports the users• Provides a theory of change and framework for
measuring success
• Builds the story, the message and the evidence (for learning, for mobilizing resources)
• Enables aggregation and collective learning about:– What dimensions of change women’s groups are
working on (and not working on), key inhibitors and accelerators
– Collective outcomes (per country, region, globally)– Evidence based communications (for leveraging more
support for women’s rights work and women’s funds)
![Page 18: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/18.jpg)
18
“Making the Case” (summary)
The world is logical The world is political
Measuring for the purpose of proving
Measuring for the purpose of improving
Simple system(cause and effect)
Complex systems(non linear, multi stakeholder)
Attribution of success Contribution to success
![Page 19: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/19.jpg)
19
Action Aid’s Global Monitoring Framework (GMF): core principles
• A Rights Based Approach• Multiple Accountabilities, of which downward is most
important (as opposed to upward, to donors)• Commitment to Women’s Rights and Gender Equality
in all we do• Constant analysis of and action on power imbalances• Transparency and proactive sharing of information in
relevant forms
• Simplified Reporting in favour of critical engagement, mutual learning and downward accountability
![Page 20: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/20.jpg)
20
GMF’s approach to measuring
• Poor people must define agenda and own “indicators” of what CHANGE looks like
• Its not the tool that matters it is the attitude and behaviour of the “user”
• Rigid frameworks less important than process and relationships
• Who wants to know – and why does it matter? Upward or downward?
• Tools and methods include story telling, participatory review and reflection processes, written reports, external reviews and peer reviews
![Page 21: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/21.jpg)
21
GMF: four areas of change
1. Critical consciousness, capacity, action of rights holders
2. Tangible changes in people’s lives/material conditions
3. Organisation, action and movement building/growth
4. Policies and practices of states & other duty bearers
![Page 22: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/22.jpg)
22
GMF: questions to ask
• What did we do?• In pursuit of which right/s?• Resulting in what changes / dimensions of
change? For whom?• With what impact on power and power
relations?
![Page 23: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/23.jpg)
23
AA’s GMF (summary)
The world is logical The world is political
Measuring for the purpose of proving
Measuring for the purpose of improving
Simple system(cause and effect)
Complex systems(non linear, multi stakeholder)
Attribution of success Contribution to success
![Page 24: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/24.jpg)
24
And what is here to stay?
![Page 25: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/25.jpg)
SIDA, 2005: The use and abuse of the Logical Framework Approach
25
Logical Framework Approach (LFA)
The first generation: 1970s and 1980s– Need for value free social science– Developed by US military >>NASA>>USAID>>DAC– Focus on the matrix
The second generation: 1990s and 2000s– Quantitative and Qualitative– Assumptions, risk analysis, stakeholder analysis
included– For the ODA community “the best of a bad bunch of
options available”– Focus on the approach (alongside the matrix)
![Page 26: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/26.jpg)
26
Logical Framework Approach(the first generation)
The world is logical The world is political
Measuring for the purpose of proving
Measuring for the purpose of improving
Simple system(cause and effect)
Complex systems(non linear, multi stakeholder
Attribution of success Contribution to success
![Page 27: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/27.jpg)
27
Logical Framework Approach(the second generation)
The world is logical The world is political
Measuring for the purpose of proving
Measuring for the purpose of improving
Simple system(cause and effect)
Complex systems(non linear, multi stakeholder
Attribution of success Contribution to success
![Page 28: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/28.jpg)
28
Be in the drivers seat…
![Page 29: Es Gfw Evaluation Sept 07](https://reader036.vdocument.in/reader036/viewer/2022070523/577ccd0b1a28ab9e788b5a21/html5/thumbnails/29.jpg)
29
• Remember that no one is completely sure about what they are doing, it is a learning process for all
• Have a Theory of Change: establish your organizations & movements ‘truths’ about power, obstacles, change agents and strategies needed
• Choose appropriate models and methods for measuring success (look for a good fit, or make them fit)
• Invest in organizational capacities• Own your way of doing it, and push
back!