human development network – the world bank approaches to evaluating impact tris lumley 31 march...
TRANSCRIPT
Human Development Network – The World Bank
Approaches to evaluating impact
Tris Lumley
31 March 2010
2
About NPC : helping charities and funders to do good, better
• Our purpose is to help charities and funders achieve greater impact
• Part think tank, part consultancy
• Founded in 2001, ~30 people today
Research Consulting Communications• social issues• structural challenges • charity analysis
• advice for charities• advice for funders• support to intermediaries
• lead the debate on effectiveness• disseminate NPC’s knowledge and tools
3
0
20
40
60
80
100
120
140
160
180
200
198
6/19
87
198
7/19
88
198
8/19
89
198
9/19
90
199
0/19
91
199
1/19
92
199
2/19
93
199
3/19
94
199
4/19
95
199
5/19
96
199
6/19
97
199
7/19
98
199
8/19
99
199
9/20
00
200
0/20
01
200
1/20
02
200
2/20
03
200
3/20
04
200
4/20
05
20
05/2
006
200
620/
07
200
7/20
08
200
8/20
09
Individual giving
GDP
Individual giving has not kept pace with GDP
1986/1987=100
4
NPC designed to test hypotheses
• Effective donors need independent analysis to target their giving
• The availability of independent analysis will catalyse more and better giving
• But rhetoric far exceeds reality:1. NGOs lack capacity to measure/analyse
2. Donors not yet driven by effectiveness/impact
5
The problems:So why aren’t charities measuring?
CultureAttitudes
MoneyIncentives
SkillsCapacity
Tools
Complexity
The challenge of measurement
Incentives: current evaluation paradigm driven by foundations, focused on programs not organisations, rarely leads to action or increased public knowledge
6
The problems:The case for measuring impact
Externalallows charities to communicate their impact to peers, donors, and funders, thereby spreading best practice and attracting funding
Internal allows charities to make strategic and operational decisions based on what works
7
Solutions:How can we move forward?
• Align incentives for NGOs to develop their own impact measurement capacity• Standard organisational reporting
• Resource to train & recruit analysts
• Specialised outcome measurement tools
• Networks & support to develop analysts
• This will lead to better data to use externally, with donors, governments, partners
• This will lead internally to continuous improvement and competition on the basis of social impact
8
Outcome measurement tools:Measuring children’s well-being
• We have designed a rigorous and robust questionnaire for 11 to 16 year olds to provide information about seven domains of subjective well-being:
Self:
1) Self-esteem
2) Resilience
3) Emotional
well-being
Relationships:
4) Friends
5) Family
Environment:
6) Home community
7) School
• This will help charities measure and demonstrate their impact, improve their work, and attract funding
• We are developing a market-ready technology solution for NGOs, schools in UK, and exploring international growth
9
The problems:What do donors want to see?
300 million people in India are struggling to survive on less than
US$1 a day
300 million people in India are
struggling to survive on less than
US$1 a day
Donors are moved to give by people rather than facts
10
The problems:So why don’t donors follow impact?
CultureAttitudes
Real motivation
s
InfoAnalysis
Philanthropy advice
The donor challenge
Motivations for giving: include rational considerations (need, impact) but overwhelmingly driven by emotional/personal (choice/control, relationships, recognition, trust)
11
Solutions:How can we move forward?
• Incorporate impact into existing donor motivations• Harness relationships• Build trust on independent analysis/impact• Reshape sector – demand and supply side• Donor education
• Strategic philanthropy can be ‘open sourced’• Coordination role for major development funders
• This will drive greater incorporation of impact into less strategic segments of donor marketplace
12
Donor education:NPC’s charity analysis framework
13
The future:How can leading funders drive change?
• Real-time evaluation (vs program evaluation)• e.g. building outcomes management information• e.g. well-being tools• e.g. outcomes measurement networks & courses
• Collaborative evaluation (in real-time)• e.g. evaluating system change – Strive (Cincinnati)
• Accessible platforms for philanthropy• e.g. supporting donor advice market• e.g. open platforms for range of donor data
- From randomised control trials to video- Programs explicitly designed for open partnership
14
Tris LumleyHead of Strategy
Skype# tris.lumleyTwitter# trisml+44 (207) 785 6326
www.philanthropycapital.org
15
Strive’s roadmap to success