success and failure in the evaluation process
TRANSCRIPT
Success and Failurein the Evaluation Process
@KatAthanasiades
Annual Conference 2015, National Harbor, MDGrants Managers Network
Tuesday, March 17, 2015
About Innovation Network
White papers with tips for grantmakers
Kat
Evaluation consulting firm working with foundations and nonprofits
Research on evaluation practice
What does success
mean to you?To your employer?
What does failure
mean to you?To your employer?
Agenda
Evaluation and grant reports
1
2
3
Evaluation and failure
Embedding “failure reporting” in grant reports
Embedding “failure reporting” in grant reports
Evaluation and grant reports
1
2
3
Agenda
Evaluation and failure
Planning
Data collection
Analysis & Reflection
Action & Improvement
1
“failure reporting”
Risk tolerance
1
1
What’s going on over here?
1
1
1
How tolerant is your
comes to failure?
organization when it
Embedding “failure reporting” in grant reports
Evaluation and failure1
2
3
Agenda
Evaluation and grant reports
Planning
Data collection
Analysis & Reflection
Action & Improvement
2
Planning
Data collection
Analysis & Reflection
Action & Improvement
2
2
Planning
What does it take to achieve goals? To achieve success?
Which strategies are most effective in moving our organization closer to our long-term goal of creating meaningful experiences?
2
Why are some grantees making more progress than others?
Basic Grantee Information
Grantee Name
Grant Amount
Topic Area
Grant Terms
Accountability Assessment
Key Deliverables/Activities
Outputs
Barriers/Opportunities
Staff involvement
Grant report example 12
Impact/Success Assessment
Change Agent Targeted
Evidence of Influence
Recommended Next Steps
Additional Follow Up Recommended
No Follow Up
Grant report example 12
1. Describe your organization’s advocacy activities and how these activities relate to your organization’s advocacy objectives
2. Describe your accomplishments in the first year of GSA funding. What advocacy related results have occurred in connection with MFH funding?
Grant report example 22
3. How have the GSA grant funds increased your organization’s capacity to perform advocacy activities?
4. What have been your organization’s challenges (frustrations, barriers, or disappointments) in the past year? How were these challenges addressed or redirected?
Grant report example 22
Organization
Outcomes
Accomplishments
Activities
Successes
Changing context—external factors
Evidence of influence
Grant report example 3
Updates on:
2
Why is it so hard to use grant reports for evaluation?
2 The shoe (data) doesn’t fit
Planning
Data collection
Analysis & Reflection
Action & Improvement
2 This is where grant reports
are often useful
Planning
Data collection
Analysis & Reflection
Action & Improvement
2 We often lack the data for
the latter stages of
evaluation
2Trust
2
What pieces of your
reports can be useful
organization’s grant
to evaluators?
Evaluation and grant reports
Evaluation and failure1
2
3
Agenda
Embedding “failure reporting” in grant reports
3 Adjust the reporting structure
Adjust the fields
3 Encourage risk tolerance
Build trust with grantees
Build trust with board
Agenda
Evaluation and grant reports
1
2
3
Evaluation and failure
Embedding “failure reporting” in grant reports
What are your
grantmaking failuressuggestions for capturing
as well as successes?
Image creditsSlide Credit
5-6 Desk pictures c/o Kat Athanasiades
7 Children in cafeteria from USDAgov flickr, Creative Commons: http://www.flickr.com/photos/usdagov/7995804478/
12 Roulette from Scott Beale / Laughing Squid flickr, Creative Commons: laughingsquid.com or http://www.flickr.com/photos/laughingsquid/5249116537
13-14 Spotlight/light hole from abooth202 flickr, Creative Commons:http://www.flickr.com/photos/abooth202/2628358466
15 Mountain landscape c/o Kat Athanasiades
20-21 Road map from sidelong flickr, Creative Commons:http://www.flickr.com/photos/sidelong/234192777/
28 Child in shoes from rumpleteaser flickr, Creative Commons: http://www.flickr.com/photos/rumpleteaser/2940847467
31 Man and tiger from dobs flickr, Creative Commons: http://www.flickr.com/photos/dobs/10726756606
34 Cranes from 96dpi flickr, Creative Commons: http://www.flickr.com/photos/96dpi/3227807209
35 Kitten and hedgehog from elenitaspics flickr, Creative Commons: https://www.flickr.com/photos/elenitaspics/11254227916