moving from evaluation to learning peter york
DESCRIPTION
Moving from Evaluation to Learning by Peter YorkTRANSCRIPT
![Page 1: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/1.jpg)
Program Evaluation:
Moving from Evaluation to Learning
![Page 2: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/2.jpg)
2
Session Agenda
• Introductions
• The Need for Learning
• Why Nonprofits Aren’t Learning
• Evaluation vs. Evaluative Learning
• Nonprofit Evaluative Learning – Steps, Processes & Tools
![Page 3: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/3.jpg)
The Need for Learning
![Page 4: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/4.jpg)
4
Leadership = Success
Leadership is key to:
Growth
Sustainability
Lifecycle Advancement
Effectiveness
![Page 5: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/5.jpg)
5
There’s More to Leadership…
• Advancing effective communication of mission and vision to
internal and external stakeholders
• Engaging internal and external stakeholders in planning
• Taking decisive action when faced with challenges
• Making decisions anchored in cost-effectiveness
• Demanding accountability that includes demonstrated
success with those being served/targeted
Only one in four nonprofit organizations is well led…
![Page 6: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/6.jpg)
6
The Key to Leading Is Learning
Only one in four nonprofit organizations is an effective learner…
The following organizational learning behaviors are significantly and
singly the biggest predictor of organizational leadership:
• Creating sophisticated financial, operational, programmatic and
environmental data-gathering and learning processes
• Spending time leveraging program data for making meaning,
decisions, designs and plans, not accountability or validation
• Infusing learning into ALL planning, anchored in program success
• Taking immediate and decisive action, particularly at a human
resource level, as indicated by strategy research findings
![Page 7: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/7.jpg)
Why Aren’t Nonprofits Learning?
![Page 8: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/8.jpg)
8
The Investment Model Is All Wrong
• There’s no “outcome synchronicity” between the
investor and investee
• Effectiveness and accountability must be viewed
through the measurement of proximate effect
• Proximate cause-and-effect is the only way!
• Learning requires understanding the cause, not the
effect
• There’s no Research & Development for
programs/initiatives
![Page 9: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/9.jpg)
9
What Results Do Investors, Business Leaders, & Consumers Want?
Example: In-Home Support Services to New Parents, For-Profits vs. Nonprofits
Investors
(Funders)
Business
LeadersConsumers
For-Profit Results:Just Give Me (or Show Me) the Direct
Results, Please…
Nonprofit Accountability:Just Give Me the Direct Results, But Somehow Prove to
“Them” That We Can Do Much More…
Recuperation, parent-child bonding, healthy
adjustment to family change, tools for care and
feeding, stress-reducing routines and habits, better
communication skills with providers
• Same Direct Results as the For-Profit Business
• Child Development, School Readiness, Crime Reduction
• Parental Reduction in Child Abuse & Neglect, Maternal Health
Nonprofit
Business
Leaders &
Clients
Funders
(Investors)
![Page 10: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/10.jpg)
Why Evaluation Doesn’t Always
Lead to Learning
![Page 11: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/11.jpg)
11
Why Evaluation Isn’t Achieving Learning
1. Makes uncontrollable community impact indicators the
metric of accountability
2. Evaluates the “whole” program/strategic effort to determine
if it worked, rather than the combinations of program
components to determine how it works
3. Aspires to a scientific research design ideal that seeks
widespread generalizability, rather than contextual
applications
4. Gathers data from the wrong source
![Page 12: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/12.jpg)
Evaluation vs. Evaluative Learning
![Page 13: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/13.jpg)
13
The Difference Between Evaluation and Evaluative
Learning
Evaluation: the owner determines success by overall
wins, losses and trophies.
Evaluative Learning: the coach/manager figures out how
to improve the odds of success for the next game by
analyzing the play of the previous ones.
![Page 14: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/14.jpg)
14
ResourcesWhole
ProgramShort-Term Outcomes
Long-Term Outcomes
Community / Social Impact
EvaluationDetermining a “Whole” Program’s Long-Term Impact to Judge Social Value
![Page 15: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/15.jpg)
15
Program Resources
Program Elements
Achievable Outcomes/
Next Actions
Studies say…
Community / Social Impact
Evaluative LearningMeasuring Program Success for “Best Practice” Improvement
![Page 16: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/16.jpg)
16
Six Learning Practices that Facilitate Sustainable
Growth
Organizations are significantly more likely to sustain and grow if they
engage in the following evaluative learning behaviors:
NOT evaluating to decide if a whole program works; rather, figuring out
when and how it works.
Gathering data from recipients.
Determining outcomes by listening to recipient changes and next
actions.
Engaging people in making meaning from data.
Developing performance management processes, tools and systems.
Prioritizing cost-per-outcome.
![Page 17: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/17.jpg)
17
Evaluative Learning Facilitates Sustainable Growth
41% of nonprofits
grow faster than
the rate of inflation
over a three year
period
Nonprofits where
leaders engage in
these behaviors
are 2.5 times
more likely to
grow faster than
the rate of
inflation.
7%
5%
4%
2%
Conducting All R&D Behaviors (0% of all NPs)
Conducting Many R&D
Behaviors (5%)
Conducting Some R&D Behaviors
(45%)
Conducing Very Few R&D
Behaviors (25%)
Conducting NO R&D Behaviors
(25%)
Avera
ge A
nn
ual
Gro
wth
Rate
Average Annual Growth Rate (Mean Per Year, Based on Three Consecutive Years of Data)
![Page 18: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/18.jpg)
18
Guiding Steps for Conducting Evaluative Learning
1. Measure achievable outcomes, next actions, direct results.
2. Gather lots of data QUICKLY, by quantifying the qualitative.
3. Ask the participant/recipient.
4. Analyze data for patterns of cause-and-effect.
5. Gather a few key illustrative stories.
6. Spend time making meaning and making improvements.
![Page 19: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/19.jpg)
19
Step One: Identify Your Achievable Outcomes
• It’s all about the path to behavior change:
– Awareness +
– Knowledge +
– Attitude +
– Motivation +
– Skills +
– Opportunity +
– Behaviors =
___________________
Habits
=
![Page 20: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/20.jpg)
20
Evaluative Learning Processes to Try
1. Positive Deviant Experiment
2. Literature Review to Make the Case for Proximate Outcomes
3. Proximate Logic Model Creation
4. Survey Data Project
5. Qualitative Data Project
6. Creation of Ongoing R&D and Performance Management Tools
![Page 21: Moving from evaluation to learning peter york](https://reader033.vdocument.in/reader033/viewer/2022060123/5597d8e81a28abc45e8b45f9/html5/thumbnails/21.jpg)
Thanks!