can you prove you are making a difference? february 2015
TRANSCRIPT
![Page 1: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/1.jpg)
Is Your Nonprofit’s Work Actually Making a Difference?
DCPNI – Isaac Castillo - @Isaac_outcomes 1
Isaac D. CastilloDeputy Director
DC Promise Neighborhood Initiative@Isaac_outcomes
February 18, 2015
![Page 2: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/2.jpg)
Why Should You Care About Outcomes?
February 2015 Isaac Castillo - @isaac_outcomes 2
• LAYC domestic violence story– LeapOfReason.org– First Do No Harm…Then Do More Good
• New domestic violence program component designed to teach three things: – Partner violence is not an OK expression of
love– Partner violence is not OK in Latino culture– There are safe ways to get out of violent
relationships
![Page 3: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/3.jpg)
Learning Objectives
• Ability to define success for your organization and its programs/interventions
• Identify ways to measure progress and outcomes (using multiple approaches)
• Understand how to begin creating an organizational culture that embraces information and uses data to make programmatic improvements
February 2015 Isaac Castillo - @isaac_outcomes 3
![Page 4: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/4.jpg)
Who Counts?
• May seem like a simple thing, but even defining who ‘counts’ as one of your participants is a very important decision.
February 2015 Isaac Castillo - @isaac_outcomes 4
![Page 5: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/5.jpg)
Duplicated vs. Unduplicated Counts
• An unduplicated count is one that counts human beings – A person / human being would only be counted ONCE,
regardless of how many activities or sessions they have attended.
– Example: consider this class – what is the unduplicated count of this class?
• A duplicated count is one that counts ‘service slots’ – Not a count of human beings – rather it is a count of how
many service slots are occupied. – Example – if we all came back tomorrow – what would the
duplicated count be? Would the unduplicated count change?
February 2015 Isaac Castillo - @isaac_outcomes 5
![Page 6: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/6.jpg)
Talking and thinking about your counts
• What do you present publically? A duplicated count or an unduplicated count?
• Do you ‘count’ someone who only comes to a single event at your location?
• Do you collect different things for duplicated or unduplicated populations?
February 2015 Isaac Castillo - @isaac_outcomes 6
![Page 7: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/7.jpg)
Same Concept: Different Words
What do you call this sandwich?
February 2015 Isaac Castillo - @isaac_outcomes 7
![Page 8: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/8.jpg)
Outputs vs Outcomes
• Output measures assess what you do and who you serve. Examples include:
• Served 100 youth during summer camp
• Provided 2,250 hours of tutoring during the academic year
• 9 out of 10 youth attended at least 75 % of available art instruction classes offered
Outcome measures assess changes in your target population. Examples include:
• 75 % of youth increased their knowledge of local history during the summer camp
• 50% of youth increased math grades by one grade level during the academic year
• 25% fewer youth reported being involved in bullying over the last year
February 2015 Isaac Castillo - @isaac_outcomes 8
![Page 9: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/9.jpg)
Outputs
• Outputs DO: – Tell you about whether your program was
implemented well. For example, they indicate whether a program:• delivered the intended number of sessions• reached its intended population• resulted in adequate participation levels
• Outputs DO NOT: – Tell you if participants benefited from your program– Serve as indicators of program success or
effectiveness
February 2015 Isaac Castillo - @isaac_outcomes 9
![Page 10: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/10.jpg)
Outcomes
• Outcomes DO:
– Tell you if participants benefited from your program
– Serve as indicators of program success or effectiveness
• Outcomes DO NOT:
– Tell you about whether your program was implemented well (or provide clues about how your program improved participant outcomes)
February 2015 Isaac Castillo - @isaac_outcomes 10
![Page 11: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/11.jpg)
Do You Need To Measure Outcomes for Every Program?
• No.
• You should measure outcomes for your activities that provide a consistent set of programming or services for participants.
• Measuring just outputs is OK for some activities or programmatic approaches.
• Activities or services that are only used once by people likely will NOT produce outcomes.
February 2015 Isaac Castillo - @isaac_outcomes 11
![Page 12: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/12.jpg)
Need to Collect Both Outputs and Outcomes
• Determine if more services lead to more/better outcomes
• Determine if there is a ‘minimum’ amount of service needed to improve outcomes
• Determine if too many services actually lead to decreased or negative outcomes
February 2015 Isaac Castillo - @isaac_outcomes 12
![Page 13: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/13.jpg)
Should You Measure Outcomes For These Activities?
• An organization provides a health fair for community members– No – this is a service that a person would access once. Unlikely
this will lead to any lasting change. – But you should still track outputs – how many attended?
• An organization provides a series of classes lasting 10 weeks that teaches how to lower cholesterol. – Yes – should measure changes in knowledge, attitudes, and
possibly behavior. – Outputs should also be tracked (attendance)
• You could potentially also track: – How many health fair attendees ended up enrolling in and
completing the classes.
February 2015 Isaac Castillo - @isaac_outcomes 13
![Page 14: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/14.jpg)
Malleability: Can the Outcome Be Changed?
February 2015 Isaac Castillo - @isaac_outcomes 14
![Page 15: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/15.jpg)
Reality Check
• Is your program likely to have a measurable effect on an outcome?
– Example: A delinquency prevention program with 100 urban youth is unlikely to reduce a city’s crime rate
February 2015 Isaac Castillo - @isaac_outcomes 15
![Page 16: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/16.jpg)
Confirmatory Outcomes
• Confirmatory outcomes: outcomes you expect will change as a result of program participation– You will hold yourself responsible for confirmatory
outcomes for all (or the majority) of your regular participants.
– You will devote adequate resources and time to achieving confirmatory outcomes .
– You will measure your confirmatory outcomes
February 2015 Isaac Castillo - @isaac_outcomes 16
![Page 17: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/17.jpg)
Exploratory Outcomes
• Exploratory outcomes : outcomes potentially, but not definitely, improved by your program– You think (have a theory) that you may have an effect
on exploratory outcomes.
– But you will not hold yourself responsible for exploratory outcomes.
– You may or may not measure exploratory outcomes.
– If you do not measure your exploratory outcomes, you should have a theory (or prior research) that supports why you think the exploratory outcomes are likely to happen.
February 2015 Isaac Castillo - @isaac_outcomes 17
![Page 18: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/18.jpg)
Confirmatory vs Exploratory Outcomes Example
• A 24-week program designed to teach teens about the dangers of tobacco use and decrease (or prevent) smoking of cigarettes
• Confirmatory outcome: decrease in the self-reported number of cigarettes smoked during the past 7 days
• Exploratory outcome: decrease in the self-reported use of alcohol, marijuana, or other drugs during the past 7 days
The program SHOULD decrease use of cigarettes. And if the program decreases cigarette use, then participants MIGHT decrease use of other substances as well.
February 2015 Isaac Castillo - @isaac_outcomes 18
![Page 19: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/19.jpg)
Some Examples of Outcomes
February 2015 Isaac Castillo - @isaac_outcomes 19
Healthy eating habits
Program Outcome(s)
Sex education for teens
Parent / child communication
skills
Consumption of more fruits and
vegetables
Healthier weight
Decreased yelling / anger during conflicts
Increased knowledge of
dangers of unprotected /
early sex
Decreased risky
sexual activity
Fewer unplanned
teenage pregnancies
![Page 20: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/20.jpg)
Timing of Outcomes
• Best to think about sequence of outcomes – and not focus just on when outcomes may/should occur.
• Early / initial outcomes should lead to more outcomes, which in turn will lead to more outcomes.
• Important question: how far down the chain of outcomes will you measure?
February 2015 Isaac Castillo - @isaac_outcomes 20
![Page 21: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/21.jpg)
Short Term Outcomes
• Expect to be affected in a short period of time
• Closely related to/influenced by the program’s outputs
• Necessary steps
– Ensure program is on track
– Realistic measure of success
February 2015 Isaac Castillo - @isaac_outcomes 21
![Page 22: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/22.jpg)
Intermediate and Long-Term Outcomes
• Intermediate Outcomes– Link between short and longer-term outcomes
– Most will be confirmatory outcomes
• Long-Term Outcomes
– Change after longer period of time
– Ultimate outcomes
– Mix of confirmatory and exploratory outcomes
February 2015 Isaac Castillo - @isaac_outcomes 22
![Page 23: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/23.jpg)
How Long Will Each Type of Outcome Take?
• Will depend on the length of your intervention.
February 2015 Isaac Castillo - @isaac_outcomes 23
Short Term Intermediate Long Term
Six MonthsOne Month Twelve Months
6 Years1 Year 12 Years
![Page 24: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/24.jpg)
What is a Logic Model?
• Simple representation of a program / service / organization• Connects program services or activities to anticipated
outputs and outcomes• Logic models are never ‘finished’ – they should evolve over
time as programs change• Logic models should be for program/staff use – serve as
guide for direct service work and expected outcomes• Dozens of frameworks/formats, but all contain at least 3
things: – Activities/Services– Outputs– Outcomes
February 2015 Isaac Castillo - @isaac_outcomes 24
![Page 25: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/25.jpg)
Basic Logic Model
February 2015 Isaac Castillo - @isaac_outcomes 25
Activities
Planned services, offerings, or instruction
provided to your participants.
“What you plan to do or deliver?”
Outputs
What actually happened when you offered your
activities.
“Who actually received services
and in what amount?”
Outcomes
Changes in knowledge, attitudes,
behavior, or conditions among your participants.
“What changes actually happened
among your participants as a
result of your activities?”
![Page 26: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/26.jpg)
Simplified Logic Model
• Logic models should help you think through the flow of your program: • If you do Activity 1, you will see evidence of the activity being delivered by
measuring Output 1 and Output 2. Out of these Outputs and Activity, you would expect Outcome 1 and Outcome 2 to result.
February 2015 Isaac Castillo - @isaac_outcomes 26
Activity / Service Outputs Outcomes
Activity 1• Output 1
• Output 2
Outcome 1
Outcome 2
Activity 2• Output 3
• Output 4
Outcome 3
Outcome 4
![Page 27: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/27.jpg)
Afterschool Tutoring Logic Model
February 2015 Isaac Castillo - @isaac_outcomes 27
Activity OutputsShort-termOutcomes
Intermediate Outcomes
After-schooltutoring on
mathematics
• # of students tutored
• # of tutoring sessions offered
• # of minutes of tutoring each student receives
% of students maintaining or improving school attendance
% of students completing and submitting math homework
% of students increasing math grade
% of students with “A”s in math maintain “A”
Goal: Increase academic performance of students in mathematics
![Page 28: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/28.jpg)
The ‘logic’ of the Afterschool Tutoring Logic Model• IF your program provides after-school tutoring in mathematics, THEN you would
expect three things to happen:
– Students will attend (# of students tutored)
– Each student will attend a certain amount of tutoring sessions (# of tutoring sessions offered)
– Each student will receive a certain amount of tutoring over the program year measured in minutes (# of minutes of tutoring each student receives)
• IF you provide tutoring, and students show up and receive a certain amount of tutoring, THEN you would expect these two outcomes:
– Students maintain or improve school attendances (% of students maintaining or improving school attendance)
– Students will complete (or continue to complete) and submit math homework assignments to teachers (% of students completing and submitting math homework)
• If students attend school, complete their math homework, submit their math homework, then you would expect these two outcomes:
– Students would increase their math grades (% of students increasing math grade)
– Students that already have “A”s in math will maintain those “A”s. (% of students with As in math maintain A)
February 2015 Isaac Castillo - @isaac_outcomes 28
![Page 29: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/29.jpg)
Logic Model Formats Can Vary
Inputs Activities OutputsShort- Term Outcomes
Long-Term Outcomes
Input 1Input 2Input 3Input 4Input 5Input 6Input 7Input 8
Activity 1Activity 2Activity 3Activity 4
Output 1Output 2Output 3Output 4Output 5
Outcome 1Outcome 2Outcome 3Outcome 4
Outcome 5
February 2015 Isaac Castillo - @isaac_outcomes 29
Some logic models itemize inputs, activities, outputs, and outcomes.
![Page 30: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/30.jpg)
Logic Model With Flow
February 2015 Isaac Castillo - @isaac_outcomes 30
Inputs Activities Outputs Outcomes
Activity 3
Activity 2
Outcome 1
Output 1Activity 1
Output 3
Activity 4 Output 2
Input 2
Input1
Other logic models use arrows to show detailed progression or flow of logic. Boxes or other shapes can be used to separate different concepts within categories.
Outcome 2
![Page 31: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/31.jpg)
Logic Models Do Not Need to be Linear
February 2015 Isaac Castillo - @isaac_outcomes 31
Values
Mission
Vision
![Page 32: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/32.jpg)
Nested Logic Models
February 2015 Isaac Castillo - @isaac_outcomes 32
Activities Outputs Outcomes
Nutrition Education Program
Activities Outputs Outcomes
Student Exercise Program
Student Health Division
Educational Support Division
Parent Outreach Division
Organization
Location 1 Location 2 Location 3 Location 4
Enrichment Division
![Page 33: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/33.jpg)
Benefits of Logic Models• Set up a clear path to success or a roadmap for achieving
program goals
• Provide a visual test of whether inputs, and activities align with outputs and outcomes
• Help to get everyone “on the same page”
• Facilitate performance management and evaluation
February 2015 Isaac Castillo - @isaac_outcomes 33
![Page 34: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/34.jpg)
Who Would You Fund?
• You have $1 million to provide funding to a tutoring program for “at-risk” youth.
• You need to pick one of two programs to fund – but you can only pick one!
• Assume everything else is equal aside from the information provided on the next page. – Same service population
– Same areas of service
– Same tutoring approach
February 2015 Isaac Castillo - @isaac_outcomes 34
![Page 35: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/35.jpg)
Which Program Would You Fund?
February 2015 Isaac Castillo - @isaac_outcomes 35
Program # 1 Program # 2
Served 500 “at-risk” youth Served 50 “at-risk” youth
Provided 2,500 total hours of tutoring
Provided 2,500 total hours of tutoring
Each youth received average of 5 hours of
tutoring
Each youth received average of 50 hours of
tutoring
5 % of youth showed improved math grades on
report cards
90 % of youth showed improved math grades on
report cards
![Page 36: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/36.jpg)
Does this Change Things?
February 2015 Isaac Castillo - @isaac_outcomes 36
Mat
h P
rofi
cie
ncy
Time
Program 1 Comparison Group
Program 1
Mat
h P
rofi
cie
ncy
Program 2 Comparison Group
Program 2
Time
![Page 37: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/37.jpg)
How Do You Create an Outcome Focused Organizational Culture?
February 2015 Isaac Castillo - @isaac_outcomes 37
![Page 38: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/38.jpg)
Five Things to Keep in Mind
• Senior level leadership on outcomes
• Making outcomes someone’s responsibility
• Messaging and changing the conversation
• Making outcomes useful for front-line staff
• Think of it as multi-year culture change
February 2015 Isaac Castillo - @isaac_outcomes 38
![Page 39: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/39.jpg)
Leadership Commitment
• Organization’s leadership must buy into importance of outcomes measurement.
• Leadership must also communicate the importance of outcomes measurement throughout the entire organization.
• Work with Development/Fundraising and Finance/Accounting staff to identify sources of funding.
February 2015 Isaac Castillo - @isaac_outcomes 39
![Page 40: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/40.jpg)
Making Outcomes Someone’s Responsibility
• A staff person should be given the responsibility for making the culture change happen.
• Ideally, this staff person would have limited other responsibilities – most of their work should focus on outcomes.
• Person should be respected within organization.
• This is a people initiative, not a technology initiative.
• Technology can help, but it is not the answer.
February 2015 Isaac Castillo - @isaac_outcomes 40
![Page 41: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/41.jpg)
Messaging and Changing the Conversation
• Frame question as: “how can we provide better services to participants?”
• “Evaluation” as a positive and useful activity, not as a negative or judgmental process.
• Messaging needs to be consistent – think of this as an internal marketing activity.
• Staff and organizations will have legitimate concerns, and they should be addressed honestly.
– “Not enough time to do outcomes measurement”
– “Not enough resources to do outcomes measurement”
– “Don’t have the expertise or skills to do outcomes measurement”
February 2015 Isaac Castillo - @isaac_outcomes 41
![Page 42: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/42.jpg)
Making Outcomes Useful
• Work with staff to develop functional reports.
• Share reports with staff.
• Train staff on how to access and use their own information.
• Set aside time to talk about data and reports in meetings.
February 2015 Isaac Castillo - @isaac_outcomes 42
![Page 43: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/43.jpg)
Information to Better Serve Clients
• Collect information to modify or improve your services.
• Human beings and their behaviors are very complex – so focus on a few things and measure those as best as you can.
• You should have a clearly articulated connection between what you do and the outcomes you hope to achieve.
• In most cases, you will be collecting more than your funders require.
February 2015 Isaac Castillo - @isaac_outcomes 43
![Page 44: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/44.jpg)
Be Honest With Staff
• Focus on the positive – you will collect more information to better serve clients……
• But be honest.
– Will take some additional work on their part, especially at the beginning.
– Some people will resist the idea – “not in the job to collect data, in the job to help people”
• Provide reports back as quickly as possible. Use these to illustrate the importance of the process.
February 2015 Isaac Castillo - @isaac_outcomes 44
![Page 45: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/45.jpg)
Process Will Take Time• Most organizations will take 3-5 years to achieve the
culture change, depending on size and leadership.
• Avoid temptation to rush or overcommit.
• Set expectations up front – let everyone know the process will take years (not weeks or months).
February 2015 Isaac Castillo - @isaac_outcomes 45
![Page 46: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/46.jpg)
Be Realistic About Process
• Do not try to attempt to measure everything.– Will only result in poor quality information.– People will become frustrated or overwhelmed.
• Use what you collect.– Have reports in place where people can view and discuss
the information collected. – If you don’t plan on using it – don’t bother collecting it.
• Start small, then expand. – Better to collect 2-3 things very accurately, and use this
information, than try to collect dozens of things poorly. – Can always add on additional things to collect in future
years.
February 2015 Isaac Castillo - @isaac_outcomes 46
![Page 47: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/47.jpg)
What is Performance Management?
• Performance management is the collection of information by programs, organizations, or people for the primary purpose of program / service improvement.
• Successful performance management relies on three important features:
1) The regular and accurate collection of information, which is then…...
2) Shared in useable formats, which is then…..
3) Used to make changes in the hope of improving something (services, programs, conditions, decision making, etc.)
February 2015 Isaac Castillo - @isaac_outcomes 47
![Page 48: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/48.jpg)
Performance Management
Cycle
February 2015 Isaac Castillo - @isaac_outcomes 48
Define Success
Data Collection Methods
Train Staff
Begin Program
Ongoing Checks
Interim Reports
Complete Data Collection
Examine Data
Prepare Reports
Share Outcomes
Use Outcomes To Change Program
![Page 49: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/49.jpg)
Performance Management and Evaluation
February 2015 Isaac Castillo - @isaac_outcomes 49
Performance Management Regular measurement,
Improve outcomes
EvaluationOccasional measurement,
Improve effectiveness
![Page 50: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/50.jpg)
PM and Eval: Speed
Performance Management is rapid and frequent
February 2015 Isaac Castillo - @isaac_outcomes 50
Evaluation is methodical and delayed
![Page 51: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/51.jpg)
PM and Eval: Data Collection and Analysis
February 2015 Isaac Castillo - @isaac_outcomes 51
Performance Management is simple and requires little
expertise
Evaluation can be complex and often requires specialized training
![Page 52: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/52.jpg)
PM and Eval: Completeness
February 2015 Isaac Castillo - @isaac_outcomes 52
Performance Management gives you enough to guess at
whole puzzle
Evaluation lays out as much of the puzzle as possible
![Page 53: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/53.jpg)
Why Bother With All of This?
February 2015 Isaac Castillo - @isaac_outcomes 53
Ultimately, you should be measuring outcomes or effectiveness for a
single reason:
To better serve your clients / population.
![Page 54: Can you prove you are making a difference? February 2015](https://reader030.vdocument.in/reader030/viewer/2022032421/55a702931a28abb11f8b45ca/html5/thumbnails/54.jpg)
Isaac’s Contact Information
54February 2015 @Isaac_outcomes
Isaac D. Castillo
Deputy Director
DC Promise Neighborhood Initiative
On Twitter: @Isaac_outcomes
Email: [email protected]