measure what matters karen macdonald, bgca midwest leadership conference october 16, 2009
TRANSCRIPT
![Page 1: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/1.jpg)
Measure What Matters
Karen MacDonald, BGCA
Midwest Leadership Conference
October 16, 2009
![Page 2: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/2.jpg)
“Not everything that counts can be counted, not
everything that can be counted counts.”
![Page 4: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/4.jpg)
“In God we trust, all others bring data.”
![Page 5: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/5.jpg)
“Data is the new sexy!”
![Page 6: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/6.jpg)
Why is Outcome Measurement important?
![Page 7: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/7.jpg)
• Era of accountability
• Doing good is not enough
• Stakeholders are asking more difficult
questions
The Importance of Measurement
![Page 8: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/8.jpg)
Formal Program Evaluation Outcome Measurement
Does the program work? How well is it working?
• Proving • Improving
• Research tool • Management tool
• Applies to programs • Applies across org
• Analyzing • Learning
• Theory of Change • Best Practices
Program Evaluation vs Outcome Measurement:
![Page 9: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/9.jpg)
![Page 10: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/10.jpg)
Key Elements of Impact
• Safe, positive environment• Fun• Supportive relationships• Opportunities and expectations• Recognition
![Page 11: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/11.jpg)
Making Every Day Count
• Longitudinal Study • Tracked 7th and 8th graders
for 30 months• Linked levels of
attendance at Boys & Girls Clubs to outcomes
• Finding – MORE IS BETTER
![Page 12: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/12.jpg)
Outcome (All changes are positive changes)
Minimum Level of Attendance Over 30-Months Related to Positive Change in
Outcome
52 Days 122 Days 244 Days 366 Days
School effortConfident in abilityIntegrityCommunity service Initiate marijuana use Future connectedness
ShynessAggressionInitiate weapon carryingStopped by policeInitiate drinking Initiate Smoking
Number times skippedInitiate sexual activity
Negative peers
![Page 13: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/13.jpg)
Four things led to teen participation…
1. Fully engaged in a variety of activities
2. Friends came to the Club3. Involved in leadership roles4. First became involved as
preteens or even younger
![Page 14: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/14.jpg)
Making Every Day Count affirms…
• Attendance matters• Participation matters• Fun matters• Relationships matter• Programs matter
![Page 15: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/15.jpg)
Annual VisitsDefinition: The number of times a member
attends the Club over the course of the year.
• It’s important to track this metric.• 52 - 104 times per year is the baseline• Increase attend levels of all members
regardless of current attendance patterns
![Page 16: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/16.jpg)
A big thank you to…• Boys & Girls Clubs
of Greater Milwaukee
• Boys & Girls Clubs of Omaha
![Page 17: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/17.jpg)
Planning a Measurement Strategy:Key Steps
• Identify key organizational priorities and select
key indicators
• Track attendance and program participation• Locate and collect data• Understand and use data• Communicate results
![Page 18: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/18.jpg)
![Page 19: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/19.jpg)
•Review sources that outline your priorities
including your strategic plan
•Review the needs of your community
•Involve your board by asking questions
Identify Organizational Priorities
![Page 20: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/20.jpg)
Success Equation Components
A + B + C = D A + B + C = D
What are you ultimately trying to accomplish?
Priority OutcomesWhat changes in behavior or condition are required?
IndicatorsHow do we measure progress
against our outcomes?
Key StrategiesWhich
programs or activities will
drive results?
© 2008 Mission Measurement LLC – as adapted from Benchmarking for Nonprofits, by Jason Saul
![Page 21: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/21.jpg)
• Use your success equation and priority outcomes as your guide
• Start with the basics: attendance and program participation
• Continue with more complex indicators: changes in knowledge/attitude and changes in behavior
• Use the Outcomes Framework as a guide
• Select indicators that are feasible to collect and credible to communicate
Select Indicators to Track Progress
![Page 22: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/22.jpg)
•Practical – Can you realistically capture this data without a lot of cost or difficulty?
•Actionable – Will the data allow you to make decisions in a timely manner?
•Relevant – Does it answer the “so what” question?
•Controllable – Is it within your sphere of influence or control?
Characteristics of Good IndicatorsCharacteristics of Good Indicators
![Page 23: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/23.jpg)
• more time spent going to the Club was linked to positive change
• research supports the importance of regular attendance and the links to positive outcomes.
• at least 52 times per year is a baseline • Clubs should strive to increase the attendance levels of all
members • Average daily attendance is an important measure of
Club’s operational capacity, but does not measure individual members and is therefore not an indicator of changes for youth.
Track Attendance and Program Participation
![Page 24: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/24.jpg)
Locate and Collect DataLocate and Collect Data• Collect on a program-by-program basis• Leverage existing tools, i.e. test or surveys• Use Club-wide surveys when warranted• Tests or Surveys can be administered by hand or electronically• Results from the tests or surveys can be tracked in the member’s management system.
![Page 25: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/25.jpg)
Outcome Indicator Indicator Definition Location of data Frequency Collect Where
Report
Academic Achievement
Indicator 1
Indicator 2
Indicator 3
Locate and Collect DataLocate and Collect Data
![Page 26: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/26.jpg)
• Improve program and/or youth results • Share progress and success
• Share results alone • 50% of members participated in community
service.• Share results compared past performance
(growth/change)• Participation in community service activities has
increased by 25%.
Understand and Use ResultsUnderstand and Use Results
![Page 27: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/27.jpg)
Indicator Sample Impact Statement Links to External Research
Number of volunteer hours completed by Club members
75% of Boys & Girls Clubs of Main Street members participated in volunteer service activities in 2008 and contributed a total of 1,000,000 hours of service.
Young people who volunteer are likely to do better in school, graduate, maintain positive attitudes towards society and avoid risky behaviors (4)
Simply based on regular member attendance and program participation, Clubs can demonstrate the impact they are having by linking participation to external research on positive activities:
![Page 28: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/28.jpg)
• Explore and define how measurement can and will be used
• Setting the right tone and creating ownership and buy-in
• Setting the process in motion • Identify the processes and plans needed to bring
measurement to life
Engage your StaffEngage your Staff
![Page 29: Measure What Matters Karen MacDonald, BGCA Midwest Leadership Conference October 16, 2009](https://reader035.vdocument.in/reader035/viewer/2022062422/56649ebd5503460f94bc7589/html5/thumbnails/29.jpg)
Available Resourceswww.bgca.net/demonstratingimpact
• Demonstrating Impact Guide• Outcome Measurement Framework• BGCA - Public/Private Ventures Study• Distance Learning courses
Abigail Diner, [email protected] MacDonald, [email protected]
Romero Brown, [email protected]