measuring the effectiveness of your volunteer program meals on wheels association of america 2011...
TRANSCRIPT
![Page 1: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/1.jpg)
Measuring the effectiveness of your volunteer program
Meals On Wheels Association of America
2011 Annual Conference
August 30, 2011
Stacey McKeever, MPH, MBA
do good Consulting
![Page 2: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/2.jpg)
Introductions
• Name and MOWAA chapter
• Role with Meals On Wheels
• One specific data strength - and -
• One specific data challenge
![Page 3: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/3.jpg)
Session objectives
• Learn to collect volunteer program data
• Learn to analyze volunteer program data
• Learn to present volunteer program data
![Page 4: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/4.jpg)
Session overview
• Overview of our programs• Framework for planning our programs• Activity: program planning• How to evaluate our programs• Developing an evaluation plan• Tips on sources and uses of data• Tips of data analysis and presentation
![Page 5: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/5.jpg)
What do our programs look like?
Staffing Volunteers
Programs
OtherBudget
Service recipients
GENERAL OVERVIEW
![Page 6: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/6.jpg)
What do our programs look like?
Staffing Volunteers
Programs
OtherBudget
Service recipients
DATA COLLECTION
![Page 7: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/7.jpg)
What do our programs look like?
Staffing Volunteers
Programs
OtherBudget
Service recipients
DATA ANALYSIS
![Page 8: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/8.jpg)
What do our programs look like?
Staffing Volunteers
Programs
OtherBudget
Service recipients
DATA PRESENTATION
![Page 9: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/9.jpg)
Measurement basics: impact
• Impact: change in the health, social and economic status of the population of interest through sector-specific contributions
![Page 10: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/10.jpg)
Measurement basics: effects
• Effects: change in the knowledge, attitudes, skills, intentions, and/or behaviors of the population of interest that contributes to the desired impact
![Page 11: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/11.jpg)
Measurement basics: outcomes
• Outputs: products and services that must be in place for the effects to be achieved
![Page 12: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/12.jpg)
Measurement basics: activities
• Activities: technical and support tasks required to produce the outputs
![Page 13: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/13.jpg)
Measurement basics: inputs
• Inputs: resources required to support your activities
![Page 14: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/14.jpg)
Putting it together
• This set of inputs and activities will result in these products and services [outputs], which will facilitate these changes in the population [effects], which will contribute to the desired impact.
![Page 15: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/15.jpg)
Activity: planning your program
Inputs Activities Outputs Effects Desired Impacts
![Page 16: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/16.jpg)
Activity: planning your program
• This set of inputs and activities will result in these products and services [outputs], which will facilitate these changes in the population [effects], which will contribute to the desired impact.
![Page 17: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/17.jpg)
Evaluation status
• Novice, intermediate or expert?
• Informal or systematic data collection?
• Qualitative, quantitative or both?
![Page 18: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/18.jpg)
Evaluation
• Evaluation is a process
• Evaluation involves gathering information
• Gathered information can be qualitative, quantitative or both
• Evaluation is used to improve programs
![Page 19: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/19.jpg)
Evaluating your program
Inputs Activities Outputs Effects Desired Impacts
Did you have the inputs you needed?
Were your activities carried out as planned?
Were the planned outputs achieved?
Did the effects occur?
Was impact achieved?
![Page 20: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/20.jpg)
Evaluating your program: indicators
• Measure the links using indicators
Inputs Activities Outputs Effects Desired Impacts
![Page 21: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/21.jpg)
Output indicators
• Output indicators measure products and services provided by the program, and the quality of these products and services.
• Usually, project records are the main source of information for output indicators.
Inputs Activities Outputs Effects Desired Impacts
![Page 22: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/22.jpg)
Output indicators
• # of outreach staff trained
• % of trained outreach staff who received rating of “good” or “excellent” on final training exercise
• % of trained outreach staff who perform education and service responsibilities “well” or “very well”
Inputs Activities Outputs Effects Desired Impacts
![Page 23: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/23.jpg)
Output indicators
• To [deliver / establish]a [specific and quantified product or service]by [health workers or system] to [population of interest] by [time frame]
Inputs Activities Outputs Effects Desired Impacts
![Page 24: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/24.jpg)
Effects indicators
• Effect indicators measure the knowledge, attitudes, skills, intentions and behaviors of the population we are trying to help.
• requires measurement at the population level (such as
with a community-based survey)
Inputs Activities Outputs Effects Desired Impacts
![Page 25: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/25.jpg)
Effects indicators
• To reduce the proportion of asthmatic children in the project area who have visited an ER for asthma management in the prior 12 months to under 10% in 2 years.
• To increase the proportion of 6-11 year-old schoolchildren in the project area who can correctly identify 8 out of 10 ‘healthy’ and ‘non-healthy’ foods from 30% to 80% by the end of the academic year.
Inputs Activities Outputs Effects Desired Impacts
![Page 26: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/26.jpg)
Effect indicators
• [# or %] of
[population group of interest] who
[know / believe / do]
[specific knowledge / attitude / skill / behavior]
Inputs Activities Outputs Effects Desired Impacts
![Page 27: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/27.jpg)
Effect indicators
• To [increase / decrease] the [specific knowledge, attitudes, skills,
behaviors] among [specific population of interest] from-to [from baseline to desired level] or to [to standard level] by [time frame]
Inputs Activities Outputs Effects Desired Impacts
![Page 28: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/28.jpg)
Impact indicators
• Impact indicators measure the health, social or economic status of the population of interest.
• They are often rates or ratios and are virtually always population-based measures.
Inputs Activities Outputs Effects Desired Impacts
![Page 29: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/29.jpg)
PROGRAM MONITORING AND EVALUATION PLAN SUGGESTED FORMAT
Objectives Note: Indicators for ALL
objectives must be presented
Indicator(s) Source of data/ Data collection
method
Frequency of collection/
compilation
Comments
Additional key outputs to measure
Add lines as needed
![Page 30: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/30.jpg)
Sources of data
• Volunteer databases• Paper sign-up sheets• Excel spreadsheets• Focus groups• Surveys• One-on-one interviews• Event sign up sheets• Event organizing report• Online hours logging tool• Timesheet/timecard• Volunteer event management application• Telephone calls• Document and record analysis
![Page 31: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/31.jpg)
Uses of data
• Share with board• Share with staff• Share with funders• Use to improve program effectiveness• Use to improve volunteer experience• Use to improve service recipient experience• Organizational strategy• Staffing decisions• Budget decisions• Show community support• Design targeted recruitment and retention strategies
![Page 32: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/32.jpg)
Creating a data analysis plan
• Establish a baseline for benchmarking
• Establish goals for each quarter/year
• Establish a timeline for collecting and reporting
• Determine how to use them for internal purposes
• Create committees
• Define, define, define
• Be consistent
![Page 33: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/33.jpg)
Presenting your data
• Use charts, graphs and maps
• Incorporate quantitative and qualitative data
• Compare data historically
• Show progress against baselines
• Use timelines to show progress
• Use quotes, stories and pictures
![Page 34: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/34.jpg)
Action planning
• Where
• What
• Who
• Why
• When
![Page 35: Measuring the effectiveness of your volunteer program Meals On Wheels Association of America 2011 Annual Conference August 30, 2011 Stacey McKeever, MPH,](https://reader036.vdocument.in/reader036/viewer/2022062421/56649e5d5503460f94b559fb/html5/thumbnails/35.jpg)
For more information:
do good Consulting
201 West Green Street
Urbana, IL 61801
217-778-1687
www.dogoodconsulting.org