tips and tools for measuring program impacts: from delivering programs to reporting impacts office...
TRANSCRIPT
Tips and Tools for Measuring Program Impacts:
From Delivering Programs to Reporting Impacts
Office of Educational Innovation and Evaluation2323 Anderson Avenue, Suite 220Kansas State UniversityManhattan, Kansas 66502Phone: 785-532-5930Fax: 785-532-7185
Part I: Preparing to Evaluate Your Program
TipsFind and review evaluation instruments
Utility and feasibility
Purposes of evaluation: Accountability, Improvement, Knowledge Development, Oversight and Compliance “Who wants to know what about your program?”
Multiple, complementary sources of data
ToolsRubric: Reviewing Evaluation Instruments
Low Medium High Notes
UTILITY Value to PFTConsider: How well will the evaluation instrument gather data that aligns with the PFT action plan?
Alignment with Program CurriculumConsider: How well aligned is the evaluation instrument with the curriculum that was taught?
Part II:Collecting and Managing Data to
Measure Impacts
TipsPlan early – determine when evaluation will
take place in the programming cycle
Review and manage data as you receive it
Analysis quantitative and qualitative by hand, using Excel “Who wants to know what about your program?”
ToolsData Analysis Plan Worksheet
Question or Item Number
What did I hope to learn from this
question?
What analysis technique would assist
in that learning?
How could I present the data to convey what I
learned?
E.g., #2
If participants felt that they increased their knowledge of best management practices for a healthy yard.
Count and percentage of respondents who reported “Agree” or “Strongly Agree”
Bar or pie graph that compares “Agree” and “Strongly Agree” responses to “Disagree,” “Strongly Disagree” and “Unsure” responses
Part III:Presenting Impact Data
TipsKSRE Impact Reports
“Who wants to know what about your program?”
Illustrate your data
Be clear and concise, avoid jargon
Use the active voice
TipsChoose Graphics to Effectively Convey Data
RegionEducation (in Years of Schooling)
≤8 9-11 12 13-15 ≥16
Northeast 25.32 25.29 18.26 18.29 16.34
North Central 32.09 29.04 18.78 24.32 19.02
South 38.81 31.02 19.33 15.66 16.79
West 25.37 21.09 20.29 23.97 17.52
Example #1 – Table NOT formatted effectively
Table 1. Chewing Gum Rates in the United States by Geographic Region and Level of Education*
*Education level based on number of years of schooling. Adapted from a report of the U.S. Department of Chewing Gum.
Education Level
Region
Northeast West SouthNorth
CentralAverage
≤8 25 25 39 32 309-11 25 21 31 29 2712 18 20 19 19 1913-15 18 24 16 24 21≥16 16 18 17 19 17Average 20 22 24 25 23
Example #2 – Table IS formatted effectively
Table 2. Chewing Gum Rates in the United States by Geographic Region and Level of Education*
ToolsWhat Does the Data Mean?
What are the key findings? Write a paragraph to summarize the results.
What do you find interesting about these results? What would your audience find interesting?
E.g., After participating in the program, many participants reported feelings of increased financial abilities. In particular, a majority of participants reported increased skills to track spending (64%), to create a spending plan (71%), and to plan for unexpected expenses (77%). One respondent commented, “The skills I learned from this program will help me stretch my money to the next payday.”
QUESTIONS?
Contact Information
Office of Educational Innovation and Evaluation Kansas State University
2323 Anderson Avenue, Suite 220Manhattan, KS 66502Phone: 785-532-5930
www.ksu.edu/oeie
Sarah Bradford(785) 532-5677
Amy Hilgendorf(785) 532-5538