unit 7: using your evaluation – communicating, reporting, improving
Post on 14-Feb-2016
41 Views
Preview:
DESCRIPTION
TRANSCRIPT
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Unit 7: Unit 7: Using your evaluation – Using your evaluation –
Communicating, reporting, improvingCommunicating, reporting, improving
1. Who2. What (Content)3. How (Format)4. When
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
“Think like a wise man, but communicate in the language of the people.”
− William Butler Yeats
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Why communicate? Why communicate?
The proper function of evaluation is to speed up the learning process by communicating what might otherwise be overlooked or wrongly perceived…Success is to be judged by… success in communication…Payoff comes from the insight that the evaluator’s work generates in others.
- L. J. Cronbach
Cronbach, L.J. (1982). Designing evaluations of educational and social programs. San Francisco: Jossey-Bass, p. 8.
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Plan with the end in mind.
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Types of communications - reportsTypes of communications - reports
• Written report - long, short, summary
• Success story• Impact statement;
spotlight• Elevator story• Press release• Media appearance
• Public meeting• Memo, email, fax,
postcard• Newsletter• Personal discussion• Bulletin, brochure• Display/exhibit• Audio/video
presentation
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Communication plan Communication plan
WHO?WHAT?HOW?WHEN?
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
WHO - Potential UsersWHO - Potential Users• County board/City council (elected and appointed officials)• Funders• Program Committee• Participants• Team / Staff• Volunteers• Collaborating agencies; partners• Businesses; business groups• Police – law enforcement• Schools boards, parent-teacher organizations• Community• Church organizations• State legislators• Professional organizations• You
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
WHO is the Audience?WHO is the Audience?
1.
2.
3.
4.
INTERNAL
1.
2.
3.
4.
EXTERNAL
Primary audience or secondary audience??
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
WHAT – WHAT – will you say, report, communicate?? will you say, report, communicate??
• What does the audience care about?• What do you want the audience to have?• What type of information resonates with the
audience: numbers, quotes, stories?
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
HOW will you communicate – HOW will you communicate – what formats will you use?what formats will you use?
• Report• Impact statement• Executive summary• Personal discussion• Oral presentation• Photo display• Press release• Newsletter, bulletin• Poster• ??????
Overview
Cooperative Extension- 72 county offices- Family Living Programs- 4-H Youth Development- Community, Natural
Resources andEconomic Development
- Agriculture and NaturalResources Education
- Geologic and NaturalHistory Survey
- 1.4 million educationalcontacts per year
Continuing Education- Partnerships with 26
UW System campuses- UW-Learning
Innovations- School for Workers- 162,000 enrollments per
year
Broadcasting & MediaInnovations- Wisconsin Public
Television- Wisconsin Public Radio- Instructional
CommunicationsSystems
- Nearly 1 million viewersand listeners per week
Small BusinessDevelopment Centers- 13 campus-based
centers- Wisconsin Innovation
Service Center- 15,000 hours of
counseling per year
www.uwex.eduinfo@uwex.edu(608) 262-3786
University of Wisconsin-Extension programs serving the98th Assembly District
Representative Marc Duff
Waukesha County Extension — The Waukesha County UW-ExtensionCooperative Extension office has developed partnerships with numerousorganizations to deliver educational programs addressing local needs. The Waukesha County Extension office developed a Hunger in Waukesha
County Nutrition Survey for area communities. These communities alsoreceive nutrition education services through the Waukesha County NutritionEducation Program at sites like the WIC clinics, elderly nutrition sites andfood pantries.
For the past two years, budgeting, nutrition and food safety have been taughtthrough Waukesha County Extension’s Nutrition Education Program at theWaukesha County Jail. The program also links inmates to resources that willbe available to them once they are released.The Nutritional Education Program also works with individuals with drug andalcohol abuse problems who are living at the Waukesha County HuberFacility. The program focuses on healthy eating and the effects of drugs andalcohol on the body, food safety and budgeting.
The Parenting the First Year newsletter is distributed through CommunityMemorial Hospital in Menomonee Falls with funding from the local Kiwanisclub. The Parenting the Second and Third Year newsletter is distributedthrough Community Memorial under a grant from the Child AbusePrevention Fund (CAP). The project currently is being evaluated in order tomeasure the value to parents. As a result of this partnership, parentingeducation programs such as Home Alone, Parenting Your Teen andDiscipline vs. Punishment are also offered.
The Waukesha County Extension office provided leadership to a collaborationof 14 agencies/organizations serving children and families to conduct a two-to-three-year program to disseminate research and conduct comprehensivetraining about early learning and brain development of infants and children.
Waukesha County Extension offers a Grant Opportunities Newsletter, whichidentifies grants available to the community from a variety of state and federalagencies as well as local foundations. The list includes federal agencies, suchas the Department of Health and Human Services, Department of Housing andUrban Development, Department of Labor and Employment and TrainingAdministration, and state entities including the Department of WorkforceDevelopment, Department of Administration - Division of Housing, TEACH,Department of Transportation, Department of Natural Resources and theWisconsin Advanced Telecommunications Foundation.
The Waukesha County Extension-Urban Initiative Program in Sussex willaddress neighborhood revitalization, building a sense of community, crimeand public safety. In 2001 the program received a $10,000 CommunityDevelopment Block Grant which will allow expansion of the partnership andimplementation of an action plan in at-risk, troubled neighborhoods.
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
HOW – what format, style will you use?? HOW – what format, style will you use??
• Format depends on purpose and audience
• Written, oral• Short, long• How does the audience prefer receiving
information? How does the audience best learn?
• What do you have the resources to do?
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
WHEN will you communicate?WHEN will you communicate?
• Quarterly?• Opportunistically?• When requested?• ???
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Reporting outline follows your logic modelReporting outline follows your logic model
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Formal evaluation reports typically include…Formal evaluation reports typically include…• Abstract/executive summary• Introduction
− Purpose of the evaluation; key questions− Program background, description
• Methods/procedures− Data sources− Data collection procedures− Sampling− Limitations
• Results• Discussion• Conclusions/recommendations• References• Appendices
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Effective communicationsEffective communications• Tailor message to issue and audience• Avoid jargon and technical terms• Be clear, concise• Use active voice• Eliminate wordiness• Check writing, grammar• Be accurate, balanced, impartial• Be timely• Use graphics, quotes, photos, real stories• Consult a communications specialist• Write-rewrite-rewrite
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
MythsMyths• One report is enough.• People read written reports.• Complex analysis and big words impress people.• Oral reports have the same effect as written
reports.• Describing limitations weakens report.• Everything should be reported.• The audience knows why they are getting the
report.
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Ingredients of a Good Success StoryIngredients of a Good Success Story
• Involves human interest: catches your attention • Demonstrates behavior change; action taken
− change in practice that results in benefits to individual, community, environment...
• Uses words of/meaning for people involved • Indicates profit to the individual, if appropriate• Based on reliable, credible information • Includes numbers and narrative• Presents balanced, fair assessment
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Make sure the evaluation isn’t ignoredMake sure the evaluation isn’t ignored
• Engage stakeholders in doing the evaluation: in the design, data collection, in analysis, interpretation of results…
• Get the information to the right people• Address issues that people think are
important• Keep it in front of people• Present it in time to be useful
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Discuss limitationsDiscuss limitationsWritten reports:
• Be explicit about your limitationsOral reports:
• Be prepared to discuss limitations• Be honest about limitations• Know the claims you cannot make
− Do not claim causation without a true experimental design
− Do not generalize to the population without random sample and quality administration (e.g., <60% response rate on a survey)
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Reporting results to the mediaReporting results to the media
All Media: • Avoid using too many statistics. • Focus on the key points.• For quotes, speak more globally about
the issue.• Always give the source and timeliness
of your stats. It’s the “news peg.”Steve Busalacchi
Director, News & InformationWisconsin Medical Society
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Reporting results to the mediaReporting results to the media
Radio and TV:• Do not offer exact statistics
– ear cannot track. “73.6% of respondents”vs.“Nearly three quarters of those surveyed”
• Don’t go into great detail. Have backup info ready.
Steve BusalacchiDirector, News & Information
Wisconsin Medical Society
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Using graphicsUsing graphicsWhat do I want my audience to
learn – remember –
from my graph or table?
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Using graphicsUsing graphics• Title • Clear units of measure • Date(s) data collected• Simple, straightforward design without
“clutter”• Font size 10 point or larger• Explicit data source(s)• Sample size, if applicable for the
audience
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
GraphsGraphs
• Bar graphs: show comparisons− Stacked bar− Multiple bars− Vertical, horizontal bars
• Pie charts: show parts of a whole• Line graphs: show progress over time
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Bar graphBar graph
• Compare groups of client outcomes• Compare points in time
Each group you compare gets a “bar”
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Percentage of worksites implementing "Healthy Lifestyles" program
60%
74%
0102030405060708090
100
Williams CountyWorksites
Wisconsin Worksites
Perc
enta
ge o
f wor
ksite
s
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Horizontal Graph
Example Bar Graph
0 10 20 30 40 50 60
1
2
3
4
Series2
Series1
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Vertical Graph
Example Bar Graph
0
10
20
30
40
50
60
1 2 3 4
Series1Series2
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Column Chart
0%
20%
40%
60%
80%
100%
1
Column Chart
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Pie charts: parts of a wholePie charts: parts of a whole
• Useful for displaying proportions− Percentage of clients achieving the outcome− Percentage of participants who are boys− Percentage of sites that are rural
• Ensure individual parts add to 100%
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Pie ChartPie Chart
Example Pie Chart
123
4
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Line graphs: time linesLine graphs: time lines
• X axis conveys the time • Y axis is the variable of interest• Multiple lines or multiple line graphs
show performance of different groups
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Line GraphLine Graph
Example Line Graph
0
10
20
30
40
50
60
1 2 3 4
Series1Series2
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Area GraphArea Graph
Example Area Graph
0
10
20
30
40
50
60
70
1 2 3 4
Series2Series1
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Pictogram Pictogram End of
November Total!
University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation
Graphing using Microsoft ExcelGraphing using Microsoft Excel• Click on graph icon on toolbar, or select
“insert” and “chart”• Proceed through chart wizard
− Select the type of graph you want− Highlight the range of data to include− Set up titles, axis labels, scales, legend− Finish
• Edit the chart using drop down menu for charts
− Click on chart to highlight it− Use the top toolbar selection “chart” to edit
parts of the chart
top related