the evaluation plan. slide 2 innovation network, inc. session purpose to understand how evaluation...
TRANSCRIPT
![Page 1: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/1.jpg)
The Evaluation Plan
![Page 2: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/2.jpg)
Slide 2Innovation Network, Inc.
Session Purpose
To understand how evaluation can be useful To understand how your logic model helps to
focus an evaluation To understand both implementation and
outcome evaluation
![Page 3: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/3.jpg)
Slide 3Innovation Network, Inc.
What is Evaluation?
The systematic collection of information about a program in order to enable stakeholders to better understand the program, to improve program effectiveness, and/or to make decisions about future programming.
![Page 4: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/4.jpg)
Slide 4Innovation Network, Inc.
What’s in it for you?
Understand and improve your program Test the theory underlying your program Tell your program’s story Be accountable Inform the field Support fundraising efforts
![Page 5: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/5.jpg)
Slide 5Innovation Network, Inc.
Evaluation Principles
Evaluation is most effective when it: Is connected to program planning and delivery Involves the participation of stakeholders Supports an organization’s capacity to learn and reflect Respects the community served by the program Enables the collection of the most information with the
least effort
![Page 6: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/6.jpg)
Slide 6Innovation Network, Inc.
Logic Model
Resources
The resources dedicated to or consumed by the program
Activities
The actions that the program takes to achieve desired outcomes
Outputs
The tangible, direct results of a program’s activities
Outcomes
The benefits to clients, communities, systems, or organizations
Program Goals: overall aims or intended impacts
External Factors: what else affects the program
![Page 7: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/7.jpg)
Slide 7Innovation Network, Inc.
Putting Your Plans Together
Logic Model Resources OutcomesActivities Outputs
ImplementationActivities
OutcomesOutcomes
Data CollectionMethod Effort
IndicatorsData CollectionMethod Effort
Evaluation Plan:
Outputs
![Page 8: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/8.jpg)
Slide 8Innovation Network, Inc.
Implementation and Outcomes
Evaluating Outcomes: What changes occurred as a result of your work?
Evaluating Implementation: What did you do? How well did you do it?
![Page 9: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/9.jpg)
Slide 9Innovation Network, Inc.
Evaluating Outcomes
Outcomes: the changes you expect to see as a result of your work
Indicators: the specific, measurable characteristics or changes that represent achievement of an outcome. They answer the question: How will I know it?
![Page 10: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/10.jpg)
Slide 10Innovation Network, Inc.
Evaluating Outcomes:Indicators = What to Measure
Meaningful Direct Useful Practical
![Page 11: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/11.jpg)
Slide 11Innovation Network, Inc.
Evaluating Outcomes:Direct v. Indirect Indicators
Outcome Indicator
Participating new mothers have their children immunized.
Indirect: #/% of new mothers who are aware of importance of immunization.Direct: #/% of children of participating mothers who are up-to-date in immunizations.
Increase referrals to your services from targeted doctors.
Indirect: #/% of increase in your client base.Direct: #/% of increase in your client base from targeted doctors.
Targeted teens learn about certain environmental health hazards.
Indirect: # of students who receive brochure on topic.Direct: #/% of students who ID 3 health hazards.
![Page 12: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/12.jpg)
Slide 12Innovation Network, Inc.
Evaluating Outcomes: Template
Outcomes Indicators Data Collection Method
Level of Effort(have, low, med,
high)
![Page 13: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/13.jpg)
Slide 13Innovation Network, Inc.
Evaluating Outcomes: Example
Outcome Indicators Data Collection Method
Data Collection Effort
(have, low, med, high)
Participants learn job-seeking skills
#/% of participants who can meet criteria in mock interview
Observation of mock interview conducted at end of training session using observation checklist
Low
#/% of participants who develop a quality resume
Job counselors review resumes based on quality checklist
Have
Participants obtain and carry out job interviews
#/% of participants who go on at least 2 job interviews
Tracking by job counselors Have
![Page 14: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/14.jpg)
Slide 14Innovation Network, Inc.
Evaluating Implementation
Activities and Outputs: The “what” —the work you did, and the tangible results of that work
Additional Questions: The “why”—understanding how well you did, and why
![Page 15: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/15.jpg)
Slide 15Innovation Network, Inc.
Evaluating Implementation: Understanding how well you didWhat information will help you understand your program
implementation? Think about:
Participation Quality Satisfaction Context
![Page 16: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/16.jpg)
Slide 16Innovation Network, Inc.
Evaluating Implementation: Template
Activities Outputs &Implementation
Questions
Data Collection Method
Data Collection Effort(have, low, med, high)
Program Component Outputs
Questions
Program Component Outputs
Questions
![Page 17: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/17.jpg)
Slide 17Innovation Network, Inc.
Evaluating Implementation: Example
Activities Outputs &Implementation Questions
Data Collection Method
Data Collection
Effort(have, low, med, high)
Develop/revise curriculum for training seriesMeet with potential program clientsProvide training session series to two groups of clients
Outputs Updated curriculum2 training session series held#/rate of participation by group
Program recordsRecordsRecords, logs
Have
Have
QuestionsAre we getting the clients we expected to get?(Partic.)Are they satisfied w/ training? What did they like most, least? (Satisfaction)
Review of participant intake dataParticipant survey
Low
Med
![Page 18: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/18.jpg)
Slide 18Innovation Network, Inc.
Data Collection
Determine what methods will you use to collect the information you need?
Choose the method Decide which people, or records will be the source of
the information Determine the level of effort involved in using that
method with that population.
![Page 19: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/19.jpg)
Slide 19Innovation Network, Inc.
Data Collection Methods
Review documents Observe Talk to people Collect written information Pictorial/multimedia
![Page 20: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/20.jpg)
Slide 20Innovation Network, Inc.
Issues to Consider
Resist pressure to “prove” Start with what you already collect Consider the level of effort it will take to gather the data.
Prioritize. What do you need to collect now, and what
can wait until later?
![Page 21: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/21.jpg)
Slide 21Innovation Network, Inc.
Data Collection: Level of Effort
Instrument development Cost/practicality of actually collecting data Cost of analyzing and presenting data
![Page 22: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/22.jpg)
Slide 22Innovation Network, Inc.
Qualitative Data
Usually in narrative form—not using numbers
Collected through focus groups, interviews, open-ended questionnaire items, but also poetry, stories, diaries, and notes from observations
![Page 23: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/23.jpg)
Slide 23Innovation Network, Inc.
Quantitative Data
Pieces of information that can be expressed in numerical terms, counted, or compared on a scale
Collected in surveys, attendance logs, etc.
![Page 24: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/24.jpg)
Slide 24Innovation Network, Inc.
Both Types of Data are Valuable
Qualitative information can provide depth and insight about quantitative data
Some information can only be collected and communicated qualitatively
Both methods require a systematic approach
![Page 25: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/25.jpg)
Slide 25Innovation Network, Inc.
What do your data tell you?
Are there patterns that emerge? Patterns for sub-groups of your population? Patterns for different components of your program?
What questions do the data raise? What is surprising? What stands out?
What are other ways the data should be analyzed? What additional information do you need to collect?
![Page 26: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/26.jpg)
Slide 26Innovation Network, Inc.
Communicating Findings
Who is the information for?How will you tell them what you know?
![Page 27: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/27.jpg)
Slide 27Innovation Network, Inc.
Communicating Findings
“Information that is not effectively shared with others will not be effectively used.”
Source: Building a Successful EvaluationCenter for Substance Abuse Prevention
![Page 28: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/28.jpg)
Slide 28Innovation Network, Inc.
Audience: Who needs the findings, and what do they need?
Staff Board Funders
Partners Other agencies Public
Who are the audiences for your results? Which results?
![Page 29: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/29.jpg)
Slide 29Innovation Network, Inc.
Different ways to communicate
Written report Short summaries Film or videotape
Pictures, displays PowerPoint
presentations Graphs and other visuals
Decide what format is appropriate for
different audiences.
![Page 30: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/30.jpg)
Slide 30Innovation Network, Inc.
Whatever communication strategy you choose: Link the findings to the program’s desired
outcomesInclude the “good, the bad, and the ugly”Be sure you can support your claimsAcknowledge knowledge gaps
![Page 31: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/31.jpg)
Slide 31Innovation Network, Inc.
Continuous Learning Cycle
Logic Model
Reflection/Improvement
Evaluation Planning
Data Collection
![Page 32: The Evaluation Plan. Slide 2 Innovation Network, Inc. Session Purpose To understand how evaluation can be useful To understand how your logic model helps](https://reader035.vdocument.in/reader035/viewer/2022070401/56649f1f5503460f94c36d43/html5/thumbnails/32.jpg)
Slide 32Innovation Network, Inc.
Thanks for Your Participation!
Measure results.
Make informed decisions.
Create lasting change.Innovation Network, Inc.
1625 K Street, NW 11th Floor
Washington, DC 20006(202) 728-0727
Website: www.innonet.org
Veena: Extension 107; [email protected]
Ehren: Extension 109; [email protected]