evaluation of training and education activities
DESCRIPTION
Evaluation of Training and Education Activities. Objectives. By the end of this presentation, participants will be able to List reasons why evaluation is needed Describe different types of evaluation Discuss how to use different types of evaluation for training and education activities. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/1.jpg)
Evaluation of Training and Education Activities
![Page 2: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/2.jpg)
2
ObjectivesBy the end of this presentation, participants will be able to
List reasons why evaluation is needed Describe different types of evaluation Discuss how to use different types of
evaluation for training and education activities
![Page 3: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/3.jpg)
Evaluation (1)
As trainers, what do we want to evaluate?
3
![Page 4: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/4.jpg)
Evaluation (2) What have we accomplished?
What was the training? Where? How many people were trained? Cost?
How well have we done? Did the trainings go as planned? Did the participants like the training?
Was the training relevant? What were the strengths and
weaknesses?4
![Page 5: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/5.jpg)
Evaluation (3) How effective is the training?
To what extent did the training achieve the objectives?
Did the participants increase their knowledge and skills?
Did they change their job performance? Did health status improve?
How can we improve on it? Are the training objectives appropriate,
given the available resources?5
![Page 6: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/6.jpg)
Systematic Health Education Process
6
Development
Pilot-testing(Formative Evaluation)
Implementation(Process Evaluation)
Assess Effectiveness(Outcome and
Impact Evaluation) Evaluation
(Formative Research)
NeedsAssessment
![Page 7: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/7.jpg)
7
When to Begin Evaluation?Consider evaluation up front (in planning phase of training project)
Allocate time Allocate resources Evaluation actually occurs at every
stage of development Determine the type of evaluation you
need
![Page 8: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/8.jpg)
8
Levels of Evaluation Formative (before)
Identifies strengths and weaknesses of training/product before implementation
Process (during) Describes the implementation of the
training or distribution/use of the product
Summative/Outcome/Impact (after) Assesses the short-term and long-term
effects of the training/product
![Page 9: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/9.jpg)
Four Types of Evaluation
9
Impact
Outcome
Process
Formative
Programimplementation
ends here.
Programimplementation
occurs here.
SummativeEvaluation
![Page 10: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/10.jpg)
10
Goals and Objectives Goal
Describes the overall mission/purpose of the program Broad general statement that clearly states the
problem that will be addressed
Objectives State the expected result Specific, measurable, attainable, realistic, time-
framed Partial accomplishment of a goal
Activities Specific actions or procedures expected to occur to
achieve objectives
![Page 11: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/11.jpg)
Formative Evaluation
11
Impact
Outcome
Process
Formative
Programimplementation
ends here.
Programimplementation
occurs here.
SummativeEvaluation
![Page 12: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/12.jpg)
12
Formative Evaluation(Pilot Testing)
Conducted with the target audience Answers questions about whether
trainings/products are Understandable, relevant, attractive,
credible, acceptable, effective Identifies strong and weak points
Identifies confusing terms or concepts Allows for necessary revision prior to
implementation Prevents production of ineffective
products
![Page 13: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/13.jpg)
Formative Evaluation Methods Focus groups
With various types of participants (e.g., experienced, non-experienced)
Interviews (key informant/audience) Informal discussion Open-ended questions
Understanding Appeal Utility
Test the curriculum in the “real” environment
Readability testing (literacy and comprehension) 13
![Page 14: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/14.jpg)
14
Formative Evaluation Measures
Ease of use or intuitiveness Readability
Reading level Tone Fonts, colors, spacing
Comprehension Terminology and illustrations Concepts
![Page 15: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/15.jpg)
TB:The Hollywood
Movie
![Page 16: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/16.jpg)
Formative Evaluation Screen tests Movie screenings Focus groups
TB
![Page 17: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/17.jpg)
Process Evaluation
17
Impact
Outcome
Process
Formative
Programimplementation
ends here.
Programimplementation
occurs here.
SummativeEvaluation
![Page 18: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/18.jpg)
18
Process Evaluation(Monitoring of Training/Curriculum)
Examines the procedures and tasks involved in implementing a training/product
Assesses if the training/product is working as planned
Answers questions such as: Are the activities being implemented as
planned? Is the material being used as intended? How many participants attended the program? Were the participants satisfied with the
educational materials?
![Page 19: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/19.jpg)
19
Process Evaluation Measures Product inventory
Hard copies distributed Web pages viewed or downloaded
Attendance Checklist and timetable review Budget, $$$$ spent Audience feedback on quality of
program/material during implementation (participant reactions)
![Page 20: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/20.jpg)
Participant Reactions Do audience/participants think that:
Objectives are relevant to participants' work
Course content is realistic Training methods are suitable Facilitators are effective Training environment (classrooms, meals,
accommodation) contributes to learning
20
![Page 21: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/21.jpg)
Participant Reactions Methods
Written feedback – participants write down their reactions Ensures that all participants provide comments,
not just the outspoken ones Both open- and closed-ended questions are asked
Oral feedback and observations Observations by facilitators Facilitators meet at the end of each day to
discuss their opinions and participant comments
21
![Page 22: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/22.jpg)
Participant Reaction Form
22
![Page 23: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/23.jpg)
Process Evaluation Ticket sales Critical reviews Rotten tomatoes
TB
![Page 24: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/24.jpg)
Outcome Evaluation
24
Impact
Outcome
Process
Formative
Programimplementation
ends here.
Programimplementation
occurs here.
SummativeEvaluation
![Page 25: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/25.jpg)
25
Outcome and Impact Evaluation
Answers the questions… What have you accomplished? How well have you done it? How much have you done? How effective have you been?
![Page 26: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/26.jpg)
26
Outcome Evaluation Based on learning objectives Evaluation of learning & performance of
skills Assesses immediate or short-term effects
End of course or immediately after distribution Helps draw conclusions about effectiveness Answers questions such as:
Were learning objectives achieved? What knowledge and skills were gained as
a result of the training/product?
![Page 27: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/27.jpg)
Outcome EvaluationEffectiveness of Self-Study Modules
on Tuberculosis Knowledge Mean Scores
Pre-test Post-testPrint 74% 91%Satellite 81% 92%Web 76% 92%
(p<.01)
27
![Page 28: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/28.jpg)
Impact Evaluation
28
Impact
Outcome
Process
Formative
Programimplementation
ends here.
Programimplementation
occurs here.
SummativeEvaluation
![Page 29: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/29.jpg)
29
Impact Evaluation Gathers information about the long-range
results of the training/product and improvements in health status 3 months, 6 months, one year following
implementation Answers questions such as:
Did the user’s job performance improve? What are the changes in job practice of user or
participant? What changes were implemented in the program?
Costly, time consuming, but important
![Page 30: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/30.jpg)
30
Should You Measure Impact?
Ask yourself the following questions: Do you have the time and resources? Do you have support from management
and partners? Are you able to reach the audience at this
later date? What will motivate the audience to
participate?
![Page 31: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/31.jpg)
Impact Evaluation Methods
31
Methods Surveys Key-informant interviews Direct observation/performance checklists Document/record reviews/treatment registers Reviewing of program indicators
Timeline Depends on the goal of the training;
however, generally 1 to 6 months after training
![Page 32: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/32.jpg)
Summative Evaluation Domestic and international
box office revenues DVD sales Franchise opportunities
TB
![Page 33: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/33.jpg)
Evaluation Reports
![Page 34: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/34.jpg)
Evaluation Reports Justify program, course, or material to
stakeholders Allow for identification of areas for
improvement Provide evidence for additional needs Share lessons learned
![Page 35: Evaluation of Training and Education Activities](https://reader034.vdocument.in/reader034/viewer/2022051421/5681677a550346895ddc7da0/html5/thumbnails/35.jpg)
Most Importantly…
Plan for evaluation at the beginning and conduct
throughout development process