betty rogers presentation evaluation. 1ppt
DESCRIPTION
Team PresentationTRANSCRIPT
Team PresentationTeam PresentationEvaluationEvaluation
Team MemberBetty RogersSeptember 20, 2010
OverviewOverview
How evaluation techniques can be applied to libraries program and services.
Define evaluation
Purpose of evaluation
Evaluation Method
Evaluation Strategies libraries can use
Evaluation Types
Evaluation planning process
3
What being measured?
What does it indicate?
What is not being measured?
For What Purpose is the Evaluation?
Who are the audiences for the information from the evaluation?
From what sources should the information be collected?
When is the information needed?
“If you don’t know where you are going, how are you gonna’ know when you get there?”
Yogi Berra
DefinitionDefinition
First, evaluation is viewed as a systematic process
Second, evaluation involves collecting data
Third, evaluation is a process for enhancing knowledge anddecision making
Fourth, evaluation is a process of determining the success, impact, results, costs, outcome
SOURCE: Adapted from Russ-Eft and Preskill (2001)
Purpose of EvaluationPurpose of Evaluation
Provide “Useful Feedback
Planning/efficiency
Accountability
Implementation
Institutional strengthening
Evaluation MethodEvaluation Method
Input measurement
Output/Performance Measurement
Impact/Outcomes Assessment
Service Quality
Evaluation MethodsEvaluation Methods
Input MeasurementMoney,
Facilities,
Customers
Clients,
Program staff
Volunteers
External partners
Time
Equipment
Technology
Evaluation MethodsEvaluation Methods
Output measurement
ActivityEvents
Products
Workshops
Trainings
exhibits
Numbered of customer served
Evaluation MethodsImpact/Outcomes MeasuresIndicative of impacts or outcomes from the service, program, or library activity on those who are receiving the service
Outcome / Impactknowledge
Attitudes
Awareness
Opinions
Skills
Behavior
Educational
Environmental quality
Evaluation MethodsEvaluation Methods
Service QualityDifference between what customers expect and their perceptions of the service performance.
-Encompasses the interactive relationship between the library and the people whom it is supposed to serve
Does the service meet organizational or user expectation
Evaluation StrategiesEvaluation Strategies
Major Model Groups
Scientific-experimental models
Management-oriented systems modelsPERT(Program Evaluation and Review Technique
CPM(Critical Path Method)
Qualitative Models
Participant-oriented Models
ROI ModelROI Model
Solution Matrix • Cost-Benefit-Analysis http://www.solutionmatix.com/return-on-investment.html
Type of EvaluationType of Evaluation
Formative
Summative
Robert Stake likened the two stages of evaluation to making soup: when the chef tastes the soup,
its formative; when the diners (or a food critic) taste the soup it is summative
Types of EvaluationTypes of Evaluation
Formative Evaluation
– Needs Assessment
– Evaluability assessment
– Structured conceptualization
– Implementation evaluation
– Process evaluation
Research Methods Knowledge Base
Types of EvaluationTypes of Evaluation
Summative Evaluation
– Outcome evaluations
– Impact evaluation
– Cost-effectiveness and cost-benefit analysis
– Secondary analysis
– Meta-analysis
Research Methods Knowledge Base
Types of Evaluation
Planning Evaluation Cycle
Evaluation PhaseFormulation
Conceptualization
Detailing
Evaluation
implementation
Planning Evaluation Cycle
Planning PhaseFormulation
Conceptualization
Design
Detailing
Analysis
utilization
Planning-Evaluation CyclePlanning-Evaluation Cycle
Research Methods Knowledge Base
20
University of Wisconsin-Extension, Program Development and Evaluation
Typical activity indicators to trackTypical activity indicators to track
Amount of products, services delivered#/type of customers/clients servedTimeliness of service provisionAccessibility and convenience of service
Location; hours of operation; staff availability
Accuracy, adequacy, relevance of assistanceCourteousnessCustomer satisfaction
E.g.:# of clients served# of consultations# of workshops held# of attendees# of referralsQuality of service
Data Collection Techniques
TestsAssessmentsQuestionnairesInterviewsObservationEvaluationUse recordAnalysisContent analysisSurveys
Technique used to Evaluation my Library’s Technique used to Evaluation my Library’s Service/ FVSU H.A. Hunt Memorial LibraryService/ FVSU H.A. Hunt Memorial Library
Formative
Pretest
-Students
Summative
Survey
-faculty
Information Literacy Defined…
The ability to recognize a need for
information and to access, evaluate, and use that information
Conclusion/SummaryConclusion/Summary
In essence, Evaluations are important to library programs and services. The evaluation experience is likely to be more positive and its results are likely to be more useful if you build evaluation in form the start and make it an on-going activity.
ReferenceReference
University of Wisconsin-Extension, Program Development and Evaluation Logic Model. Retrieved. September 1, 2010; http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html
These forms of evaluation are derived from the work of Owen published in Program Evaluation: Forms and Approaches (2006) . Retrieved September 1, 2010; http://www.murdoch.edu.au/teach/carrick_evaluation/purpose_scope.html
Haycock, K. & Sheldon ,B.E (2008) .The Portable MLIS; Insights from the Experts. Westport, CT: Libraries Unlimited.
Research Methods Knowledge Base. Retrieved. September 12, 2010; http://www.socialresearchmethods.net