evaluation planning logic model development - fraser · pdf filebe an academic healthcare...
TRANSCRIPT
11
Evaluation Planning Logic Model Development
© Fraser Health Authority, 2012
The Fraser Health Authority (“FH”) authorizes the use, reproduction and/or modification of this publication for purposes other than commercial redistribution. In consideration for this authorization, the user agrees that any unmodified reproduction of this publication shall retain all copyright and proprietary notices. If the user modifies the content of this publication, all FH copyright notices shall be removed, however FH shall be acknowledged as the author of the source publication.
Reproduction or storage of this publication in any form by any means for the purpose of commercial redistribution is strictly prohibited.
This publication is intended to provide general information only, and should not be relied on as providing specific healthcare, legal or other professional advice. The Fraser Health Authority, and every person involved in the creation of this publication, disclaims any warranty, express or implied, as to its accuracy, completeness or currency, and disclaims all liability in respect of any actions, including the results of any actions, taken or not taken in reliance on the information contained herein.
Susan Chunick, DirectorLisa Kristiansen, Evaluation Specialist
Evaluation & Research Services
October 30, 2012
Department of Evaluation and Research Services‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
Library | Education | Administration | Development
33
Department of Evaluation and Research Services Contact Info – Updated October 2012
Susan Chunick
Director
604.587.4681
Julie Hadden
Research Ethics Coordinator
604.587.4436
Dr. Sonia Singh
Program Medical Director
604.541.5830
Magdalena Swanson
Research & Grant Development Facilitator
604.587.4637
Eva Johnson
Administrative Assistant
604.587.4628
Lisa Kristiansen
Evaluation Specialist
604.587.4445
Camille Viray
Educaiton & Communications Coordinator
604.587.4413
Michelle Purdon
Library Services Manager
604.851.4700 x 646832
Samar Hejazi, PhD
Epidemiologist
604.587.4438
David Whitehurst, PhD
Health Economist
604.587.5902
[email protected] http://research.fraserhealth.ca/
Parveen Sangha,
.8FTE Research Leader, Surrey Campus
44
Strategic Imperatives
Be an academic healthcare organization that improves health outcomes and health services sustainability through teaching, training and research
66
77
88
ObjectivesObjectives
Understand basic components of a logic model Understand basic components of a logic model
Develop a logic model Develop a logic model
Understand how a logic model can be used to Understand how a logic model can be used to assist planning and evaluating assist planning and evaluating programs/interventions programs/interventions
Understand and develop output and outcome Understand and develop output and outcome indicators for your logic model indicators for your logic model
99
AgendaAgenda9:00 9:00 --9:45 How to Develop a Logic Model9:45 How to Develop a Logic Model9:45 9:45 --10:15 Logic model break10:15 Logic model break--out groupsout groups10:15 10:15 --10:30 Report back10:30 Report back
10:30 10:30 --10:45 Break10:45 Break
10:45 10:45 --11:15 Output and Outcome Indicators11:15 Output and Outcome Indicators11:15 11:15 --11:45 Indicator break11:45 Indicator break--out groupsout groups11:45 11:45 --12:00 Report back12:00 Report back
Intro to Evaluation Registry and Program Evaluation Intro to Evaluation Registry and Program Evaluation Workshop Workshop
1010
Group SurveyGroup Survey
Your experience with logic models? Your experience with logic models?
1111
Key ResourceKey Resource
““A Guide to Planning and Conducting A Guide to Planning and Conducting Program EvaluationProgram Evaluation”” 2009. Fraser Health 2009. Fraser Health Department of Evaluation and Research Department of Evaluation and Research Services.Services.
DERS website DERS website http://research.fraserhealth.ca/evaluation/servhttp://research.fraserhealth.ca/evaluation/serv ices_for_program_evaluation/guide_to_planniices_for_program_evaluation/guide_to_planni ng_and_conducting_program_evaluation/ng_and_conducting_program_evaluation/
1212
What are Logic Models Used for ?What are Logic Models Used for ?
Planning and Evaluating Planning and Evaluating Programs/ProjectsPrograms/Projects
A Program/Project/Intervention is: A Program/Project/Intervention is: ““Any group of related, complementary Any group of related, complementary
activities intended to achieve specific activities intended to achieve specific outcomes or results.outcomes or results.””
1313
Programs/InterventionsPrograms/InterventionsDirect service Direct service CommunicationCommunicationEducation and TrainingEducation and TrainingResearch initiativesResearch initiativesSurveillance systemsSurveillance systemsLaboratory diagnosticsLaboratory diagnosticsCommunication campaignsCommunication campaignsInfrastructure Infrastructure –– IT systems, buildingIT systems, buildingAdministrative systems Administrative systems
141414
1515
Program/Project Profile/DescriptionProgram/Project Profile/Description
1.1. Background and ContextBackground and Context
What is the program/project?What is the program/project?
Why was the program/project established? Why was the program/project established?
Related research/experience or Related research/experience or similar/complimentary program/projects?similar/complimentary program/projects?
2.2. Program/Project goals/purposeProgram/Project goals/purpose
LongerLonger--term, how linked to organizational goals?term, how linked to organizational goals?3.3. Target populationTarget population
Who is the program/project intended to serve? Who is the program/project intended to serve? Benefits?Benefits?
1616
INTRO to LOGIC MODELSINTRO to LOGIC MODELS
1717
What Does a Logic Model Look What Does a Logic Model Look Like?Like?
181818
Simplest Form
INPUTS ACTIVITIES OUTPUTS OUTCOMES
1919
Complex Logic Model Complex Logic Model
http://http://waynefoods.wordpress.comwaynefoods.wordpress.com/home/pro/home/pro gramgram--logiclogic--model/model/
2020
212121
A logic model isA logic model is……
A depiction of a program showing what the A depiction of a program showing what the program is planned to do and accomplishprogram is planned to do and accomplish
A series of A series of ““ifif--thenthen”” relationships that, if relationships that, if implemented as intended, lead to the desired implemented as intended, lead to the desired outcomes outcomes
The core of program planning and evaluationThe core of program planning and evaluation
Tool to facilitate transparency Tool to facilitate transparency
Tool to help anticipate unintended Tool to help anticipate unintended consequencesconsequences
222222
Other similar approaches to logic Other similar approaches to logic models: models:
Theory of changeTheory of change
Program actionProgram action
Model of changeModel of change
Conceptual mapConceptual map
Outcome mapOutcome map
Program logicProgram logic
Program theoryProgram theory
2323
How Logic Models Help How Logic Models Help Planners/EvaluatorsPlanners/Evaluators
Better able to address such questions as:Better able to address such questions as:
How is the program/project supposed to work?How is the program/project supposed to work?
Where do the assumptions in the model hold and Where do the assumptions in the model hold and where do they break down?where do they break down?
Where are the gaps or unrealistic assumptions in the Where are the gaps or unrealistic assumptions in the model?model?
Which components are not implemented in practice?Which components are not implemented in practice?
2424
How Logic Models Help How Logic Models Help Planners/EvaluatorsPlanners/Evaluators
Enhancing Critical and Enhancing Critical and Logical ThinkingLogical Thinking
2525
A Series of Logical StepsA Series of Logical Steps
2626
Enhancing Critical Thinking Enhancing Critical Thinking
E.g. E.g. ““This webThis web--based patient selfbased patient self--care care management tool management tool will show how will show how improvements in information sharingimprovements in information sharing of a of a patientpatient’’s chronic disease health s chronic disease health information among members of the information among members of the patientpatient’’s health care team, including the s health care team, including the patient themselves, patient themselves, will improve healthwill improve health..””
272727
What We Invest What We Invest Human, Fiscal, Material, OrganizationalHuman, Fiscal, Material, Organizational ResourcesResources
INPUTSINPUTS
ResearchResearchPlanning Planning
Evaluation Evaluation
Systematic Reviews of EvidenceNeeds Assessment Population Data
ResourcesResources
Funding Funding Labour (paid/unpaid) Labour (paid/unpaid)
PartnersPartnersFacilitiesFacilities
EquipmentEquipmentMaterialsMaterials
TechnologyTechnologyTime Time
Demand (people) Demand (people)
282828
What Do We Do? What Do We Do? And for Whom? And for Whom?
ACTIVITIESACTIVITIESRefer, Treat, Discharge Refer, Treat, Discharge Train, teachTrain, teachDeliver servicesDeliver servicesDevelop products & Develop products & resourcesresourcesNetwork with othersNetwork with othersBuild partnershipsBuild partnershipsAssessAssessFacilitateFacilitateWork with the mediaWork with the media
Customers: Customers:
Patients Patients
ClientsClients
ResidentsResidentsPartners: Partners:
AgenciesAgenciesDecision makersDecision makers
FH ManagementFH ManagementPolicy makersPolicy makers
GovernmentGovernment
2929
What Does a Program Produce?What Does a Program Produce?
OUTPUTSOUTPUTS
# of Patients treated/receive care# of Patients treated/receive care# of Clients taught# of Clients taught# of Workshops delivered# of Workshops delivered# of Communication plans developed# of Communication plans developed# of Decisions made# of Decisions made
3030
OUTCOMESOUTCOMESWhat results for individuals, groups, organization levelsWhat results for individuals, groups, organization levels....……
SHORTSHORTLearningLearning
Changes in Changes in (quantify/qualify)(quantify/qualify) AwarenessAwareness KnowledgeKnowledge AttitudesAttitudes SkillsSkills OpinionOpinion AspirationsAspirations MotivationMotivation
Behavioral Behavioral intentintent
MEDIUMMEDIUMActionAction
Changes in Changes in (quantify/qualify)(quantify/qualify)
Behavior Behavior DecisionDecision--makingmakingPoliciesPoliciesSocial actionSocial action
LONGLONG--TERMTERMConditionsConditions
Changes in Changes in (quantify/qualify)(quantify/qualify)
ConditionsConditionsSocial (wellSocial (well-- being)being)HealthHealthEconomicEconomicCivicCivicEnvironmental Environmental
C H A I N OF O U T C O M E S
3131
Logic Model ExamplesLogic Model Examples
3232Source: University of Wisconsin-Extension, Program Development and Evaluation
Complex FormsComplex FormsGraphic display of boxes and Graphic display of boxes and
arrows; vertical or horizontalarrows; vertical or horizontal Relationships, linkagesRelationships, linkages
Any shape possibleAny shape possible Circular, dynamicCircular, dynamic Cultural adaptations; storyboardsCultural adaptations; storyboards
Level of detail & appropriate levelLevel of detail & appropriate level** SimpleSimple ComplexComplex
Multiple modelsMultiple modelsMultiMulti--level programslevel programsMultiMulti--component programscomponent programs
3333
““ReadingReading”” a Logic Modela Logic Model
Inputs, Resources Activities OutcomesOutputs
Resources tooperate program
- Human- Financial
If you have resources, then you can use them to accomplish planned activities
- Process - Event
If you accomplishactivities,then you will (should) deliver product/service
- Products of activities
If you produce outputs,then participantswill benefit in aparticular way-Health- Knowledge- Skills
3434
An Everyday Logic ModelAn Everyday Logic Model
Getpills
Pills taken
PainGoneYes or No
Return to work
Connect program components by a series of “If… Then” statements (WK Kellogg Foundation, 2004)
$$$
Car
Problem/Need: I have a headache!
Input Activities Output Outcome
3535
An Everyday Logic ModelAn Everyday Logic Model
1.Son goes to Store
2.Selects bread3. Pays for
bread
2 loaves bought
Sandwich made
Mom is happy!
Connect program components by a series of “If… Then” statements (WK Kellogg Foundation, 2004)
Activities Outputs Outcome
3636
Diabetes Clinic Diabetes Clinic -- EducationEducation
Dieticians
Nurses
MDs
Resources
Guidelines
Other?
Inputs
Patient assessmen ts
Referred to classes
Taught self- manage- ment
Case manage- ment (1-1)
Patients followed
Activities Outputs Short- Medium
Long Term
# of Patients assessed
# of Patients referred
# of Patients taught
# of Patients managed 1-1
# of Patients return for annual visit
Patients apply knowledge to self-care
Patients’ blood glucose levels remain stable
Improved overall health
3838
3939
Important to rememberImportant to remember……
Planned program activitiesPlanned program activities
Expected outputs and outcomesExpected outputs and outcomes
Your planned work intended outcomesYour planned work intended outcomes
Outcomes are intended, but not directly attributableOutcomes are intended, but not directly attributable
Based on:Based on:
Assumptions about program components Assumptions about program components
Assumptions about the relationship between Assumptions about the relationship between activities/outputs/outcomesactivities/outputs/outcomes
4040
For planningFor planning
Clarify the problem or issueClarify the problem or issue
What is the need?What is the need?
Who is the target population?Who is the target population?
What is the objective of the program?What is the objective of the program?
Why do you think your program will work? Why do you think your program will work?
Funding decisions are more favorable if you can Funding decisions are more favorable if you can demonstrate clearly how and why the program will demonstrate clearly how and why the program will succeedsucceed
4141
For implementationFor implementation
What is the relationship between the planned activities and intended results?
How does the target population access the program?
Identify indicators of performance, collect and report on information
Communicate with program recipients, stakeholders how the program intended to operate
4242
Looking forward to EvaluationLooking forward to Evaluation
““If you donIf you don’’t know where yout know where you’’re going, re going, how are you gonnahow are you gonna’’ know when you get know when you get
there?there?””
-- Yogi BerraYogi Berra
4343
What else might affect a What else might affect a programprogram’’s/projects/project’’s success?s success?
4444
ContextContext
Environmental factorsEnvironmental factors
Influence program/projectInfluence program/project
Factors not under programFactors not under program’’s/interventions/intervention’’s s control control
Called: Called: •• Intervening variablesIntervening variables•• Confounding variablesConfounding variables•• Stuff that gets in the wayStuff that gets in the way
4545
Logic ModelsLogic Models
Accompanied by text that describes context, need, target population, external environment, future considerations
4646
Group ActivityGroup Activity
Thinking about the goals of your Thinking about the goals of your program/intervention, develop the first 3 program/intervention, develop the first 3 components of a logic model. components of a logic model.
1.1. ContextContext2.2. InputsInputs3.3. ActivitiesActivities
4747
Report BackReport Back
4848
BREAKBREAK
4949
5050
Indicators 101Indicators 101
What is an indicator?What is an indicator?
Succinct Succinct measuresmeasures that indicate progress towards that indicate progress towards achieving the goals of a programachieving the goals of a program
The use of Indicators in the logic model context The use of Indicators in the logic model context ……
Indicators are measures used to determine if Indicators are measures used to determine if components in logic model have been achievedcomponents in logic model have been achieved
When collected and interpreted, indicator data will help When collected and interpreted, indicator data will help determine whether the program is operating as shown determine whether the program is operating as shown in the logic model.in the logic model.
5151
Why Use Indicators?Why Use Indicators?
UnderstandingUnderstanding
how an how an intervention (research or service design) is is working and how it might be improvedworking and how it might be improved
Performance monitoringPerformance monitoring
assess if an employee is performing at the expected levelassess if an employee is performing at the expected level
identify areas for improvementidentify areas for improvement
AccountabilityAccountability
inform an assessment of the outcomes & efficiency of an inform an assessment of the outcomes & efficiency of an initiative/program (summative evaluation)initiative/program (summative evaluation)
5252
Linking Indicators with Linking Indicators with Logic Model ComponentsLogic Model Components
process process indicatorsindicators
output output indicatorsindicators
outcome outcome indicatorsindicators
input input indicatorsindicators
5353
Diabetes Clinic Diabetes Clinic -- EducationEducation
INPUTS
Dieticians
Nurses
MDs
Resources
Guidelines
Other
Patient assessments
Referrals to classes
Self- management education class delivery
Case management (1-1)
Patient follow up
Outputs
Patients assessed
Patients referred
Patients taught
Patients managed 1-1
Patients return for annual visit
Patients apply knowledge to self-care
Patients’ blood glucose levels remain stable
Improved overall health
ACTIVITIESSHORT/MED
TERMOUTCOMES
OUTPUTS LONGOUTCOMES
# pts assessed
# pts referred
# pts at annual visit
% of patients with stable blood glucose
levels
# hospital visits for diabetes orCVD
5454
Considerations When Considerations When Choosing IndicatorsChoosing Indicators
5555
Is the indicator relevant?Is the indicator relevant?
Is the indicator appropriately & clearly defined?Is the indicator appropriately & clearly defined?
Will data be available for the indicator and can it Will data be available for the indicator and can it be systematically collected?be systematically collected?
Are data for the indicator currently being Are data for the indicator currently being collected or can costcollected or can cost--effective instruments be effective instruments be developed?developed?
Will the indicator provide sufficient information Will the indicator provide sufficient information about a condition or result to convince about a condition or result to convince supporters and supporters and skepticsskeptics??
Is the indicator quantitative or qualitative?Is the indicator quantitative or qualitative?
Questions to AskQuestions to Ask
5656
Qualitative IndicatorsQualitative Indicators
Qualitative indicators such as selfQualitative indicators such as self--reported reported descriptions of perceptions are important.descriptions of perceptions are important.
Qualitative indicators may provide Qualitative indicators may provide valuable information about facilitators and valuable information about facilitators and barriers to the success of an intervention.barriers to the success of an intervention.
It is useful to collect both quantitative & It is useful to collect both quantitative & qualitative indicators to provide a broader qualitative indicators to provide a broader context for the interpretation of the context for the interpretation of the program/research findings.program/research findings.
5757
AcronymsAcronyms
5858
Good IndicatorsGood Indicators
SMART
S SpecificM MeasureableA AchievableR RelevantT Time-bound
DUMB
D DoableU UseableM MeasurableB Believable
OROR
5959
Output and Outcome Output and Outcome IndicatorsIndicators
6060
OutputOutput IndicatorsIndicators
Output indicatorsOutput indicators
Measure products or actions Measure products or actions that are directly attributable to that are directly attributable to program operations. program operations.
Usually Usually countcount things produced things produced by the project/program. by the project/program.
6161
OutputOutput IndicatorsIndicators
Output indicators Output indicators are under of the direct influenceare under of the direct influence of the program and typically measure of the program and typically measure ……
efficiencyefficiency of the program output of the program output •• number of products produced or services delivered per cost number of products produced or services delivered per cost
incurred or a period of timeincurred or a period of time
qualityquality of the program outputof the program output•• Percent of products meeting predetermined standardsPercent of products meeting predetermined standards
6262
OutcomeOutcome IndicatorsIndicators
Outcome indicatorsOutcome indicators
Measure Measure effects or effects or resultsresults of a programof a program..
Relate to project/program purpose/goals. Relate to project/program purpose/goals.
Can sometimes be out of the direct sphere of Can sometimes be out of the direct sphere of controlcontrol of the program administrationof the program administration•• Degree of control depends on whether short term or Degree of control depends on whether short term or
longer term longer term ““distancedistance”” of the outcome from the of the outcome from the program sphere of control. Program logic plays a role program sphere of control. Program logic plays a role here.here.
May be quantitative (measures) or qualitative May be quantitative (measures) or qualitative (opinions, experiences).(opinions, experiences).
636363
Outputs versus OutcomesOutputs versus Outcomes
Depends on the program goalsDepends on the program goals
Example (1): Breast Health ClinicExample (1): Breast Health Clinic
Output:Output: Breast health diagnoses are provided Breast health diagnoses are provided within 14 days within 14 days
Outcome:Outcome: Patients receive treatment within Patients receive treatment within ‘‘xx’’ days of diagnosis days of diagnosis (program goal)(program goal)
6464
Outputs versus Outcomes (cont)Outputs versus Outcomes (cont)
Example (2): SeniorsExample (2): Seniors’’ Clinic Clinic
Output:Output: Falls prevention plans prepared for Falls prevention plans prepared for seniors seniors
Short Term/Intermediate Outcome:Short Term/Intermediate Outcome: Falls Falls reducedreduced
Medium/Intermediate Outcome:Medium/Intermediate Outcome: Decreased Decreased Emergency admissions due to fallsEmergency admissions due to falls
Long Term Outcome:Long Term Outcome: Decreased Decreased ““falls falls relatedrelated”” healthcare costshealthcare costs
6565
Selecting Selecting OutcomeOutcome IndicatorsIndicators
Identifying Indicators Identifying Indicators -- Ask:Ask:
How will you know when the outcome has How will you know when the outcome has occurred?occurred?
What does the outcome look like when it What does the outcome look like when it occurs?occurs?
What will you see?What will you see?
Choosing Indicators Choosing Indicators –– Consider:Consider:
Can you observe and measure this Can you observe and measure this indicator?indicator?
Does the indicator tell you that the outcome Does the indicator tell you that the outcome has been achieved?has been achieved?
Would you know if an outcome has been Would you know if an outcome has been achieved without this indicator? achieved without this indicator?
6666
Selecting Outcome IndicatorsSelecting Outcome Indicators
II’’ll know it when I ll know it when I see it rulesee it rule
II’’ll know thatll know that the the <<outcomeoutcome> has > has been reached, been reached, improved, usedimproved, used……..
When I seeWhen I see an an increase in increase in <<indicatorindicator>>
6767Source: University of Toronto Centre for Health Promotion
6868
Group ActivityGroup Activity
Identify one indicator for the following Identify one indicator for the following components of your logic modelcomponents of your logic model
Outputs Outputs
Short/mediumShort/medium--term outcometerm outcome
LongLong--term outcometerm outcome
Describe how they are related (how are Describe how they are related (how are outputs linked with outcomes?)outputs linked with outcomes?)
Describe why the indicators you have Describe why the indicators you have chosen are SMART.chosen are SMART.
6969
Report BackReport Back
7070
What is Evaluation?What is Evaluation?Evaluation is a ProcessEvaluation is a Process
A disciplined approach to organizing thinking A disciplined approach to organizing thinking about any intervention or activity or set of about any intervention or activity or set of activities to maximize its/their probability of activities to maximize its/their probability of success success
A systematic investigation of the merit, worth, or A systematic investigation of the merit, worth, or significance of an object significance of an object
Embraces decisions, processes and outcomes Embraces decisions, processes and outcomes
7171
Evaluation is a ProcessEvaluation is a Process
Evaluation method may differ depending Evaluation method may differ depending on the program/intervention/initiativeon the program/intervention/initiative
No one No one ‘‘rightright’’ way of doing evaluationway of doing evaluation
Don't worry about the plan being perfectDon't worry about the plan being perfect
Evaluation planning is an iterative processEvaluation planning is an iterative process
Evaluation combines theory & practice Evaluation combines theory & practice
7272
What Evaluation is NotWhat Evaluation is Not
Does not assign Does not assign ‘‘blameblame’’
7373
Why Evaluate?Why Evaluate?Appreciative InquiryAppreciative Inquiry
To create To create ““information for planning, designing, information for planning, designing, implementing, and assessing results of our implementing, and assessing results of our efforts to address and solve efforts to address and solve problemsproblems…”…”
~ ~ McDavidMcDavid et al. 2006et al. 2006
7474
When is Evaluation Challenging?When is Evaluation Challenging?
The program has no clear directionThe program has no clear direction
Stakeholders/Decision makers cannot agree on Stakeholders/Decision makers cannot agree on the program the program oror evaluation objectivesevaluation objectives
When there are limited resources/lack of will to When there are limited resources/lack of will to conduct a sound evaluationconduct a sound evaluation
There are no questions about, or lack of There are no questions about, or lack of interest in the fate of, the programinterest in the fate of, the program
Timing to meet external pressures is unrealisticTiming to meet external pressures is unrealistic
Issel, 2004
7575
Standards for Standards for ““Effective EvaluationEffective Evaluation””
UtilityUtility: : Serve the information needs of Serve the information needs of intended user *UTILIZATION FOCUSintended user *UTILIZATION FOCUS
FeasibilityFeasibility: : Be realistic, prudent, diplomatic, Be realistic, prudent, diplomatic, and frugal and frugal
ProprietyPropriety : : Be conducted legally, ethically, and Be conducted legally, ethically, and with due regard for the welfare of those involved with due regard for the welfare of those involved and those affectedand those affected
AccuracyAccuracy: : Reveal and convey technically Reveal and convey technically accurate informationaccurate information
See also: CDC Evaluation Working Group; Stufflebeam, 1999;http://evaluationcanada.ca/site.cgi?s=6&ss=10&_lang=en
7676
Who is the Evaluation For?Who is the Evaluation For?
The intended The intended user(suser(s) may be: ) may be:
Your and your teamYour and your team
Your bossYour boss
Your bossYour boss’’s bosss boss
Your bossYour boss’’s s bossboss’’ss bossboss
7777
3 Steps for an Evaluation3 Steps for an Evaluation
PreparePrepare
Getting people togetherGetting people together
Administrative considerationsAdministrative considerations
PlanPlan
Write program profileWrite program profile
Develop program theory and logic modelDevelop program theory and logic model
Identify intended Identify intended user(suser(s) and ) and use(suse(s))
Identify Evaluation FocusIdentify Evaluation Focus
Develop evaluation questionsDevelop evaluation questions
Develop evaluation methodologyDevelop evaluation methodology
EvaluateEvaluate
Conduct evaluationConduct evaluation
Develop recommendationsDevelop recommendations
Report findingsReport findings
Use findingsUse findings
7878
Key ResourceKey Resource
““A Guide to Planning and Conducting A Guide to Planning and Conducting Program EvaluationProgram Evaluation”” 2009. Fraser Health 2009. Fraser Health Department of Evaluation and Research Department of Evaluation and Research Services.Services.
DERS website DERS website http://research.fraserhealth.ca/evaluation/servhttp://research.fraserhealth.ca/evaluation/serv ices_for_program_evaluation/guide_to_planniices_for_program_evaluation/guide_to_planni ng_and_conducting_program_evaluation/ng_and_conducting_program_evaluation/
7979
IntroducingIntroducing the Evaluation Registrythe Evaluation Registry
8080
Registry PurposeRegistry Purpose
Online tool for organizing, cataloguing and Online tool for organizing, cataloguing and sharing information related to the evaluation of sharing information related to the evaluation of FH programs. FH programs.
Created to support the Research and Academic Created to support the Research and Academic Development strategic imperative. Development strategic imperative.
Helps meet the Canadian Council on Health Helps meet the Canadian Council on Health Services Accreditation standards for FH as Services Accreditation standards for FH as ““a a Learning Organization and Achieving Positive Learning Organization and Achieving Positive Outcomes."Outcomes."
8181
FeaturesFeatures
Uses Uses RefWorksRefWorks and and RefShareRefShare: a web: a web-- based database that stores, organizes, based database that stores, organizes, catalogues and enables sharing of catalogues and enables sharing of information and documents information and documents
Stored in Canada on the Scholars Portal Stored in Canada on the Scholars Portal at the Ontario Council of University at the Ontario Council of University Libraries (OCUL) in Toronto, Ontario Libraries (OCUL) in Toronto, Ontario
8282
Accessing theAccessing the Evaluation RegistryEvaluation Registry
Click on the highlighted link or go to Click on the highlighted link or go to http://http://refworks.scholarsportal.info/refshare/refworks.scholarsportal.info/refshare/ ?site?site=010711166072400000=010711166072400000. .
8383
The EndThe End
