betty bekemeier follow-up ilinc session may 31, 2007 & making sense of what happens as a result...
TRANSCRIPT
Betty Bekemeier
Follow-up iLinc session
May 31, 2007
&Making sense of what happens as a result of our efforts
Logic Models
Outcome Measurement
Ways to Think about Outcome Measurement
Performance measurement
Evaluation
Logic models
. . .these are not mutually exclusive!
Why Do We Need to Do This?
To provide accountability.
To improve program quality.
To make decisions about resource allocation.
To help programs market themselves.
To understand our contribution to our community’s health.
How Do We Do This?
Set a goal.
Define activities.
Identify inputs or resources.
Define outcomes.
Develop outcome measures or indicators.
Build consensus around these measures.
Activities Outputs Outcomes GoalsResources
Logic Models: “So That” Chains
Resources Outputs Outcomes GoalActivities
So That So That So That So That
Process
Developing an Outcome Evaluation Logic Model
Program inputs. Elements or ingredients that constitute the program.
RESOURCES
Process
Developing an Outcome Evaluation Logic Model
Program inputs. Elements or ingredients that constitute the program.
RESOURCES
Methods for providing the program. Specific processes or events undertaken.
ACTIVITIES
Process
Developing an Outcome Evaluation Logic Model
Program inputs. Elements or ingredients that constitute the program.
RESOURCES
Methods for providing the program. Specific processes or events undertaken.
Units of service or product units. How many, over what duration?
ACTIVITIES OUTPUTS
Process
Developing an Outcome Evaluation Logic Model
Program inputs. Elements or ingredients that constitute the program.
RESOURCES
Methods for providing the program. Specific processes or events undertaken.
Units of service or product units. How many, over what duration?
Short, intermediate, or longer-term changes anticipated in participants’ lives or in organizational or community conditions.
OutcomesACTIVITIES OUTPUTS OUTCOMES
Process
Developing an Outcome Evaluation Logic Model
Program inputs. Elements or ingredients that constitute the program.
RESOURCES
Methods for providing the program. Specific processes or events undertaken.
Units of service or product units. How many, over what duration?
Short, intermediate, or longer-term changes anticipated in participants’ lives or in organizational or community conditions.
Ultimate impact(s) expected to occur, usually beyond what one program alone can achieve.
OutcomesACTIVITIES OUTPUTS GOALSOUTCOMES
Process
Developing an Outcome Evaluation Logic Model
Program inputs. Elements or ingredients that constitute the program.
RESOURCES
Methods for providing the program. Specific processes or events undertaken.
Units of service or product units. How many, over what duration?
Short, intermediate, or longer-term changes anticipated in participants’ lives or in organizational or community conditions.
Ultimate impact(s) expected to occur, usually beyond what one program alone can achieve.
OutcomesACTIVITIES OUTPUTS GOALSOUTCOMES
Why a Logic Model?
A logic model: Clarifies each program element. Shows the relationship of
resources and activities to expected outcomes.
Summarizes how program parts relate to the whole.
Makes explicit the theory of how the program works.
Identifies categories to measure in the program evaluation.
Material was used and adapted from: Organizational Research Services, Inc. The Washington State Public Health Nursing
Directors
Sponsors are: NWCPHP The Washington State Department of Health The federal Health Resources and Services
Administration (HRSA)
Acknowledgements
100% of CH program managers use logic model outcome measures to annually evaluate program effectiveness by Dec. 2008
100% of CH Programs have identified at least two outcome measures that demonstrate program benefits to the community by Dec. 2008
90% of all CH staff receiving logic model training can define the components of the model and describe at least two benefits of using this model when surveyed 6 months post training
Number of CH staff trained in logic model development
Number of program staff meetings allocating time for work on logic models
Number of logic models developed
Number of contacts with NWCPHP for technical assistance with logic model development
Number of outcome measure consultations with KCHD Epidemiology staff
7 CH staff trained in Logic Model 101
Additional logic model training is available from NWCPHP (iLinc) and other sources
Technical assistance in logic model development is available from NWCPHP
KCHD Epidemiology staff are available for consultation in developing outcome measures
Example: Kitsap County Health District
Integrating the Use of Logic Models into all KCHD Community Health Programs
Resources Activities
Outputs Outcomes GoalTrain all CH staff in logic models
Allocate time at monthly staff meetings for logic model development, critique and mentoring
Use program logic model as annual work plan
Incorporate the use of logic models into performance expectations for CH Program Managers
All CH Programs use outcome measures as an evaluation tool to guide practice
CME breakfast with peer educator at the staff meeting for OB and family practice MDs
Contact partners to initiate a new standing order at Snohomish Co. hospitals for post-partum Tdap
Develop education for providers and patients:
BrochuresFlyers
Worked with IACW Adult Committee in developing chart for pregnancy and immunizations
Example: Snohomish Health District Logic Model
Description: Prevention of Pertussis in Newborns
Resources Activities
Outputs Outcomes Goals
One breakfast meeting with 22 providers attending
Education developed:
1 brochure1 letter
5 flyers for different ethnic groups
Contacted medical directors at 5 hospitals to discuss the initiation of standing orders for Tdap
Mailed 2000 pregnancy and immunization charts to OB and family practice MDs in WA
Providers educated about the importance of Tdap in post-partum women
Patients educated through flyers of the need to get a Tdap vaccine
Standing orders in place in all 5 hospitals that deliver babies in Snohomish Co.
Stronger partnership with drug representatives in promoting education of physicians
Increased request from practitioners for pregnancy and immunization charts
Reduction of pertussis in un- or under-immunized newborns and infants in Snohomish Co.
Staff: D and Gayle
Journal articles: research and statistics
Existing dept meeting for OB and family practice MDs in Snohomish Co.
Knowledge of standing order process:
Hospital medical directors
Drug reps:
Speaker
IACW partnership
Session 2: Live iLinc presentation on June 6, 2007, 1:00–2:30 am Same format as this session: further interactive
support, working through participants’ logic models in process.
Online module 30-minute narrated, web-based module
• http://www.nwcphp.org/outcome
Additional Opportunities