evaluation in health promotion
DESCRIPTION
Evaluation in Health Promotion. Presentation by Irving Rootman to SFU Class on Principles and Practices of Health Promotion November 1, 2010. Description of Evaluator. “The evaluator counts the ants at the picnic of progress” (Mohan Singh). Outline. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/1.jpg)
Evaluation in Health Promotion
Presentation by Irving Rootman to SFU Class on Principles and Practices of Health Promotion
November 1, 2010
![Page 2: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/2.jpg)
Description of Evaluator
“The evaluator counts the ants at the picnic of progress” (Mohan Singh)
![Page 3: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/3.jpg)
Outline
What distinguishes evaluation in health promotion from evaluation in other fields?
RE-AIM Framework
![Page 4: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/4.jpg)
Definition of Evaluation in Health Promotion (EWG, 2001)
“Evaluation is about the systematic examination and assessment of features of a programme or other intervention in order to produce knowledge that can be used by stakeholders for a variety of purposes”
![Page 5: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/5.jpg)
Principles for Evaluation of Health Promotion Initiatives (EWG, 2001)
Participation Appropriateness Multiple methods Capacity-building
![Page 6: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/6.jpg)
Conclusion-Recommendation(EWG, 2001) Those who have a
direct interest in a health promotion initiative should have the opportunity to participate in all stages of its planning and evaluation
Encourage the adoption of participatory approaches to evaluation that provide meaningful opportunities for involvement of all those with a direct interest
![Page 7: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/7.jpg)
Types of Participatory Research (Green et al., 1997) Participatory Action
Research (PAR)
Participative Research
Collaborative Inquiry
Participatory Rural Appraisal (PRA)
Appreciative Inquiry
Dialectical Research
Conscientizing Research
Emancipatory Research
Participatory Learning Research
Empowerment Evaluation
![Page 8: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/8.jpg)
Definition of Participatory Action ResearchDefinition of Participatory Action Research
Systematic investigation…
Actively involving people in a co-learning process…
For the purpose of action conducive to health**
--not just involving people more intensively as subjects of research or evaluation
*
(Green et al., 1997)
![Page 9: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/9.jpg)
To generate knowledge about persons without their full participation in deciding how to generate it, is to misrepresent their personhood and to abuse by neglect, their capacity for autonomous intentionality. It is fundamentally unethical.
Heron, J. (1996) Co-operative Inquiry, London, Sage
![Page 10: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/10.jpg)
Conclusion-Recommendations(EWG, 2001) The use of randomized
control trials to evaluate health promotion initiatives is in most cases inappropriate, misleading and unnecessarily expensive
Support the use of multiple methods
Support further research into the development of appropriate approaches to evaluating health promotion initiatives
![Page 11: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/11.jpg)
Additional Conclusions about Evaluation in Health Promotion (EWG, 2001)
1. Evolving
2. Valuable
3. Not enough
4. Many planning models
5. Multi-disciplinary
6. Evaluators play many roles
7. Theory is essential
![Page 12: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/12.jpg)
Dimensions of RE-AIM Model (Glasgow, 2004) Reach: What % of potentially eligible participants will take part and how
representative are they? (Individual Level)
Effectiveness: What impact did the intervention have? (Individual Level)
Adoption: What % of settings and intervention agents will participate and how representative are they? (Setting Level)
Implementation: To what extent are the intervention components delivered as intended? (Setting/Staff Level)
Maintenance: What are the long-term effects (Individual Level); To what extent are intervention components continued or institutionalized? (Setting Level)
(Glasgow& Linnan, 2008)
![Page 13: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/13.jpg)
Example of Use of RE-AIM Model: Diabetes Self-Management Projects Both used SCT and Social Ecological Model,
similar measures, adult P.C. diabetes patients, similar recruitment methods
Differed in intensity, implementation and other features
![Page 14: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/14.jpg)
Example of Use of RE-AIM Model (Cont.)Program 1: In-Office Self-Management Touch-screen computer program prior to office visit
Followed by R.N. review of action plan
Follow-up phone call
Took 30-45 minutes; 5minutes for R.N.
Used regular staff in 30 clinics in Colorado
![Page 15: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/15.jpg)
Example of Use of RE-AIM Model (Cont.)
Program 2: Linked Health Coach S.M. 2 two- hour visits to health educator
Patient worked through problem-solving , computer-administered program to produce action plan
![Page 16: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/16.jpg)
Example of Use of RE-AIM Model (Cont.)
Comparison of Programs: Program 1 produced better “reach” Program #2 produced slightly less
improvement on P.A. and H.E. but larger QOL Change
Largest difference in “adoption” with 20% of MD’s willing to participate in #2 v.s. 6% in #1
Both produced excellent “implementation”
![Page 17: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/17.jpg)
Example of Use of RE-AIM Model (Cont.)Comparison (Cont.): #1 cost $222, #2 $547 per participant Both improved self-efficacy, perceived support in
comparison to controls Few relations between mediators and outcomes
Conclusions: Likely more health plans would adopt #1 because of
lower cost and greater cost-effectiveness Lower adoption rates of #1 by MD’s needs to be
addressed
![Page 18: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/18.jpg)
“Reach” Challenges and Remedies (Glasgow and Linnan, 2008)
Challenge: Sampling
Remedies: Population-Based
Recruitment
Over-recruitment
Report on representativeness
Limit exclusion criteria
![Page 19: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/19.jpg)
“Effectiveness” Challenges and Remedies (Glasgow and Linnan, 2008)
Challenges: Understanding
Outcomes Knowledge of mediators
Conflicting/ambiguous results
Inadequate control conditions
Remedies: Assess broad set of
outcomes Include mediator
measures Sub-group analyses
Design control condition to fit question
![Page 20: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/20.jpg)
“Adoption” Challenges and Remedies (Glasgow and Linnan, 2004)
Challenges: Program only studied in
optimal conditions
Program not adopted
Remedies: Involve potential
adoptees
Approach settings early
![Page 21: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/21.jpg)
“Implementation” Challenges and Remedies (Glasgow and Linnan, 2004)
Challenges: Protocols not delivered as
intended
Unable to answer questions about cost, time or staff requirements
Deciding if program adaptation is good or bad
Remedies: Assess nature of treatment Involve practitioners in
program design
Vary staff characteristics, and evaluate staff impact and costs
Specify critical theoretical components and elements that can be adapted
![Page 22: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/22.jpg)
“Maintenance” Challenges and Remedies (Glasgow and Linnan, 2004)
Challenges: Program effects not
maintained
Attrition of settings, delivery staff and participants
Remedies: Include maintenance
phase in plan
Plan for institutionalization and sustainability
Take steps to evaluate and report on attrition
![Page 23: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/23.jpg)
RE-AIM: Strengths/Weaknesses
Strengths: Comprehensive Focus on population impact Multi-level Relatively easy to use Useful for program and policy development Builds on other theories at different levels
Limitations: Not much research using model Not necessarily “participatory”
![Page 24: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/24.jpg)
RE-AIM: Implications for Aging
Has been used in studies of older adults in relation to: Physical Activity Chronic disease management Nutrition Heart Disease prevention
![Page 25: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/25.jpg)
Example: Home-based Exercise
Purpose: Evaluate home-based exercise program for older adults
Methods: 105 frail homebound older adults recruited from 10 Faith in Action Sites; Volunteer trainers assisted subjects; Surveys at baseline and after 4 months in program; RE-AIM used as conceptual framework; focused on “adoption” and “implementation”
Findings: Participants exercised average of 2.2. times per week; improved social functioning
Conclusion: Evidence for A and I components of modelEtkin, C.D. et al., 2006
![Page 26: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/26.jpg)
References Etkin, E.D., et al. (2006). Feasibility of Implementing the Strong
for Life Program in Community Settings, The Gerontologist, 46: 284-292.
Glasgow, R.E. and Linnon, L.A. (2008). Evaluation of Theory-Based Interventions. In Glanz, et al., Health Behavior and Health Education: Theory, Research and Practice, Forth Edition, San Francisco: Jossey-Bass.
Green et al., Participatory Research…Ottawa: Royal Society of Canada, 1997. www.lgreen.net/guidelines.html
Rootman, I., Goodstadt, M., Hyndman, B., McQueen, D., Potvin, L., Springett, J., Ziglio, E. (Eds.),(2001). Evaluation in Health Promotion: Principles and Perspectives, Copenhagen: European Regional Office of the World Health Organization.
![Page 27: Evaluation in Health Promotion](https://reader033.vdocument.in/reader033/viewer/2022051218/56815751550346895dc4f466/html5/thumbnails/27.jpg)