using re-aim as a tool for program evaluation from research to practice
TRANSCRIPT
![Page 1: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/1.jpg)
Using RE-AIM as a tool for Program Evaluation
From Research to Practice
![Page 2: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/2.jpg)
What is RE-AIM
• RE-AIM is an acronym that consists of five elements, or dimensions, that relate health behavior interventions:– Reach the target population– Efficacy or effectiveness– Adoption by target settings or institutions – Implementation - consistency of delivery of
intervention – Maintenance of intervention effects in individuals
and populations over time
• Commonly used for evaluating efficacy trials (Phase III Research)
![Page 3: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/3.jpg)
Defining Elements
Reach — The absolute number, proportion, and representativeness of individuals who participate in a given program.
Representativeness refers to whether participants have characteristics that reflect the target population's characteristics. For example, if your intent is to increase physical activity in sedentary people between the ages of 35 and 70, you wouldn't test your program on triathletes.
![Page 4: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/4.jpg)
Defining Elements
Efficacy/Effectiveness — The impact of an intervention on important outcomes. This includes potential negative effects, quality of life, and costs.
![Page 5: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/5.jpg)
Defining Elements
Adoption — The absolute number, proportion, and representativeness of settings and staff who are willing to offer a program.
![Page 6: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/6.jpg)
Defining Elements
Implementation — At the setting level, implementation refers to how closely staff members follow the program that the developers provide. This includes consistency of delivery as intended and the time and cost of the program.
![Page 7: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/7.jpg)
Defining Elements
Maintenance — The extent to which a program or policy becomes part of the routine organizational practices and policies. Within the RE-AIM framework, maintenance also applies at the individual level.
At the individual level, maintenance refers to the long-term effects of a program on outcomes after 6 or more months after the most recent intervention contact.
![Page 8: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/8.jpg)
How do elements relate to planning?
• As you design, plan, or evaluate a health behavior intervention, there are questions that you should ask yourself.– Reach: HOW DO I REACH THE TARGETED
POPULATION FOR INTERVENTION?– Efficacy or effectiveness: HOW DO I KNOW
THAT MY INTERVENTION IS EFFECTIVE?– Adoption HOW DO I DEVELOP
ORGANIZATIONAL SUPPORT TO DELIVERY THE INTERVENTION?
![Page 9: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/9.jpg)
How do elements relate to planning?
• As you design, plan, or evaluate a health behavior intervention, there are questions that you should ask yourself.– Implementation: HOW DO I ENSURE THE
INTERVENTION IS DELIVERED PROPRLY?– Maintenance: HOW DO I INCOPROATE THE
INTERVENTION SO IT IS DELIVERED OVER THE LONG TERM?
![Page 10: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/10.jpg)
How is RE-AIM different from other evaluation approaches?
RE-AIM draws upon previous work in several areas including:
Diffusion of innovations, multi-level models, and Precede-Proceed.
The primary ways that it is different is that it (a) is intended specifically to facilitate translation of research to practice, (b) it places equal emphasis on internal and external validity issues and emphasizes representativeness, and (c) it provides specific and standard ways of measuring key factors involved in evaluating potential for public health impact and widespread application.
![Page 11: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/11.jpg)
Is RE-AIM used to design programs, or just to evaluate them?
• It is both. Although used more commonly at present to report results or compare interventions, it is also intended as a planning tool.
![Page 12: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/12.jpg)
An Example
• Physical Activity Promotion in Primary Care: Bridging the Gap Between Research and Practice– Eakin, Brown, Marshall et al. (2004).
![Page 13: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/13.jpg)
Translating and Disseminating Evidence-based Falls Prevention Programs into Community
Li et al. (2008). American Journal of Public Health
An Example of Application
![Page 14: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/14.jpg)
Primary Aim:
(a) To translate an evidence-based Tai Chi exercise fall intervention into a community-based program for implementation with older adults; and
(b) Using the RE-AIM framework (Glasgow et al., 1999), to pilot implement the program with a primary focus on reach, uptake (adoption), and implementation.
![Page 15: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/15.jpg)
Secondary Aim:• To evaluate the effectiveness of the program
with respect to improvements in physical performance measures germane to falling.
Tertiary Aim:• To evaluate program maintenance with
respect to the extent to which older adults would continue to practice Tai Chi beyond the period of the initial evaluation.
![Page 16: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/16.jpg)
Translation
Translate an evidence-based Tai Chi exercise fall intervention into a community-based program for implementation with older adults
• Identify training objectives and elements• Identify end users and dissemination partners• Develop a dissemination package• Expert evaluation • Pilot testing
![Page 17: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/17.jpg)
Program Evaluation
Reach: A total of individuals/providers responded to the program promotion
Effectiveness: defined as change in physical performance outcome measures taken at baseline and again at 12 weeks termination.
Adoption (or uptake): defined as the proportion of local community (senior activity) providers that agreed to participate and implement the program.
![Page 18: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/18.jpg)
Implementation: defined as the extent to which providers' implemented a Tai Chi class to participating older adults and the ability to conduct the various elements of program protocols, including the use of implementation plan, a 2-times weekly program schedule, distribution of program supplements (i.e., videotape and a guidebook), a class attendance rate of 75% or better over the 12-week class period.
![Page 19: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/19.jpg)
Maintenance: defined at both the service provider level and participant level. At the provider’s level, it was defined as providers’ willingness to consider the program to be part of routine organizational provisions (assuming adequate financial resources). At the participant level, it was defined as the extent to which improvements in participants’ physical performance were sustained, and their continued practice of Tai Chi, 8-weeks after completion of the class.
![Page 20: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/20.jpg)
REACH: 87% (by study criteria); 45% by client attendance
Dissemination Outcome
ADOPTION: Six senior activity centers from five communities: 100% adoption
IMPLEMENTATION: 75% completed; >85% class attendance; average 32 min. of home practice
![Page 21: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/21.jpg)
Effectiveness: Improved physical performance and quality-of life measures (a)Functional Reach; (b) Up and Go; (c) Chair Stands, (d) 50-foot speed walk, and SF-12.
Maintenances: Five centers continued; 87% participants continued
Dissemination Outcome
ImpactAdopted by the State of Oregon: being implemented in four counties; three more starting this year
![Page 22: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/22.jpg)
![Page 23: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/23.jpg)
![Page 24: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/24.jpg)
![Page 25: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/25.jpg)
![Page 26: Using RE-AIM as a tool for Program Evaluation From Research to Practice](https://reader035.vdocument.in/reader035/viewer/2022062318/551b7763550346a10a8b4c9d/html5/thumbnails/26.jpg)
Conclusion-Tai Chi
The evidence-based Tai Chi program is practical to disseminate and can be effectively implemented and maintained in community settings