Evaluating Dissemination of AHRQ CER Products

Darren Mays, PhD, MPH

Department of OncologyGeorgetown University Medical Center

Lombardi Comprehensive Cancer Center Washington, DC

Research to Practice Gap

Bernhardt, Mays, & Kreuter, 2011

How will iADAPT help?

• Poised to make progress– What approaches work? For whom? In what

settings/conditions?• Presents an evaluation challenge– Creative methods/approaches– Diverse populations– Different clinical areas

• Need for a flexible evaluation framework

What is RE-AIM?

• Evaluate public health impact• Focus on dissemination• Barriers include design, setting, approach• Impact assessed on multiple domains• RE-AIM domains:– Reach, Efficacy/Effectiveness, Adoption,

Implementation, Maintenance

Glasgow, Vogt, & Boles, 1999; Glasgow, Lichtenstein, & Marcus, 2003

RE-AIM Domains

• Reach– Did the CER products reach the intended

population(s)?• Participation rate(s), characteristics, baseline “risk”

• Efficacy/effectiveness– What is the impact on intended outcomes?• Clinical outcomes, CER product utilization, occurrence

of harms/unintended consequences

Refer to RE-AIM domains handout; Glasgow et al., 2006

RE-AIM Domains

• Adoption– Did the intended units use the CER product(s)?

• Participation and characteristics of setting(s), delivery agents, barriers to adoption

• Implementation– Were the CER products implemented as intended?

• Adherence, fidelity, technical success

• Maintenance– What is the long-term impact of CER products?

• Long-term efficacy/effectiveness, sustained implementation, barriers to long-term use

Refer to RE-AIM domains handout; Glasgow et al., 2006

Determining Impact

• Quantitatively determining impact• Original application– Reach x Efficacy = Impact

• RE-AIM overall impact– Product of all 5 domains– Requires quantifiable measures

Glasgow, Vogt, Boles, 1999; Glasgow et al. 2006

Application to iADAPT?R E A I M Audience(s) Clinical Area(s)

In PersonCHW OutreachAcad. Detailing

MedMed

MedMed

LowLow

LowLow

LowMed

PatientProvider

DiabetesDiabetes

GroupCER TrainingSchoolGroup Therapy

MedHighLow

LowLowMed

MedMedMed

MedLowLow

MedLowMed

PolicyPatientPatient

MultipleHeart DiseaseDiabetes

eHealthClinic KioskWeb Patient Portal

MedMed

MedMed

LowMed

HighHigh

MedHigh

PatientPatient &Provider

DiabetesDiabetes

Print/MediaTargeted Video Med Med Med Med Med Patient Heart Disease

Adapted from Glasgow et al., 2001

A Closer Look

Clinic Kiosk• R: n = 200 patients

Well-controlled diabetes • E: Small effect size

200 CERSGs (~1/pt.)• A: 75% of clinics• I: 50% completion rate

Technical problems• M: Few support resources

Limited patient interest

Web Portal• R: n = 1,000 patients

Poor diabetes control• E: Medium effect size

2,500 CERSGs (~2.5/pt.)• A: 100% of clinics• I: 75% completion rate

Few technical problems • M: Minimal maintenance

Low-cost to direct patients

Conclusions

• A flexible evaluation framework• Multi-domain evaluation approach• Identify facilitators, barriers, and future

directions• Creative approaches may be needed!

RE-AIM Resources

• NCI DCCPS web site for RE-AIM– http://cancercontrol.cancer.gov/IS/REAIM

• Resources include:– Figures/graphics illustrating key concepts– Checklists and planning tools– Example measures– Publications, presentation

ReferencesBernhardt, JM, Mays, D, & Kreuter, MW. (2011). Dissemination 2.0: Closing

the gap between knowledge and practice with new media. J Health Comm, 16(S1), 32-44

Glasgow, RE, Vogt, TM, & Boles, SM. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. AJPH, 89(9), 1322-1327

Glasgow, RE, et al. (2001). The RE-AIM framework for evaluating interventions: What can it tell us about approaches to chronic illness management. Patient Ed. & Counsel., 44, 119-127.

Glasgow, RE, Lichtenstein, E, & Marcus, AC. (2003). Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. AJPH, 93(8), 1261-1267

Glasgow, RE, et al. (2006). Using RE-AIM metrics to evaluate diabetes self-management support interventions. AJPM, 30(1), 67-73