1 national indicators and qualitative studies to measure the activities and outcomes of cdc’s prc...
DESCRIPTION
Outline of Presentation Overview of National PRC Program Evaluation Purpose Indicators Qualitative studies Activities supplementing evaluation Lessons LearnedTRANSCRIPT
![Page 1: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/1.jpg)
1
National Indicators and National Indicators and Qualitative Studies to Measure the Qualitative Studies to Measure the Activities and Outcomes of CDC’s Activities and Outcomes of CDC’s
PRC ProgramPRC ProgramDemia L. Sundra, MPH
Prevention Research Centers Program
AEA 2006, Portland, OR
The findings of this presentation are the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention
![Page 2: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/2.jpg)
PRC National Evaluation Contributors PRC National Evaluation Contributors 2004-Present2004-Present
Collaborative Evaluation Design Team (CEDT)Geri Dino (co-chair), West Virginia University
Cynthia Boddie-Willis, MA Dept. of HealthJosefina Dhungana, UCLA community representative
Katie Barnes, UNC Chapel Hill community representativeKenneth McLeroy, Texas A & M
Thomas Reischl, Univ. of MichiganDelores Pluto, Univ. of S. CarolinaMichelle Kegler, Emory University
David Collins, Morehouse School of Medicine
PRC Program Office at CDCEduardo Simoes (co-chair), Demia Sundra, Jo Anne Grunbaum,
Margaret Kaniewski
ORC MacroNicola Dawkins, Doryn Chervin, Stephanie Kamin, Amee Bhalakia,
Gayle Payne and Jesse Gerwig-Moore
![Page 3: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/3.jpg)
Outline of PresentationOutline of Presentation
Overview of National PRC Program Evaluation Purpose Indicators Qualitative studies Activities supplementing evaluation
Lessons Learned
![Page 4: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/4.jpg)
Levels of PRC EvaluationLevels of PRC Evaluation
Evaluation of PRC researchVs.
Local evaluation of single research center Vs.
Evaluation of national program/network of 33 centers
![Page 5: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/5.jpg)
Project DEFINE Project DEFINE 2004-Present2004-Present
DDelivering elivering EEvidence and vidence and FFindings: indings: IImplementing the mplementing the NNational ational EEvaluationvaluation
Developing a systematic approach that will allow the PRC program to describe and assess program inputs, activities, outputs and outcomes over time
Guided by CDC Framework for Program Evaluation in Public Health1
1Centers for Disease Control and Prevention. Framework for Program Evaluation in Public Health. MMWR 1999;48(No. RR-11). 03/05 from: http://www.cdc.gov/eval/framework.htm
![Page 6: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/6.jpg)
Purposes of the PRC National Purposes of the PRC National EvaluationEvaluation
The national evaluation will accomplish the following purposes:
Primary purpose: To demonstrate national program accountability
Secondary purpose: To facilitate program improvement
![Page 7: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/7.jpg)
Evaluation Approach Evaluation Approach
Quantitative Indicators33 PRCs Qualitative Studies
Document Review – 33 PRCsInterviews – Sample of PRCs
NATIONAL EVALUATION ACTIVITIES
Supplements Helps Identify
Case Studies & Stories from single PRCs
COMMUNICATION/DISSEMINATION ACTIVITIES
![Page 8: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/8.jpg)
Program IndicatorsProgram Indicators
Quantitative Indicators33 PRCs
NATIONAL EVALUATION ACTIVITIES
![Page 9: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/9.jpg)
Program IndicatorsProgram Indicators Addressing national accountability
Looking for the overarching critical components of PRCs that are feasible and meaningful to count and summarize
Balancing burden and highlighting variety
Collecting data through PRC Information System
![Page 10: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/10.jpg)
Development of IndicatorsDevelopment of Indicators CEDT rated indicators: Useful, feasible,
meaningful, relevant Ongoing work with CEDT to refine
Developed corresponding questions for PRC Information System
Sent out for feedback from all PRCs Finalizing changes/beginning Information
System development
![Page 11: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/11.jpg)
Current Status of IndicatorsCurrent Status of Indicators
23 program indicators cover inputs, activities, outputs, and outcomes of a national prevention research program
Research Training Partnerships Dissemination
Reflect variety of program perspectives
Information System is in development
![Page 12: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/12.jpg)
Examples of Research Examples of Research Indicator ConceptsIndicator Concepts
ACTIVITIES Research
projects
INPUTS Project funding Academic institution
support
OUTCOMES Distribution of products Dissemination and
adoption of interventions Policy and environmental
changes
OUTPUTS Effectiveness of
interventions Publications
![Page 13: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/13.jpg)
Sample Reporting – Fictitious Examples of Sample Reporting – Fictitious Examples of Research Indicator DataResearch Indicator Data
Across 33 PRCs, there are 287 projects, totaling $147 million in funding
Funding: 32% CDC, 15% NIH, 52% Other 82% of projects are research
In the past 5 years of research at 33 PRCs, 29 interventions were funded out of core PRC Program dollars:
10 interventions found promising 4 found effective 1 found adoptable
![Page 14: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/14.jpg)
Examples of TrainingExamples of TrainingIndicator ConceptsIndicator Concepts
INPUTS Project funding
OUTPUTS Number of people trained Number of students
working with PRCs
ACTIVITIES Trainings, by
topic, audience, format, duration
![Page 15: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/15.jpg)
Sample Reporting – Fictitious Examples of Sample Reporting – Fictitious Examples of Training Indicator DataTraining Indicator Data
Nationally, 1,484 individuals attended training programs sponsored by PRCs in 2006. The numbers and types of individuals trained include the following:
105 Policy Makers 541 Students 287 Public Health and Health Care Practitioners 311 Community Members 151 Researchers 89 Project Staff
![Page 16: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/16.jpg)
Evaluation Approach Evaluation Approach
Qualitative StudiesDocument Review – 33 PRCsInterviews – Sample of PRCs
NATIONAL EVALUATION ACTIVITIES
![Page 17: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/17.jpg)
Qualitative Study DevelopmentQualitative Study Development Used for concepts
critical to program priority for some stakeholders suitable for qualitative work
Reserved resources from beginning
![Page 18: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/18.jpg)
Four Descriptive Studies Four Descriptive Studies Interaction between PRC researchers and
their communities around core prevention research project
PRCs’ characteristics related to organizational and partnership structures
Variety, goals, and contextual factors of PRCs’ core prevention research
Diversity of PRC training, technical assistance, and mentoring with communities and partners
![Page 19: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/19.jpg)
Data Collection MethodsData Collection Methods Document Review – example topics
Staffing counts and structures Community Committee Guidelines
Interviews – example topics Participation of communities in research Benefits of being in the PRC network
![Page 20: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/20.jpg)
Supplementing Evaluation Data Supplementing Evaluation Data
Supplements Helps Identify
Case Studies & Stories from single PRCs
COMMUNICATION/DISSEMINATION ACTIVITIES
![Page 21: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/21.jpg)
Case Studies and StoriesCase Studies and Storieswww.cdc.gov/prcwww.cdc.gov/prc
![Page 22: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/22.jpg)
Case Study – WVU/Not-On-TobaccoCase Study – WVU/Not-On-Tobacco
![Page 23: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/23.jpg)
Story Story – WVU/Not-On-Tobacco– WVU/Not-On-Tobacco
![Page 24: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/24.jpg)
Evaluation Approach Evaluation Approach
Quantitative Indicators33 PRCs Qualitative Studies
Document Review – 33 PRCsInterviews – Sample of PRCs
NATIONAL EVALUATION ACTIVITIES
Supplements Helps Identify
Case Studies & Stories from single PRCs
COMMUNICATION/DISSEMINATION ACTIVITIES
![Page 25: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/25.jpg)
Lessons LearnedLessons Learned
Expect to use mixed methods
Local feedback was critically important to national evaluation strategy
Advisory group (CEDT) feedback Broader PRC Program feedback
Keep your eye on the prize Accountability of national program helps everyone
![Page 26: 1 National Indicators and Qualitative Studies to Measure the Activities and Outcomes of CDC’s PRC Program Demia L. Sundra, MPH Prevention Research Centers](https://reader036.vdocument.in/reader036/viewer/2022062503/5a4d1ae27f8b9ab0599778d4/html5/thumbnails/26.jpg)
www.cdc.gov/prc
Demia L. Sundra, MPHHealth Education Specialist, PRC Program,
Division of Adult and Community Health,National Center for Chronic Disease Prevention and Health Promotion