colorado core vipp evaluation · colorado core vipp evaluation lindsey myers, mph injury and...

Colorado Core VIPP Evaluation

Lindsey Myers, MPH Injury and Substance Abuse Prevention Section Manager

Plan to Write a

CDC Evaluation Framework Steps

Engaging Core Stakeholders • Priority areas selected based on CO VIP Strategic Plan

(motor vehicle, falls, prescription drug overdose, suicide)

• Identified stakeholders for each topic area (small groups of 10-12 people)

• Met two times to develop a logic model and select proximal and distal objectives for the CDC BIC template

Focus Area Logic Models

CDC SMART Objective Guidance

CDC SMART Objective Template

Action Planning • Developed action plans for each priority area

• Revisited logic model as needed

• Action plan syncs with logic model

• Developed “At a Glace” version of action plan for partners

TMV Action Plan

Focus Area Logic Models

TMV Action Plan at a Glance

Evaluation Plan Components

Assumptions and Links to Other Documents • Logic Model Exists

• Evaluation Plan is distinct, but inclusive of evaluation that is part of the Action Plan

• Action Plan and Evaluation is within Budget

• Tool development may need to happen during implementation

• Data Analysis Plan

• Data Collection Instruments

1. Abstract/Executive Summary a. General purpose and value for the evaluation (in a few

sentences)

b. Objectives of the evaluation

c. Brief summary of evaluation questions

d. Note the program being evaluated, type of evaluation, and intended audience

e. Utilization of findings (What action will be taken as a result of the evaluation? How much influence do you expect the evaluation to have?)

f. Evaluation Team

2. Background about the Program Being Evaluated a. Narrative description of any context that may affect the

evaluation

b. Logic Model (including short-term, mid-term, and long-term outcomes)

c. Action Plan

d. Evaluability of the program (Clear statement of what the program intends to achieve; activities clearly related to the intended outcomes; SMART objectives)

e. Implementation stage and stage of development of the program (e.g. developmental, program planning, program implementation, program maintenance)

3. Stakeholders and Primary Intended Users

a. Name internal and external stakeholders

b. Evaluation interests of stakeholders (Why do they care? How will they use the findings?)

c. Stakeholder engagement (What and when is the best way to engage stakeholder—planning the evaluation, implementing the evaluation, disseminating results of the evaluation)

d. Disseminating final results to stakeholders (What methods will be used to share results to whom and when?)

Stakeholder Table

4. Background of the Evaluation a. Type of evaluation (Impact, Outcome, Cost-effectiveness,

Process, Program Monitoring, etc)

b. Current level of evidence for the intervention

c. Framework or theory for the evaluation (CDC Evaluation Framework, Participatory/Empowerment, Developmental; Utilization)

d. Overall evaluation design (quantitative, qualitative, mixed-method design)

5. Methodology For each evaluation question include:

a. Indicators

b. Data sources/participants

c. Data collection instruments

d. Data analysis and interpretation plan

e. Known limitations of the methodology or analysis

f. Timeframe and responsibility

Overall CO Core VIPP Logic Model

Core VIPP Evaluation Plan

Full Core VIPP Evaluation Plan

Keep in Off the Shelf • Revisit plan at least twice a year (usually around the

interim progress report and annual progress report for CDC)

• Refine measures

• Evaluation reports

• Continuous quality improvement

Questions?