name of project: oral health evaluation...focus. these priority areas were refined to establish...

FY 2017 MDHHS Deliverable Submission Form

Name of Project: Oral Health Evaluation

Document Name: Evaluation Plan for the Year 1 Implementation of the 2020 Michigan State Oral Health Plan

Project Number: G-64928-11

Program Name: Center for Child and Family Health

Director or

Coordinator Name:

Diane Wing, Project Coordinator

Submitted By: Diane Wing

Type of Deliverable: (Please check one)

Final report

Specialized report

Publication

Articles published

Brochures

Software application

Web screen shot

No report, publication, or study

Deliverable pending

OTHER material

If OTHER, please describe:

Evaluation Plan

0

Prepared by:

Center for Child and Family Health

Michigan Public Health Institute

Evaluation Plan for

Year 1 Implementation of the

2020 Michigan State Oral Health Plan

1

Contents Introduction ................................................................................................................................................ 2

Plan Description ......................................................................................................................................... 2

Evaluation ................................................................................................................................................... 3

Purpose .................................................................................................................................................... 3

Evaluation Team .................................................................................................................................... 3

Evaluation Questions ............................................................................................................................ 3

Evaluation Design .................................................................................................................................. 4

Data Collection ........................................................................................................................................... 5

Data Analysis .............................................................................................................................................. 5

Data Interpretation .................................................................................................................................... 5

Data Dissemination .................................................................................................................................... 6

Appendix A: Logic Model for the 2020 Michigan State Oral Health Plan ........................................ 7

Appendix B: Key Partner Survey ............................................................................................................ 8

2

Introduction The Michigan Department of Health and Human Services (MDHHS) was awarded a five-year

grant (2013/2018) from the Center for Disease Control and Prevention (CDC). The purpose of

the funding was to support Michigan in strengthening and maintaining a strong oral health

program. As recommended by the CDC, one of the core aspects of a strong oral health program

is the development of an oral health plan, as it provides a framework for action to

collaboratively accomplish the goals and objectives that will work to improve oral health in

Michigan.

The MDHHS contracted with the Center for Child and Family Health (CCFH) at the Michigan

Public Health Institute (MPHI) to implement a one-year evaluation (2016/2017) of stakeholder

implementation of the State Oral Health Plan. The purpose of the evaluation is to assess

stakeholders’ engagement in implementing strategies within the plan as well as to monitor

progress toward goal achievement within the first year of plan dissemination. A participatory

approach will be used in the evaluation, as the evaluators will work closely with project

partners and other stakeholders to ensure that the evaluation objectives and methods align with

the project goals and that useful results are produced.

Plan Description The MDHHS Oral Health Program and the Michigan Oral Health Coalition (MOHC) worked

together to develop a five-year state oral health plan with the ultimate goal of achieving optimal

oral health among all Michiganders. Key oral health partners and stakeholders across the state

worked together to develop the plan; establishing goals, implementation steps, and a

monitoring plan. The plan additionally serves as a tool for identifying and enlisting

collaborators and partners needed to ensure its success. The process of developing the plan

began in April 2015 and continued through to December 2015 when the final plan was

completed.

Through an action-planning process, participating stakeholders and key oral health partners

developed a practical vision for the plan and identified three key priority areas in which to

focus. These priority areas were refined to establish clear goals, create SMART objectives, and

to identify activities and strategies to focus the work of stakeholders and key oral health

partners throughout the state. Progress towards these goals and objectives will be reviewed on

an annual basis and shared with key oral health partners. The intended outcomes of the plan

are to increase the number of stakeholders that work to enhance professional integration

between providers across the lifespan, increase knowledge and awareness of the importance of

oral health to overall health, and to increase access to oral health care among underserved

populations with a long-term outcome of achieving optimal oral health for all Michiganders.

The logic model for the State Oral Health Plan is presented in Appendix A.

3

Evaluation Purpose

This evaluation will focus on the implementation of the 2020 Michigan State Oral Health Plan

by assessing stakeholder engagement in carrying out the plan within the first year since plan

dissemination. In addition, progress toward goal achievement will be monitored for select

strategies to establish baseline measures for plan monitoring. The information gathered

through this evaluation will be used to inform the MDHHS Oral Health Program and key

partners of stakeholder engagement and of progress achieved within year one of the plan and to

identify any potential areas for improvement.

Evaluation Team

The evaluation team consists of staff from the MDHHS Oral Health Program (OHP) and

program staff from the Center for Child and Family Health (CCFH) at the Michigan Public

Health Institute (MPHI). Staff from the OHP will provide feedback on the evaluation design

and planning and provide input on the interpretation of evaluation findings. Staff from MPHI

will lead the evaluation and ensure that evaluation activities are conducted as planned. Table 1

presents the roles and responsibilities of the evaluation team.

Table 1: Roles and Responsibilities of Evaluation Team Members

Individual Role Responsibilities

MDHHS OHP Staff:

Chris Farrell, Susan Deming,

Sandy Sutton, Jill Moore, Beth

Anderson

Stakeholder/Funder

Provide feedback on evaluation design

and planning and provide input on

interpretation of findings

Coordinate data collection with key oral

health partners

MPHI CCFH staff:

Crystal Tyler, Diane Wing,

Morgan Richardson

Evaluator, Data Analysis

Oversight of all evaluation activities to

ensure the evaluation is conducted as

planned

Gather and analyze data

Compile results and write final report

Evaluation Questions

As the 2020 Michigan State Oral Health Plan is in its first year, the evaluation will consist of

process evaluation questions addressing stakeholder engagement and short-term outcome

questions to establish baseline measures for plan monitoring.

Overarching evaluation question: In the first year since plan dissemination, how engaged are

stakeholders and key oral health partners in implementing the 2020 Michigan State Oral Health

Plan?

In 2016, how many key oral health partners reported reading the 2020 Michigan State

Oral Health Plan? If they didn’t, why not?

4

In 2016, to what extent did key oral health partners feel their organizations’ goals

aligned with the goals in the 2020 Michigan State Oral Health Plan?

In 2016, did key oral health partners use the 2020 Michigan State Oral Health Plan as a

source to make decisions or develop plans for action?

In 2016, how was the 2020 Michigan State Oral Health Plan used to make decisions or

develop plans for action?

Short-term outcome questions:

Professional Integration

How many oral health partners had a formal relationship with providers from different

healthcare disciplines?

Has the number of formal relationships increased since the release of the plan?

How many key partners promoted the viewing of one or more Smiles for Life online

training modules?

Has the number of key partners who promoted viewing of the Smiles for Life modules

increased since the release of the plan?

How many key partners viewed one or more Smiles for Life online training modules?

Has the number of key partners who viewed one or more Smiles for Life modules


Health Literacy

How many key partners use social media to educate providers/consumers on oral

health?

Have key partners use of social media to educate providers or consumers on oral health


How many key partners shared social media messaging from the MDHHS OHP and /or

MOHC?

How many key partners worked to educate pregnant women on oral health preventive

measures?

How many key partners worked to create and support county advocacy networks across

the state of Michigan?

How many key partners collaborated with Michigan’s public school districts to increase

oral health awareness activities?

Evaluation Design

A non-experimental study design will be used for the evaluation. Data gathered will be used to

assess stakeholder engagement and monitor the implementation of activities outlined in the

2020 Michigan State Oral Health Plan.

5

Data Collection A key partner survey will be used to assess stakeholder engagement and to monitor progress

toward goal achievement for select strategies outlined in the plan. The OHP Director will email

the electronic survey to key oral health partners via Survey Monkey in February of 2017. The

purpose of the survey will be to gain a better understanding of how key partners used the State

Oral Health Plan in the past year (2016), whether or not it was used to develop plans for action,

and what parts of the plan, if any, key partners have implemented. To establish baseline

measurements for plan monitoring, survey participants will be asked whether or not they

implemented specific strategies. Members from the evaluation team will export the data from

Survey Monkey into a Statistical Package for the Social Sciences (SPSS) database. The survey

tool is attached in Appendix B.

Data Analysis Both quantitative and qualitative methods will be used in data analysis. The Statistical Package

for the Social Sciences (SPSS) will be used to conduct analyses on quantitative data. Descriptive

statistics, including frequency distribution and central tendencies, will be used to report

findings on stakeholder engagement and short-term outcomes. Responses to open-ended

questions will be assessed to identify patterns or trends.

Data Interpretation The MPHI evaluation team with the OHP Program Director and staff will meet to interpret the

evaluation findings. From this meeting, evaluation findings will be finalized and

recommendations will be made accordingly. Table 2 presents the timeline for evaluation

activities.

Table 2: Timeline for Evaluation Activities

Dec Jan Feb Mar Apr

2016

Evaluation Planning X

Survey Tool Development X

2017

Finalize Survey Tool X

Email survey to key partners X

Analysis of key partner survey data X

Develop success story draft X X

Final success story/dissemination X

6

Data Dissemination The evaluation team at MPHI will provide monthly updates on evaluation activities to the OHP

Program Director and staff through monthly conference calls. A success story formatted report

will be developed and delivered to the OHP in April, 2017. The success story will be

electronically delivered to key oral health partners and will be made available on the MDHHS

OHP website.

7

Appendix A: Logic Model for the 2020 Michigan State Oral Health Plan

Inputs Activities Outputs Long-Term

Outcomes

CDC

MDHHS Oral

Health Program

(OHP)

Broad array of

stakeholders

Data on oral

health indicators

in Michigan

Stakeholders are

surveyed to

identify cursory

focus areas

Stakeholder

meeting is

convened to

initiate strategic

planning

Oral health burden

is assessed

Oral health mission

is established

Potential goals and

objectives are

established

Plan

implementation is

evaluated and

progress towards

goals achievement

is monitored

Stakeholders

review draft of

plan and submit

feedback to

MDHHS OHP

State Oral Health

Plan, supported

by stakeholders, is

complete

State Oral Health

Plan is presented

and disseminated

(e.g. Michigan

Oral Health

Coalition Spring

meeting, MDHHS

website)

Intermediate

Outcomes

Short-Term

Outcomes

Increase in the

number of

stakeholders that

work to:

Enhance

professional

integration

between

providers across

lifespan

Increase

knowledge and

awareness of

importance of

oral health to

overall health

Increase access

to oral health

care for

underserved

and/or hard to

reach

populations

Enhanced

professional

integration

between providers

across the lifespan

Increased

knowledge and

awareness of the

importance of oral

health to overall

health

Increased access to

oral health care

among

underserved

and/or hard to

reach populations

By 2020,

Michiganders

have the

knowledge,

support and

care they need

to achieve

optimal oral

health

8

Appendix B: Key Partner Survey

The 2020 Michigan State Oral Health Plan, which was released in January 2016, is adocument designed to guide policy makers, providers, public health workers,community members and other stakeholders through the next 5 years as they workto improve oral health in Michigan. As it has been a full year since the 2020 StateOral Health Plan was released, we would like to ask some questions of key oralhealth partners on how the plan was used in 2016. These questions will inquire onthe use of the State Oral Health Plan but also on some specific strategies outlined inthe plan.

The survey will take approximately 10 to 15 minutes to complete. Your answers willbe kept confidential and combined with responses of other survey participants. Your open and honest answers are important to the yearly monitoring of planimplementation and to evaluating the overall progress toward accomplishing thegoals outlined in the State Oral Health Plan.

If you have any questions about this survey, please contact Chris Farrell, OralHealth Program Director at the Michigan Department of Health and Human Servicesat (517) 335-8388 or at [email protected].

Introduction

Michigan State Oral Health Plan Key Partner Survey

1. Would you like to participate in the survey?

Yes

No

1


2. How would you describe yourself or the organization you represent? (please select only one)

Coalition or Council

Community-based organization

Educator

Government or Policymaker

Insurer or Third Party Payer

Professional Organization

Provider/Health Care Professional

Public Health Agency

Other (please specify)

3. Are you or your organization a member of the Michigan Oral Health Coalition?

Yes

No

2

Next we would like to ask you about how you used the State Oral Health Plan in the past calendaryear (2016).


4. Have you received a copy of the State Oral Health Plan (either in print or electronically)?

Yes

No

I don't know

5. Have you had the opportunity to read the State Oral Health Plan?

Yes, all

Yes, most

Yes, some

No, not at all

3


6. How aligned or not aligned is your or your organization’s work with the State Oral Health Plan?

Completely aligned

Mostly aligned

Somewhat aligned

Not at all aligned

I don’t know

4


7. Please check the areas in which there is alignment between your or your organization’s goals and theState Oral Health Plan’s goals (please check all that apply).


Health Literacy

Increasing access to oral health care

5


8. In the past year (2016), have you personally used the plan to make decisions or develop plans foraction?*

Yes

No

6


9. Can you briefly describe how you or your organization used the plan to make decisions or developplans for action? Please provide a short paragraph as to what was accomplished, the purpose of thework, the target population, and whether there was a start and end date or if this is ongoing work.

10. Which overall goal of the State Oral Health Plan did this focus on? (please select one goal it mostapplies to)*


Health Literacy


7


11. Within the professional integration goal, which specific objective did this relate to? (please select all thatapply)

Increase the number of health care providers who have formal relationships

Increase the number of educational opportunities that allow oral health and other health providers to work as a single team toaddress patient needs

Establish an equitable payment rate for oral health services among both medical and dental providers

Update oral health guidelines within the Michigan Quality Improvement Consortium (MQIC) guidelines

Increase the number of programs that educate oral health providers on the social determinants of oral health among underservedpopulations

None of these

8


12. Within the health literacy goal, which specific objective did this relate to? (please select all that apply)

Develop and promote consistent messages to educate providers and consumers on oral health through the internet

Increase the number of programs and/or interventions that educate parents on how to prevent early childhood caries amongchildren 0 to 3 years

Increase consumer and health care provider use of evidence-based prevention strategies

Create and support county advocacy networks across that state of Michigan

Collaborate with Michigan's public school districts, including school health and safety programs and school based health centers,to increase oral health awareness activities

None of these

9


13. Within the increasing access to oral health care goal, which specific objective did this relate to? (pleaseselect all that apply)

Decrease the number of children, young adults, adults, and older adults who are underinsured or without dental insurance

Reduce the number of children, young adults, adults, and older adults who experience difficulty, delays or barriers to receivingoral health care

Increase the number of infants, children, and young adults who receive comprehensive dental services

Increase the number of children and adults with disabilities who receive comprehensive dental services

Increase the number of pregnant women who receive comprehensive oral health care

None of these

10


14. Are you or your organization willing to use the State Oral Health Plan as a source to make decisions ordevelop plans for action?*

Yes

No

11


15. Please describe what you or your organization would like to work on in 2017. For example, in a fewsentences describe the work you would like to accomplish, the purpose of the work, the intended targetpopulation, etc.

16. Which overall area of the State Oral Health Plan would this focus on? (please select one goal it mostapplies to)*


Health Literacy


12


17. Within the professional integration goal, which specific objective would this relate to? (please select allthat apply)

Increase the number of health care providers who have formal relationships

Increase the number of educational opportunities that allow oral health and other health providers to work as a single team toaddress patient needs

Establish an equitable payment rate for oral health services among both medical and dental providers

Update oral health guidelines within the Michigan Quality Improvement Consortium (MQIC) guidelines

Increase the number of programs that educate oral health providers on the social determinants of oral health among underservedpopulations

None of these

13


18. Within the health literacy goal, which specific objective would this relate to? (please select all thatapply)

Develop and promote consistent messages to educate providers and consumers on oral health through the internet

Increase the number of programs and/or interventions that educate parents on how to prevent early childhood caries amongchildren 0 to 3 years

Increase consumer and health care provider use of evidence-based prevention strategies

Create and support county advocacy networks across the state of Michigan

Collaborate with Michigan's public school districts, including school health and safety programs and school based health centers,to increase oral health awareness activities

None of these

14


19. Within the increasing access to oral health care goal, which specific objective would this relate to?(please select all that apply)

Decrease the number of children, young adults, adults, and older adults who are underinsured or without dental insurance

Reduce the number of children, young adults, adults, and older adults who experience difficulty, delays or barriers to receivingoral health care

Increase the number of infants, children, and young adults who receive comprehensive dental services

Increase the number of children and adults with disabilities who receive comprehensive dental services

Increase the number of pregnant women who receive comprehensive oral health care

None of these

15


Name:

Organization name:

Email:

20. Please provide your name, your organization’s name, and your email address so that we can contactyou to learn more about the proposed work.

16


21. Can you please tell us why you haven’t read the State Oral Health Plan?

I have been too busy to read it

I was not interested in reading it

I forgot that I received it

I wasn’t sure where to find it

The plan seemed too long

The plan was not clearly written

The plan doesn't pertain to the work I do

I wasn't aware the plan existed

Other (please specify)

17

Now we would like to ask you about some specific strategies outlined in the plan and the work youmay have completed in 2016 that was related to these.

The State Oral Health Plan details objectives and activities that work to enhance professionalintegration between oral health providers, medical providers, and social services providers acrossthe lifespan.


22. In the past year (2016), did your organization have a formal relationship with other providers fromdifferent healthcare disciplines?

Yes

No

I don't know

23. Has the number of formal relationships you or your organization had with providers from differenthealthcare disciplines increased since the release of the 2020 Michigan State Oral Health Plan in 2016?

Yes

No

I don't know

18

Smiles for Life is a National Oral Health Curriculum that produces online educational resources toensure the integration of oral health and primary care.


24. In the past year (2016), have you or your organization promoted the viewing of one or more of theSmiles for Life online training modules?

Yes

No

NA - I/my organization do not/does not work in this capacity

25. Has the number of Smiles for Life modules you or your organization promoted the viewing ofincreased since the release of the 2020 Michigan State Oral Health Plan in 2016?

Yes

No

I don't know

NA - Not applicable

26. In the past year (2016), have you viewed one or more of the Smiles for Life online training modules?

Yes

No


27. Has the number of Smiles for Life modules you viewed increased since the release of the 2020Michigan State Oral Health Plan in 2016?

Yes

No

I don't know

NA - Not applicable

19

The State Oral Health Plan details objectives and activities that work to increase knowledge andawareness of the importance of oral health to overall health among health professionals, policymakers, and consumers


28. In the past year (2016), did you or your organization use social media to educate providers and orconsumers on oral health?

Yes

No

I don’t know

29. Has your use of social media to educate providers or consumers on oral health increased since therelease of the 2020 Michigan State Oral Health Plan in 2016?

Yes

No

I don't know

30. Do you or your organization share social media messaging (e.g. posts on Twitter or Facebook) from theMDHHS Oral Health Program or the Michigan Oral Health Coalition?

Yes, always

Yes, occasionally

No, never

I don't know

20

An objective within the State Oral Health Plan’s Health Literacy goal is to increase parentknowledge on how to prevent early childhood caries among children 0 to 3 years.


31. In the past year (2016), did you or your organization work to educate pregnant women on oral healthpreventive measures?

Yes

No

NA - I/my organization do not/does not work with this population

32. In the past year (2016), did you or your organization work to educate parents of children 0-3 years onhow to prevent early childhood caries?

Yes

No

NA - I/my organization do not/does not work with this population

21

An objective within the State Oral Health Plan’s Health Literacy goal is to create and support countyadvocacy networks across the state of Michigan.


33. In the past year (2016), have you or your organization worked to recruit an Oral Health Champion inyour legislative district?

Yes

No

I don’t know


34. In the past year (2016), have you or your organization worked to align local/legislative district levelcoalition activities and messages with Michigan Oral Health Coalition activities and messages?

Yes

No

I don’t know


35. In the past year (2016), have you or your organization worked to maintain relationships with policymakers to advocate for oral health?

Yes

No

I don’t know


22

An objective within the State Oral Health Plans’ Health Literacy goal is to collaborate withMichigan’s public school districts, including school health and safety programs and school basedhealth centers, to increase oral health awareness activities.


36. In the past year (2016), have you or your organization provided or facilitated oral health educationwithin Michigan’s school system? This could include promoting or working within school-based sealant programs, integrating oral healthmessages within the K-12 school environment, engaging partners from the Department of Education onintegrating oral health messages into the school curriculum, etc.

Yes

No

I don’t know


23

This concludes the survey! Thank you for your participation!


24

name of project: oral health evaluation...focus. these priority areas were refined to establish...

Documents