Skills for Implementing New Practices in Child

Welfare SettingsSeptember 16, 2020Annual Conference

Public Children Services Association of Ohio

Acknowledgements

(R34DA046913) (Bunger)

(US Department of Justice/OH Attorney General 2019-VOCA-132234677)

(S4A 76434) (Bunger)

Learning Objectives

Increase participant’s knowledge of the stages of implementation (EPIS framework)

1Increase participants’ readiness for leading implementation initiatives in their agencies.

2Provide participants with implementation planning resources

3

Today’s Agenda

1. Introductions 2. Intro to Implementation 3. Exploration 4. Preparation 5. Implementation 6. Sustainment 7. Wrap Up

1. Intro’s WELCOME!• Put your name, agency, and role in the chat

1. Intro’s WELCOME!• Put your name, agency, and role in the chat

2. Intro to Implementation What is Implementation?

What is Implementation?

• Active and planned efforts to mainstream an innovation within an organization (Greenhalgh, et al 2004)

Intervention

The evidence-based prevention

or treatment intervention to be

implemented

Implementation Strategies

Deliberate efforts to integrate an EBI into routine practice settings

Outcomes

Implementation Service Delivery Client

e.g. fidelity, acceptability, feasibility

e.g. efficiency, timeliness, satisfaction

e.g. safety, permanency, well-being

Proctor et al., (2009). Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges.

The Implementation Research Framework

The thing you are implementing…

… how you are implementing

Implementation Strategies

Planning• Assess needs• Assess readiness

and barriers• Visit other sites• Develop a plan• Tailor strategies to

address barriers• Build-buy in• Engage families• Recruit leaders• Issue mandates• Build a coalition• Contracts and

partnerships

Educate• Develop &

distribute materials

• Ongoing training• Train the trainer• Consultation• Learning

Collaborative• Engage local

opinion leaders• Educate families• Work with

universities

Finance• Provide incentives• Use capitated

payments• Penalize• Place on fee-for-

service list• Access new

funding• Make billing easier

Restructure• Revise roles• Create a team• Change site• Equipment• Change records

system

Quality Management• Develop fidelity

monitoring system• Audit and

feedback• Reminders• Provide technical

assistance• Provide clinical

supervision• Ongoing team

meetings• PDSA cycles

Policy• Change

accreditation standards

• Change liability laws

• Create licensure or credentialing standards

Powell, B. J., et al. (2012). A compilation of strategies for implementing clinical innovations in health and mental health.

What strategies should you use in your agency?

When?

How Much?

What information should inform your plan?

0

20

40

60

80

100

120

140

160

180

200

May June Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June July

2014 2015 2016

GWC Implementation Strategies - Total Hours

Policy QM Restructure Finance Ed Planning

*Launch*

Pre-Implementation Active Implementation

*Provider Transition*

Bunger, AC, et al (2017) Tracking Implementation Strategies Prospectively: Description of A Practical Approach and Early Findings.

https://episframework.com

2

Each phase is influenced by factors at multiple levels

Moullin, J.C., et al. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implementation Sci 14, 1 (2019).

•Developed to understand implementation in PUBLIC systems

•Emphasizes collaboration with partners

•Encourages us to be rigorous in our planning AND pragmatic

•Approaches implementation as an iterative and dynamic process

•Begins with sustainment in mind

Step 1: Exploration Evaluate need and fit of an EBP

**Adoption decision driven by data and information

What does our community

need?

What are the most effective

models?

How well do they fit our context?

Step 1: Exploration

What does our community

need?

What are the most effective

models?

How well do they fit our context?

• Form an implementation team/steering committee

• Conduct a needs assessmentTIP!- Use SACWIS, charts & other existing data- Draw on local reports- Referrals to external providers- Focus groups w/caseworkers, kids, families- Gather feedback from other sites- Engage coalition/collaboratives

Step 1: Exploration

*****In the CHAT*****What data sources have you used to understand needs in your community? (or… what data would you LIKE to have?)

What does our community

need?

What are the most effective

models?

How well do they fit our context?

• Form an implementation team/steering committee

• Conduct a needs assessment• Identify models/practices

• How effective are they?• Do they address your target outcome?

Step 1: Exploration Child Welfare-Related EBP Registries:• Title IV-E: https://preventionservices.abtsites.com/• CEBC4CW: https://www.cebc4cw.org/

*****In the CHAT*****What data sources have you used to understand needs in your community? (or… what data would you LIKE to have?)

Walsh, C., Rolls Reutz, J., & Williams, R. (2015). Selecting and implementing evidence-based practices: A guide for child and family serving systems [2nd ed.]

What does our community

need?

What are the most effective

models?

How well do they fit our context?

Step 1: Exploration

• Form an implementation team/steering committee

• Conduct a needs assessment

• Identify models/practices• How effective are they?• Do they address your target outcome?

• Decide with your team, committee, or group

What does our community

need?

What are the most effective

models?

How well do they fit our context?

Families

SystemOrganizationHow well does it fit with our agency?:- Values?- Culture/Climate?- Capacities?- Leadership support?- Knowledge/skills?

How well does it fit with system?- Is it consistent with our

policy/legislative requirements?- Can we fund it now? In the

future?- Leadership support?

How well does it fit with our families?- Support from consumer groups?- Effective for different demographic

groups?- Addresses the

Step 1: Exploration

ADOPT!

Step 1: Exploration

Case Example

Step 1: Exploration Resources

● California Evidence-Based Clearinghouse for Child Welfare (CEBC4CW)○ Exploration Worksheet ○ EBP Selection Worksheet

● Community Toolbox○ Help Taking Action

Troubleshooting Guide: We need to understand the community or situation better

Step 2. Preparation

Develop your implementation plan• Anticipate major challenges• Identify strategies to promote• Adapt EBP if needed

Engage Stakeholders

Train your staff

2

Develop Your Implementation Plan• Medicaid? Child welfare funding? Foundation grant? Research grant?How will you fund services?

• Space? Equipment? IT support? Buy-in from staff and Collaborative partners?What resources are needed?

• Who will staff the program? Supervise? Provide support? Do you have to hire? Who will deliver services?

• Self-referrals? From caseworkers? External providers? What is the process? Eligibility criteria? Materials needed?How will you generate referrals?

• One-time training? Boosters/reminders? Expert or developer led? Train the trainer? Consultation or TA? Training for leaders?How will you train/support staff?

• Established measure or new one needed? Data collection procedures? How will you report back? How often?How will you assess fidelity?

• What would you adapt? Why? Will it compromise core components?Do you need to adapt?

• Who's buy-in do you need? How will you reward and support?How do you build a supportive climate?

Step 2: Preparation

Step 2: Preparation

Inner Setting (Organization) Outer Setting (System)

Agency sizeHierarchyCaseloadsLeadershipRole specializationValuesCaseworker Knowledge/skillsSupervisor capacityData systems

Federal or state legislationLocal resources and programming

FundingClient advocacy (major lawsuits)

Partnerships with other organizationsInformation

Values/Expectations of Judges

Implementation is HARD

Strategize around major challenges

POLL – What are the top 3 challenges to implementing in your agency?

Step 2: Preparation

Case Example

Step 2: Preparation Resources

● QIC for Adoption & Guardianship Support & Preservation○ Teaming Structure &

Charter

● Sample Implementation Plan

● The Fidelity, Adaptation, Sustainability, and Training (F.A.S.T.) Lab○ Framework for Reporting

Adaptations & Modifications (FRAME)

Step 3. Implementation

GO!

• Continuous assessment• Adjust strategies

**Emphasis on quality management and reaching fidelity

Fidelity = the degree to which a program is delivered as intended (Proctor et al, 2011; Carroll et al, 2007)

• This is our goal!• Reflects program quality • Measured in terms of the core components of the intervention

are delivered (did they get the full ‘dose’?)• We monitor fidelity and use these data to identify gaps and

problem-solve

Step 3: Implementation

0

20

40

60

80

100

120

140

160

180

200

May June Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June July

2014 2015 2016

GWC Implementation Strategies - Total Hours

Policy QM Restructure Finance Ed Planning

*Launch*

Pre-Implementation Active Implementation

*Provider Transition*

Bunger, AC, et al (2017) Tracking Implementation Strategies Prospectively: Description of A Practical Approach and Early Findings.

Remember this?? Buckle up… It’s ALL quality management ☺ 🡪🡪

Quality Management• Develop fidelity monitoring

system• Audit and feedback• Reminders• Provide technical

assistance• Provide clinical supervision• Ongoing team meetings• PDSA cycles

Step 3: Implementation**Developing an action plan to address challenges**

Many times it’s also…- Addressing

resistance to change- Managing

conflict/distress- Motivating others- Building relationships- Celebrating

accomplishments!

Let’s hear about your experiences!

Can you tell us about a time where you, your team, or your agency overcame (or observed) an implementation challenge?

• What was the problem? • What strategies did you use?• How well did it work?

Please unmute yourself and share!

Step 3: Implementation

Case Example

https://u.osu.edu/ohiostart/evaluation/dashboard/

Step 3: Implementation Resources

• The Center for Implementation• Core Competencies for Implementation Practice

• Community Toolbox• Help Taking Action Troubleshooting Guide: We need to

assure better conditions for implementation.

Step 4. Sustainment

“The degree to which the new services or changes are institutionalized at different levels in the service setting” (Knight et al., 2015).

**Our goal is to sustain the program’s benefit!

Step 4: Sustainment

Funding – Are funds available in the future? Is rate sufficient?

But it’s more than money…

What does it take to sustain a program and its benefits?

Step 4: SustainmentPrograms are more likely to be sustained when they FIT with the context

Avoid Program Drift – deviate from program protocol. So we need ongoing…

•Environmental Support (public, champions)•Funding Stability •Partnerships •Organizational capacity (management, systems, staff)•Program Evaluation (evidence that your program works)•Program Adaptation (to new evidence, environment)•Communications (within and outside of agency)•Strategic Planning (roles, plan for the future)

Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013). The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change.

Schell, S. F., Luke, D. A., et al (2013). Public health program capacity for sustainability: a new framework.

**POLL**

Vote for the top 3 domains that you think are most important to sustaining new programs in child welfare.

Step 4: Sustainment

Case Example

Step 4: Sustainment Resources

Center for Public Health Systems Science, Washington University in St. Louis• Program Sustainability Assessment

Tool (PSAT) www.sustaintool.org• Sample Sustainability Plan

PSAT Domains Assessed

Wrap-Up

• What are you preparing to implement over the next 6 months? Next year?

• What information would be most helpful to you as you plan?

• What is the biggest implementation challenge for your agency? The child welfare system?

Thank You!

Questions? Bunger.5@osu.edu

References (Page 1 of 2)

Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health, 38(1), 4–23. https://doi.org/10.1007/s10488-010-0327-7

Becan, J. E., Bartkowski, J. P., Knight, D. K., Wiley, T. R. A., DiClemente, R., Ducharme, L., ... Aarons, G. A. (2018). A model for rigorously applying the Exploration, Preparation, Implementation, Sustainment (EPIS) framework in the design and measurement of a large scale collaborative multi-site study. Health and Justice, 6(1).

Bunger, A. C., Powell, B. J., Robertson, H. A., MacDowell, H., Birken, S. A., & Shea, C. (2017). Tracking implementation strategies: A description of a practical approach and early findings. Health Research Policy and Systems, 15(1), 1–12. https://doi.org/10.1186/s12961-017-0175-y

California Evidence-Based Clearinghouse for Child Welfare (CEBC) (2015). CEBC Guide to Selecting and Implementing Evidence-Based Practices: Exploration Worksheet. Retrieved from: https://www.cebc4cw.org/files/ExplorationWorksheet-onlineprint-E3.pdf

California Evidence-Based Clearinghouse for Child Welfare (CEBC) (2015). CEBC Selection Guide for EBPs in Child Welfare: Selection Guide Worksheet. Retrieved from: https://www.cebc4cw.org/files/SelectionGuideWorksheet-onlinelinked-E8.pdf

Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation Science, 2(1), 40.

Center for Public Health Systems Science (n.d.) Program Sustainability Assessment Tool (PSAT): Sample Sustainability Action Plan. St. Louis, MO: Washington University in St. Louis. Retrieved from: https://sustaintool.org/wp-content/uploads/2018/04/Sample-Plans-EvaluationPartnersCommunications.pdf

Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013). The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implementation Science, 8(1), 117.

Community Toolbox (n.d.) We need to understand the community or situation better. Help Taking Action Troubleshooting Guide. Retrieved from: https://ctb.ku.edu/en/understand-community-or-situation-better

Community Toolbox (n.d.) We need to assure better conditions for implementation. Help Taking Action Troubleshooting Guide. Retrieved from: https://ctb.ku.edu/en/assure-better-conditions-implementation

Exploration, Preparation, Implementation, Sustainment (EPIS) Implementation Framework (n.d.). Retrieved from: https://episframework.com

Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: Systemic review and recommendations. The Millbank Quarterly, 82(4), 581-629.

Knight, D. K., Belenko, S., Wiley, T., Robertson, A. A., Arrigona, N., Dennis, M., … Leukefeld, C. (2015). Juvenile justice—translational research on interventions for adolescents in the legal system (jj-trials): a cluster randomized trial targeting system-wide improvement in substance use services. Implementation Science, 11(1).

References (Page 2 of 2)

Moore, J, & Khan, S. (2020). Core competencies for Implementation Practice. Toronto, Canada: The Center for Implementation. Retrieved from: https://thecenterforimplementation.com/core-competencies

Moullin, J. C., Dickson, K. S., Stadnick, N. A., Rabin, B., & Aarons, G. A. (2019). Systematic review of the exploration, preparation, implementation, sustainment (EPIS) framework. Implementation Science, 14(1). https://doi.org/10.1186/s13012-018-0842-6

National Quality Improvement Center for Adoption & Guardianship Support and Preservation (QIC-AG). (2015). Implementation Tools: Teaming Structure & Charter. U.S. Department of Health and Human Services, Administration for Children and Families Children's Bureau: Retrieved from: https://www.qic-ag.org/imt/

Powell, B. J., McMillen, J. C., Proctor, E. K., Carpenter, C. R., Griffey, R. T., Bunger, A. C., … York, J. L. (2012). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review : Mcrr, 69(2), 123–57.

Powell, B.J., Haley, A.D., Patel, S.V. et al. (2020). Improving the implementation and sustainment of evidence-based practices in community mental health organizations: a study protocol for a matched-pair cluster randomized pilot study of the Collaborative Organizational Approach to Selecting and Tailoring Implementation Strategies (COAST-IS). Implementation Science Communications, 1(1), 1-13. https://doi.org/10.1186/s43058-020-00009-5

Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 24-34.

Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., Hensley, M., 2011. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38, 65–76. https://doi.org/10.1007/s10488-010-0319-7

Schell, S. F., Luke, D. A., Schooley, M. W., Elliott, M. B., Herbers, S. H., Mueller, N. B., & Bunger, A. C. (2013). Public health program capacity for sustainability: a new framework. Implementation Science, 8(1), 1-9.

Title IV- E Prevention Services Clearinghouse (n.d.). U.S. Department of Health and Human Services (HHS) Administration for Children and Families (ACF). Retrieved from: https://preventionservices.abtsites.com/

Walsh, C., Rolls Reutz, J., & Williams, R. (2015). Selecting and implementing evidence-based practices: A guide for child and family serving systems [2nd ed.]. San Diego, CA: California Evidence-Based Clearinghouse for Child Welfare.

Wiltsey Stirman. S., Baumann, A. A., & Miller, C. J. (2019). The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implementation Science, 14(58), 1-10. https://doi.org/10.1186/s13012-019-0898-y