figure 1. key steps and timeline

2006 2007 Nov Dec Jan Feb Mar Apr May June

Planning Phase Implementation Phase

Needs and Resource

Assessment

Strategic Action Identification

Program Logic & Outcome Definition

Begin Implementation, outcome evaluation, and

social marketing

Sustainability Activities

Update Strategic Plan

Mobil

izat

ion

Begin Drafting

Strategic Plan

Co

ntr

act

wit

h L

ead

Ag

ency

Continue refinement of Strategic Plan

Prepare

Outcome Report

2007 2008

July Aug Sept Oct Nov Dec Jan Feb

Implementation Phase

Continue with program implementation, utilize data to promote quality improvement, strategically evaluate program marketing,

Sustainability Activities

Identify funding sources; strategize redeployment or blending of funds; improve/continue community engagement in “Learning Collaborative”; continue refinement of strategic plan;

process & outcome evaluations to aid in data-supported decision making

Figure 1. Key steps and timeline

From Mandates to Reality: The use of a community-based model to effectively implement evidence-based practicesSuzanne E.U. Kerns1, Andrea Parrish2, Eric Trupin1, Eric J. Bruns1

1 University of Washington School of Medicine2 Behavioral Health Resources, Olympia, WA

The mandate…… The reality…

Evidence-Based Practices Proviso

• In 2006, the Washington State House and Senate approved a Legislative Proviso for the Mental Health Division

• Mission: Establish a pilot program to expand the use of evidence-based mental health services to children

• Program site selected through a “Request for Proposals” process, open to counties, Regional Support Networks (RSNs), and Indian Nations

Requirements• Commitment to work with

community partners– Consumer/family representatives– Representatives from local MH,

juvenile justice, child welfare, and other child-serving areas such as education and health care

• Identify areas of need from these stakeholders

• Identify the service or services to be provided based on community needs and resources

• Select an empirically supported treatment from a list of options developed by a panel of subject matter experts

• Participate in efforts that will ensure adherence to the chosen empirically supported treatment (ESTs) and evaluate the outcomes of implementation

• University of Washington, Division of Public Behavioral Health and Justice Policy (DPBHJP) to provide support and assistance in all phases of the pilot program

Consultation Provided under contract as part of

the Proviso DPBHJP to facilitate community

process to select, implement, evaluate, and sustain ESTs

Largely modeled after Ohio’s “Partnerships for Success” process (see description in blue box below)

Targeted Outcomes

• Development of sustainable cross-agency planning entities• Creation of local training and coaching consortia to support EBP

implementation • Increased adoption and implementation of EBPs• Development of data collection infrastructures• Significant increases in external grant funding• Leveraging, blending, and pooling of previously “siloed” internal funding

sources to support EBP implementation

Partnerships for Success

Core values

• Community based• Participatory• Data-informed• Balancing a Holistic Continuum

of Approaches

Partnerships for Success (PfS) model

• A comprehensive, community-based participatory approach towards strategically identifying empirically-supported treatments for prevention, early intervention and treatment of child mental and behavioral health problems

• Developed by Al Neff & Dave Julian (Ohio State University Center for Learning Excellence)

• Currently being utilized in 44 counties in Ohio• Established cost savings of $11.52 for every dollar invested in PfS

The proposal submitted by Thurston-Mason (T-M) Counties was chosen for funding from among several competitive proposals. The initial frame of the proposal was targeting the needs of youth with complex behavioral and mental health concerns and who are involved in multiple systems. The T-M core team identified and brought together a broader group of community stakeholders to actively participate in a strategic approach to building a “Learning Community” and increasing their community’s capacity to respond effectively to child and adolescent problem behaviors.

MethodCommunity Process

An adaptation of the Partnerships for Success model (described below) was utilized to frame the community process. The first step was to convene key community stakeholders, representing a broad range of child-serving agencies (e.g., schools, juvenile justice, mental health, child welfare), about ESTs and Partnerships for Success. A small group (Core Team) was established to oversee and guide the effort.

The key steps and associated timeline are presented in Figure 1. Because of the legislative mandate to have services available approximately 6 months after initiation of the project, the process was dramatically condensed from a more ideal planning process timeframe of 1 year.

Choosing a priority program

For more information about Partnerships for Success, contact Melissa Ross at:

Ohio State Center for Learning Excellence807 Kinnear Road, Columbus, Ohio 43212 (614) 292-0175  Email: ross.565@osu.edu

www.pfsacademy.org

Priority Program : Multisystemic Therapy (MST)

MST was selected from a menu of 8 programs presented to the community on the basis of their ability to address known needs (multi-system involved youth), target populations, target impacts (family, school, youth, and community domains), and alignment with guiding principles. An iterative consensus process was utilized to choose the practice.

Partnerships for Success – Perceived Benefits for Thurston & Mason Counties• Promotion of systematic thinking about community needs

◙Generated “outside of the box” thinking◙Provided context for a strategic planning process that can be generalized to additional

projects and programs• Initiation of a new program for youth involved in multiple systems with community buy-in from

multiple stakeholders◙Organized system to pull key community members together with a common focus

• Technical assistance and support facilitated the utilization of the Partnerships for Success model

◙Promoted context for University-community partnership• Opportunities for networking and contacts resulted in infrastructure for adoption of additional

programs◙e.g., Foster Care Assessment Program; Family Integrated Transitions program

ChallengesImplementation • Timeline – had to greatly condense

activities• Contracts

• How to make “process” activities fit within typical contract language

• Sustaining Community Team as “Operating System”

• Intersection of MST with Community Mental Health• Navigating the role of the

purveyor in service delivery• Fitting MST into existing billing

structure and infrastructure Evaluation• Identifying a reasonable set of

evaluation priorities in the face of so many proposed layers of activity (impacting on youth, program, staff, stakeholder, community)

• Anticipating length of time to realize long-term impacts

• Determining feasible methods to rigorously test the impact of the model, given the above complexity and likelihood of diffusion of effects across so many "links" in the logic chain

Figure 2. Guiding Principles

Full family partnership

Sustainability

Data Driven Decision

Making

Cultural and Linguistic

Competence

Full Partnership

and Support to Provider

Staff

Shared Leadership

Fit to Community

Needs

Community Collaboration

Accessibility

Guiding Principles

Table 1. Target Impacts

Next Steps- Fine-tune implementation to

ensure high fidelity - Redeploying of funds for Tribal

planning process- Enhance Community Team’s role

as an “Operating System”- Address issues related to

sustainability

InputsInputs ActivitiesActivities OutcomesOutcomes Long-Term ImpactsLong-Term Impacts

Sys

tem

Leve

lS

yste

m L

eve

lB

rid

ge L

eve

lB

rid

ge L

eve

lP

ract

ice L

eve

lP

ract

ice L

eve

l

Proviso funding

Regional Support Network

Mental Health Division Community & Tribal stakeholders

Behavioral Health Resources

In-kind resources

University of Washington

Project workgroupsMultisystemic Therapy intervention team

Administrative supports

Identification of Core Team members to provide project oversight and coordination

Collect data on needs and resources from a community perspective

Identify gaps in services and service priorities (i.e., targeted impacts and target population)

Identify empirically-supported treatment/s

Develop an implementation plan in alignment with guiding principles

Utilize data (e.g., process evaluation) to make decisions

Conduct planning process with Tribal communities

Articulate goals and guiding principles Implement MST with fidelity Participate in outcome evaluation

Ability to serve children and adolescents within their own communities

Expanded services and supports

Improved access to services

Cost savings Reduced

disparities for minority youth (incl. Tribal youth)

Enhanced ability to receive funding to address children’s mental health issues

Ability to quickly mobilize around additional opportunities or problem-solving needs

Improved social supports for families in the community

More skilled provider staff

Youth prevented from deeper penetration in systems

+

+

+

Mental health Sx and functional impairment

Out-of-home placements

Academic, social, and emotional health

Recidivism Family

functioning

Community able to respond to identified needs

Coordinated services for youth involved in multiple systems

Implementation Enhanced

collaboration with UW

Development of an EBP workgroup (community team)

Participate in Partnerships for Success collaborative process

Review data on community resources and child well-being indices

Review implementation progress and outcome data

Participate in development of a sustainability plan

Cross agency collaboration and fiscal blending

Improvements in collaborative planning between community & state partners

Strategic plan Sustainability plan

Figure 3. Logic Model (Theory of Change)

DomainDirection of

change

Family

Family functioning ↑

Parent education ↑

Parent/School communication ↑

Family engagement ↑

Domestic violence ↓

Parental conflict ↓

Use of foster care ↓

School

School success ↑

School discipline ↓

Youth

Aggressive/Defiant behavior ↓

Substance use/abuse ↓

Placement disruptions ↓

Use of JJ facilities ↓

Suicide/suicidal gestures ↓

Abuse/Neglect trauma ↓

Community

Resource access ↑

Community support ↑

Stigma ↓

Access to services ↑

Highlights from Needs Assessment•Annually, approximately 1,800 youth through age 20 were enrolled in the public mental health system in Thurston and Mason counties.

•Approximately 9% of these youth account for more than half of the mental health expenditures. These youth are involved in multiple systems.

•The average MH treatment cost for youth involved in multiple systems was $19,742, compared with $1,773-$3,032/youth in only one system.

•Needs Assessment survey revealed that the primary reason for their being a gap in services and resources is “access to and availability of services.”

EvaluationData collection strategies: • Community Surveys (Community Team)• Key Informant Interviews (Mental Health Division, RSN,

Community Mental Health Center, Core Team)• MST Implementation Surveys (MST Service Providers)Measured Outcomes:• Outcomes and Long-Term Impacts articulated in logic model

(Figure 3)