© the carey group1 iaccac building and sustaining an ebp organization: soup to nuts indiana,...

© The Carey Group 1

IACCAC

Building and Sustaining an EBP Organization: Soup to Nuts

Indiana, November, 2012

Mark Carey

The Carey Group

[email protected]

651-226-4755


Objectives

• Examine a model of building and sustaining an organization (State of Pennsylvania)

• Assess own progress


Building and Sustainingan EBP Agency

A state model:

- Pennsylvania’s JJSES

- Dr. Lipsey’s SPEP process

3


• Started in 2011• Goal is for each of the juvenile justice systems

within the 67 counties to adopt EBP• Leadership team with three agencies led the effort

– Juvenile Court Judges’ Commission– Pennsylvania Commission on Crime and Delinquency– Council of Chief Juvenile Probation Officers

• Premise: a well planned and supported effort will improve odds of successful implementation

Juvenile Justice System Enhancement Strategy (JJSES)


A Well Planned Effort

• Guided by the science of implementation• Premise:

– Taking time up front will pay dividends later– There is a proper sequence– Entice others toward change versus forcing

• Four stages– Each stage builds off the other– Each stage has a stage leader– Must gain proficiency in one before move onto the next

(each Stage has a proficiency checklist)

Four Stages


Stage One

Organizational Readiness

- Organizational / systemic change is challenging

- Leaders should ensure that the climate of their organization is conducive to change

- Are direct service staff ready to move forward?

- Are supervisors supportive of change?

- Is there good communication up and down the “chain of command”?

- Tools will be available to assist jurisdictions


• An “Organizational Readiness Assessment” will help identify possible barriers to implementation

• Based on worker productivity research

• To be given to all staff and followed through

Tool One: Organizational Readiness Assessment


• Starting initiatives is easy

• Sustaining them takes persistency and strategy

• What will “change” cost from a human, political, fiscal, and mission point of view?

• A cost and benefit “guidebook” will be made available to help administrators analyze costs associated with JJSES implementation

• It is recommended that work teams, comprised of a diagonal slice of the agencies, be created to examine these and other issues.

Tool Two: Cost/Benefit Analysis


• A variety of training curricula is available to be delivered by others or in-house including

– A two hour EBP overview for stakeholders– A six hour EBP introduction– Motivational interviewing (2 days)– Four core competency– Carey Guide training– Leadership training

Tool Three: Training Curriculum


• At the end of the analysis, an eighteen month action plan is recommended

• Provides staff with direction of agency vision

• Helps leadership plan for next couple of years including budget priorities

Tool Four: Action Planning Template


Stage Two

Initiation

- This stage is to prepare agency for behavioral change techniques

- By themselves they do not change behavior

- Without them it is not likely they can achieve risk reduction


• Practitioners learn skills to prepare individuals for change

• Often takes over a year to become proficient, with booster trainings and quality assurance checks

• Sets the stage for juvenile justice professionals and youth to work on issues identified through the assessment and case planning process

• A strategy is being developed that would provide financial assistance to jurisdictions (through PCCD) to support MI training and evaluate its effectiveness

Motivational Interviewing


• Using valid and reliable screening and assessment instruments can significantly improve decision-making

• Moves us away from what we think works, feels good, customs and intuition

• A more professional way of doing business

• Is research-based, and has definable decision-making criteria

• Standardizes decision-making within an organization

• Addresses disproportionate minority contact concerns

• Overrides are allowed, but must be approved and monitored

Structured Decision-Making


• Evaluates the need for secure detention, based on objective criteria

• Risk factors determine the likelihood of re-offending prior to court, and appearance in court

• Standardizes decision-making

• Can help address disproportionate minority contact (DMC) concerns

• Process includes consideration of alternatives to detention

• PCCD requires utilization of detention assessment instruments with funded Evening Reporting Centers

• 4 juvenile probation departments currently involved in Juvenile Detention Alternatives Initiative (JDAI)

Detention Assessment Instruments


• A 52-item, valid and reliable, self-report screening instrument that identifies potential mental health and substance abuse needs

• Administration time: 10 – 15 minutes

• 5th grade reading level

• Currently being utilized in 25 juvenile probation departments, as well as detention centers and YDCs

• Hardware and software costs supported by PCCD

• Referrals for clinical evaluations should be made for juveniles identified as high risk

Massachusetts Youth Screening Instrument

Second Version (MAYSI-2)


• Implementation costs supported by $75,000 grant from PCCD to Pa. Council of Chief Juvenile Probation Officers

• Fifty-one counties, involving three “phases,” have been trained on the YLS. A 4th phase is planned during 2012.

• Integrated into the Pa. Juvenile Case Management System– YLS-related data can be generated

Youth Level of Service (YLS/CMI)


• Process to help juvenile probation officers generate the same score/come to the same conclusion after administering a screening or assessment instrument

• Injects a Quality Assurance aspect to screening and assessment

• Underscores the importance of initial training

• Underscores the importance of booster training

• Underscores the importance of supervisory oversight

Inter-rater Reliability


• Develop “SMART” case plans based on risk and need– Specific, Measurable, Attainable, Relevant, and Time-bound

• Should target interventions to identified needs

• Needs should be prioritized

• Strengths should be built upon

• Balanced and restorative justice principles should be addressed

• A standardized case plan is being developed in the Pa. Juvenile Case Management System (PaJCMS) that will be populated with YLS data

Case Plan Development


Stage Three

Behavioral Change

- This stage is to conduct the interventions that work to reduce risk

- Includes preparing staff to become skill teachers

- Requires partnership with local service providers


• Most higher risk juveniles are lacking pro-social skills(i.e. conflict resolution, anger management, problem-solving, controlling emotions)

• The most effective interventions are behavioral

• “Skill practice” involves:– observing others– practicing new behavior– receiving feedback on the practiced behavior, and– applying the behavior in real life situations

• In a risk reduction model, juvenile probation officers’ roles include teaching skills

• POs must have the training, skills and tools to perform this role

Skill Building and Tools


• “Thinking errors” include:– Rationalizing and justifying anti-social or delinquent behavior– Misinterpreting social cues– Under-developed moral reasoning– A sense of entitlement– A failure to assess consequences of actions– A lack of empathy for others– Poor problem-solving and decision-making skills

• Research indicates that cognitive behavioral interventions (i.e. “Aggression Replacement Training” or “Thinking for a Change”) can significantly reduce recidivism (20-30 %)

Cognitive Behavioral Interventions


• The 3 fundamental principles of Evidence-based Practice are: Risk, Need, and Responsivity

• Responsivity is the least understood and least applied principle:– Few people really understand what it means– Very few standardized instruments to measure it– Additional resources or services may be needed to address it

• Responsivity’s three main components:

1. Align supervision and treatment with the juvenile’s learning style

2. Match the characteristics of the juvenile with those of the officer or service provider

3. Match the skills of the officer / service provider with the type of intervention

Responsivity Tools


EBP Programs

Big Brother Big Sisters Life Skills Training Program

Functional Family Therapy Multidimensional Treatment Foster Care

Project Toward No Drug Abuse Olweus Bullying Prevention Program

Aggression Replacement Training . The Incredible Years

Strengthening Families Promoting Alternative Thinking Strategies

Multisystemic Therapy

Evidence-based Programming and Interventions

The Evidence-based Prevention and Intervention Support Center (EPISCenter at Penn State Univ.) supports the dissemination, quality implementation, sustainability, and impact assessment of proven-effective prevention and intervention programs, and conducts original research to advance the science and practice of evidence-based programs.


• Service providers need to understand and align themselves with the principles of evidence-based practice

• When services (community or residential) are matched to youths’ crime-producing (criminogenic) needs….the lower the chance of repeat offending

• In other words, the right services for the right youth

• A “Service Delivery Matrix” can be used to organize services by domain, and risk level within those domains

• We should not mix low risk youth with moderate and high risk youth during programming

Service Provider Alignment


Sanctions and Rewards

• Behavioral change is promoted when you use a balanced approach in applying:– sanctions for antisocial behavior – rewards to positively reinforce pro-social behavior

• Desired and prohibited behaviors, and consequences, must be understood

• Local policy should support a graduated responses framework

Graduated Responses


Stage Four

Refinement

- This stage is to designed to measure progress and improve quality

- Many of these activities could be/should be employed earlier but were put in Stage Four in order to reduce the sense of being overwhelmed and to gain proficiency in basic areas first


Policy Alignment

• Do local policies support evidence-based practice?

• Do court recommendations reflect the risk, need and responsivity of juveniles?

• Do caseload sizes reflect offenders’ risk levels?

• Are SMART case plans developed and monitored?

• Do low-risk offenders receive minimal supervision?

• Are low risk offenders separated from high risk offenders during programming?

• Do juvenile probation officers receive training to develop skills necessary to deliver evidence based programming?

• Do supervisors monitor and reinforce juvenile probation officer skills?

Stage Four: Refinement


• Manage data to develop “dashboards” for juvenile probation administrators

• Potential measures include:– Number of children in placement– Total active cases– Cases pending– Types of offenses of active cases– Caseload sizes of probation officers– YLS-related data (risk and need levels)– Service-related data in case plans

Performance Measures


• Contracts with juvenile justice service providers could be standardized to require agreed upon evidence-based performance measures and outcome data

• Contracts could require service providers to adhere to the research principles – Dosage should be consistent with what research indicates is

effective– Do providers have internal protocols for monitoring adherence to

written policies?

• Fiscal policy should support evidence-based programming

EBP Service Contracts


Other Items

All Stages

- The following areas cut across all four Stages


• Although juvenile probation departments typically don’t provide delinquency prevention services, all should be encouraged to promote delinquency prevention activities as the most cost-effective approach

• Community members should be engaged

• “Community-level” risk factors and service needs should be identified (what are the real problems facing your community? What can we do about it?)

• Cross-system collaboration is key (children and youth, mental health, drug and alcohol, education, etc.)

Delinquency Prevention


• Goal is to avoid significant long-term consequences of adjudications of delinquency

• Initiative began with the goal of diverting children with mental health problems from formal juvenile court processing

• Initial concept was expanded to provide diversion opportunities to all appropriate youth

• A “Guide to Developing Pre-Adjudication Diversion Policy and Practice in Pennsylvania” was developed

• “The Pennsylvania Juvenile Collateral Consequences Checklist” was published

• In 2011, PCCD announced up to $2M in funding to support pre-adjudication diversion programs

Diversion


• Initiative began in conjunction with PA’s MH/JJ targeted area of improvement

• Premised on the believe that we can, and should, do a better job of involving families in the JJ process

• A monograph entitled “Family Involvement in Pennsylvania’s Juvenile Justice System” was developed

• A training curriculum for juvenile probation officers on family involvement is being developed

• A handbook for youth and families is being developed

Family Involvement


• How well are we doing?– Do we know?

• A definition of recidivism has been established– An adjudication of delinquency or criminal conviction for a

misdemeanor or felony offense within 2 years of case closing• Recidivism data for cases closed in 2007 is being generated• Data must be analyzed, and used to assist in decision-making

Data Driven Decision-Making


• CQI should be reflected in all activities– Inter-rater reliability checks– YLS booster training– Motivational Interviewing booster training and observation– Quality of case plan development– Program “fidelity to the model”– Use of “Dashboards”

• All staff must understand the importance of CQI• Supervisors are critical to ensuring CQI • Training programs around CQI must be developed

Continuous Quality Improvement (CQI)


• We currently have sufficient research and evidence-based tools to greatly improve the outcomes and cost-effectiveness of most JJ systems.

• The main barriers are organizational– implementing and sustaining evidence-based tools and practices in routine JJ operations.

• We have a framework for system-level adoption of evidence-based practice:– Graduated sanctions applied commensurate with risk (risk assessment tools)– An array of effective evidence-based programs (SPEP)– Matching of programs to offenders’ criminogenic needs (needs assessment

tools)– Concentration of supervision and program resources on higher risk offenders

(disposition matrix for structured decision making)

Summary

Source: Dr. Mark Lipsey


Building and Sustaining an EBP Agency

• Fill out the assessment

© the carey group1 iaccac building and sustaining an ebp organization: soup to nuts indiana,...

Documents

carey group

mark careythe carey

stage twoinitiationthis

othereach stage

ebp organization

behavioral change techniques

sustainingan ebp agency

forcingfour stageseach