August 4, 2012

Juvenile Justice System Enhancement

Strategy Implementation

Manual

A “How To” Guide

JJSES | STAGE ONE

1

Introduction Most of the early adopters of evidence-based practices have “muddled through” a set of

planning activities culminating in sometimes-effective, sometimes-ineffective decisions.

Fortunately, much has been learned through these early, pioneering efforts. In addition,

organizational change research has helped guide successful change strategies. The Juvenile

Justice System Enhancement Strategy (JJSES) Implementation Manual “How-To” Guide was

developed to shorten the learning curve for evidence-based practices (EBP) planners and

reduce the muddling through events. This Manual was written for those individuals within

Pennsylvania’s juvenile probation departments who serve in a leadership or coordinating role

around evidence-based practices. The work by these leaders and coordinators in aligning

policy and practice with risk reduction research evidence requires a thoughtful, strategic plan

and process. Pennsylvania’s juvenile justice system seeks to take advantage of the lessons

learned from early adopters, research, and JJSES resources to take a more direct route for

achieving system enhancements.

This How-To Guide provides local probation leaders and coordinators with a step-by-step

process through each of the four recommended Stages of Implementation (Readiness,

Initiation, Behavioral Change and Refinement). Each Stage is divided into activities to be

conducted in the beginning, middle and end of the Stage and contain the following:

Outcomes: Key outcomes to be achieved as a result of completion of the Stage

activities

Competencies: Staff competencies to be acquired as a result of the training conducted

within the Stage

Activities: Recommended activities to be employed or initiated by the probation

department during the corresponding Stage

Support: The type of Stage supports and resources made available to assist

probation agencies implement JJSES successfully

Proficiency: A checklist to determine if the department has met the proficiency

benchmarks to be ready to engage in the activities of the next Stage

Performance A set of core performance measures to measure progress in achieving

Measures: JJSES goals

Resources For A list of additional resources related to the Stage activities

More Information:

JJSES | STAGE ONE

2

I. STAGE ONE

Outcomes by the End of Stage One1

1. 90% of the probation staff will become knowledgeable about evidence-based practices

based on an 80% or better score on an EBP pre/post test

2. 75% of the local justice system stakeholders will become knowledgeable about

evidence-based practices based on an 80% or better score on an EBP pre/post test

3. The department will have completed a human, fiscal, and data cost analysis of JJSES

4. The department will have assessed its organizational readiness for JJSES

implementation and adequately addressed any shortcomings as determined by the

pre/post Likert Readiness to Change survey

5. The department will have completed an eighteen month action plan

Competencies Achieved in Stage One

Knowledge about research findings on risk reduction

Awareness of human, fiscal, and data costs in implementing EBP through JJSES

Leadership and organizational change skills

1 The outcomes listed throughout the Chief’s How-To Guide should be viewed as drafts. That is, until baseline data is

collected it is not clear what is reasonable to expect or what a plausible target should consist of. The outcome figures are best estimates at this stage of JJSES development.

Stage One: Readiness

As a result of this stage, the jurisdiction will acquire knowledge about risk reduction research and analyze the fiscal, human resource, and data costs

required to implement Pennsylvania’s Juvenile Justice System Enhancement Strategy. The jurisdiction will examine its own capacity for change by

assessing its readiness.

JJSES | STAGE ONE

3

Activities and JJSES Support

Summary of Stage Beginning: The first series of activities are designed to get the

probation department and stakeholder leadership on board with the fundamental research that

has guided changes in juvenile justice risk reduction practices. Key to this stage is helping

juvenile justice system administrators fully understand why aligning the juvenile probation

department’s policies and practices with research evidence is consistent with their public safety

mission. If juvenile justice professionals are not convinced that research evidence can improve

public safety outcomes, then the JJSES will not make sense or be sufficiently supported.

Activities. The first step in Stage One is to

understand the framework of the JJSES model

and to conduct initial EBP training. The

recommended activities include:

1. Review the results of the County EBP

Implementation Survey conducted in the

summer of 2012. The survey describes

the manner in which departmental

policies and practices are aligned with

EBP, as well as what resources and

technical assistance may be needed.

2. Communicate with a JJSES Point of

Contact (POC) if there is a question

regarding JJSES implementation or a

need for technical assistance. A POC list

has been established that enables a

county to contact individuals who can

respond to questions or connect counties

to available resources or technical

assistance. The POC list was developed

to give counties access to consultation with colleagues with technical expertise, with

whom they may have a shared collaborative history, or from a county similar in size,

demographics or location.

3. Train probation staff on EBP. This training provides every staff member with the

foundational EBP knowledge on which future EBP training will be built.

4. Train stakeholders, service providers, and victim advocates on EBP. In addition to

department staff, others involved with the department will need to be familiar with the

JJSES Support for Beginning of Stage One

Monograph on JJSES

Intro to EBP training curriculum (one day) for probation staff

Intro to EBP training curriculum (two

hours) for stakeholders

EBP training for new staff on EBP at the orientation trainings

A listing of data inputs and outcome

data needed to be collected locally for this Stage (forthcoming)

Technical assistance

JJSES | STAGE ONE

4

same foundational EBP knowledge. This shared knowledge base will help build support

for future policy and practice changes.

5. Conduct an EBP knowledge test before and after the training. Administering the pre-

post EBP knowledge test (see enclosed) will increase the confidence of the department

that core EBP information has been retained (as measured by pre and post scores) or to

develop a plan to address deficiencies (in the event of unsatisfactory scores).

6. Deliver or communicate with a POC to obtain the Staff Introduction to EBP Training

PowerPoint (6 hours) and a “Stakeholder Introduction to EBP Training” PowerPoint (2

hours).

Additional Resources. For more reading on the core principles for evidence-based

practices, the following resources are available

Implementing Evidence-Based Practice in Community Corrections: The Principles of Effective

Interventions, National Institute of Corrections. See http://nicic.gov/Library/019342

Tools of the Trade: A Guide to Incorporating Science Into Practice, National Institute of

Corrections. See http://nicic.gov/Library/020095

Coaching Packet: Implementing Evidence Based Practices, Center for Effective Public Policy.

See http://www.cepp.com/documents/Implementing Evidence Based Practices.pdf

Evidence-Based Practice to Reduce Recidivism: Implications for State Judiciaries. See

http://community.nicic.gov/media/p/6977.aspx

Colorado Probation and Evidence-Based Practices: A systemic view of the past, present &

future of EBP in Colorado Probation. See

http://www.courts.state.co.us/userfiles/file/Administration/Probation/APPA_Presentation_Materia

l/EBP_Report.pdf

JJSES | STAGE ONE

5

Summary of Middle of Stage One: The next set of activities in the Readiness Stage of

JJSES is to examine the department’s organizational capacity to undergo a long-term re-

engineering process. This self-examination spans a number of areas such as leadership and

staff alignment with the values and expectations of an EBP department, resource capacity,

timing, organizational robustness, stakeholder support, and management commitment. It is not

advised that a department begin an EBP initiative

until it is confident that it has sufficient stamina,

resiliency, and desire to successfully implement the

required changes. If the organization is not fully

prepared or the existing environment is not

conducive to this change effort, the JJSES will not

take hold and efforts to re-engage staff and

stakeholders in the future will be more challenging.

Activities. The activities in the middle of Stage

One are designed to build on those completed at

the beginning of Stage One. Once the department

is fully aware of the demands of an EBP

environment it can better determine whether it

wants to proceed. Therefore, the activities in the

middle of Stage One are designed to help the

department assess whether it is prepared to move

ahead. This part of the Stage includes an

assessment process to help answer the readiness

question and to assist in preparing the department for the type of changes that follow. The

activities to be completed include:

1. Conduct an organizational readiness survey. This survey assesses the department’s

“health” which is directly tied to likelihood of achieving a successful change initiative.

The survey, to be taken by all department personnel, measures perceptions of effective

communication, staff alignment with department mission, trust in leadership, etc. The

survey is enclosed.

2. Complete a cost benefit analysis. EBP will likely require the shifting of resources and

implementing policy decisions that will result in “trade-offs.” For example, the

department may need to terminate some activities in order to provide resources for

another; establish a continuous quality improvement process; and ensure that personnel

will have the time to integrate EBP activities into their day-to-day work. The cost benefit

analysis will allow the department to examine what it hopes to gain through JJSES and

determine whether it is prepared to make the cost-related decisions to achieve these

gains. The cost benefit analysis worksheet is enclosed.

JJSES Support for Middle of Stage One

Organizational Readiness survey

(enclosed)

Cost Benefit Analysis Worksheet

(enclosed)

Technical assistance

JJSES | STAGE ONE

6

3. Determine level of technical assistance support needed, if any. Counties may elect to

either: a) conduct a self-administered organizational readiness survey, cost/benefit

analysis, and an action plan for implementing EBP (see enclosed); or b) request an

independent EBP readiness and/or alignment assessment to be conducted by a team

assembled from the POC list. This assessment will examine existing policy and practice

to provide the agency with a sense of how its operation is currently prepared for, and in

alignment with, EBP and where it needs to change in order to be consistent with

research evidence. The self-administered tools are enclosed and an independent

assessment can be accessed by contacting a POC.

4. Develop an action plan. An action plan can be developed once the organizational

readiness and cost benefit analyses have been completed and areas of concern

identified by the readiness survey have been addressed. Anticipated costs will need

to be identified and addressed by the department and stakeholders. A “roadmap”

will need to be constructed to give stakeholders and all juvenile court personnel a

clear sense of where the department intends to go, what the activities will look like,

and when they will be accomplished. An action plan template is enclosed.

Additional Resources.

For more reading on implementation and planning issues, the following resources will be made

available on the JJSES website:

1. Implementation Research: A Synthesis of the Literature, authored by Fixsen, D. L.,

Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Tampa, FL: University

of South Florida, Louis de la Parte Florida Mental Health Institute, The National

Implementation Research Network (FMHI Publication #231). Available at

http://ctndisseminationlibrary.org/PDF/nirnmonograph.pdf

2. The Change Book; a Blueprint for Technology Transfer, Addiction Technology Transfer

Center Network. Published in 2010 by the Addiction Technology Transfer Center

(ATTC) National Office University of Missouri-Kansas City, Missouri through a

cooperative agreement from the Substance Abuse and Mental Health Services

Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT). See

http://attcnetwork.org/explore/priorityareas/techtrans/docs/TheChangeBook2010.NOfl.pdf

3. Roadmap for Evidence-Based Practices in Community Corrections, Crime and Justice

Institute at Community Resources for Justice (2010). Commonwealth of Virginia. Report

supported by Grant No. 2007-DD-BX-K011, Bureau of Justice Assistance, Office of

Justice Programs. Available at http://cjinstitute.org/files/Roadmap_Final.pdf

4. A Ten-Step Guide to Transforming Probation Departments to Reduce Recidivism.

Council of State Governments Justice Center, Fabelo, T., Nagy, G., and Prins, S.

Report supported by the Bureau of Justice Assistance (BJA), Office of Justice Programs,

JJSES | STAGE ONE

7

U.S. Department of Justice (under grant # 2009-DD-BX- K139); the Pew Center on the

States Public Safety Performance Project; the Texas Department of Criminal Justice–

Community Justice Assistance Division

(TDCJCJAD); and the Travis County

Community Supervision and Corrections

Department (CSCD) Adult Probation

Department. Available at

http://knowledgecenter.csg.org/drupal/content/te

n-step-guide-transforming-probation-

departments-reduce-recidivism

5. Step-by-Step Planning Guide; Six Phases

Toward Implementing Evidence-Based

Practices for Risk Reduction, The Carey

Group. Available at

http://www.thecareygroupinc.com/complimentary

resources

Summary of Stage End: The final set of

activities in the Readiness Stage involve acting on

the information collected in the beginning and

middle portions of Stage One. For example, the

organizational readiness survey and initial EBP

trainings will likely reveal a set of issues that

require immediate attention. The alignment and cost benefit analyses will likely lead to activities

for the action plan. These Stage End steps are designed to put in motion the preparation for

activities to be commenced in Stage Two.

Activities. The following activities are derived from those prompted by the Middle Stage (i.e.,

surveys and analyses) and include:

1. Follow up to readiness survey. It is critical that department staff who took the survey see

the results, understand the department strengths, and be given assurance that areas of

concern will be addressed. Often, agencies will create workgroups at this stage to review

survey results and develop an action plan. By addressing these organizational climate

issues the department will be in a better position to implement EBP change initiatives and

garner staff buy-in and support.

2. Stakeholder Involvement. Jurisdictions will differ in the level of desired stakeholder

involvement. For example, in some cases, the attorney for the Commonwealth and

defense counsel will want to be immersed in the policy-related discussions at the juvenile

probation department level and in other counties they will only want to be informed. Each

department will need to determine the level of stakeholder involvement desired at this

JJSES Support for End of Stage One

A listing of local data needed to be

collected for this Stage (forthcoming)

Action planning template (enclosed)

Organizational change and project management leadership training

Proficiency checklist to advance to

Stage Two

Technical assistance

JJSES | STAGE ONE

8

stage of the JJSES implementation. As significant policy decisions are made over time,

stakeholder involvement will likely need to increase.

3. Develop an eighteen month action plan. To address the issues identified in Stage One,

the department should initiate an action planning process with the goal of identifying the

concrete action steps that should be taken over the course of eighteen months. This

action plan will provide juvenile court personnel and stakeholders with a roadmap

regarding the what, who, and when of JJSES implementation. It will also give the staff a

clear picture of what JJSES entails and how their job functions will likely be affected over

the course of JJSES implementation. The action plan template is enclosed.

4. Administer a Stage One proficiency checklist. To help determine readiness to participate

in Stage Two of the JJSES the department is encouraged to complete a proficiency

checklist. This self-administered checklist includes a list of the activities to be completed

in the Initiation Stage and an assessment as to whether the department is ready to move

to Stage Two. The Stage One proficiency checklist is enclosed.

5. Request assistance for Stage Two. Once the proficiency checklist is completed,

department leadership should indicate its commitment to move ahead to Stage Two and, if

needed, request technical assistance (training, access to curricula/documents,

consultation services, etc.).

6. Organizational change and project management leadership training. JJSES will be

delivering a leadership training curriculum to assist Chief Probation Officers with practical

advice on how to successfully implement the JSES. This training will involve hands-on

workshops that include short knowledge building sessions and follow-up work sessions

where local JJSES issues are integrated into the curriculum exercises. This process will

make the training practical, in that it has real-time application and learnings that are

immediately transferrable to the JJSES effort.

JJSES Statement of Purpose

We dedicate ourselves to working in partnership to enhance the capacity of

Pennsylvania’s juvenile justice system to achieve its balanced and restorative

justice mission by:

• Employing evidence-based practices, with fidelity, at every stage of the

juvenile justice process;

• Collecting and analyzing the data necessary to measure the results of

these efforts; and, with this knowledge,

• Striving to continuously improve the quality of our decisions, services and

programs.

JJSES | STAGE ONE

9

Additional Resources.

None

Performance Measures

Forthcoming

JJSES | STAGE ONE

10

Cost Benefit Analysis Worksheet

Purpose: A department that makes a commitment to align its practice with research evidence

for risk reduction will likely experience significant challenges regarding how they invest their

resources. Certain functions and activities will need to be added, others dropped or modified.

Personnel requirements will likely change including how personnel are assigned, what staff

qualifications are needed, determining caseload and workload size, etc. In some cases,

activities and resources will need to be added such as those required when adopting screening

and assessment instruments or cognitive behavioral groups. Policies and practices with

stakeholders could change such as altering the format of predisposition reports to reflect

assessment information or modifying the expectations in service provider contracts. The cost-

benefit worksheet is designed to help administrators move ahead with their commitment toward

evidence-based practices with their “eyes wide open.” In this way, the department can plan for

resource shift or acquisition. It examines three primary areas: human, fiscal, and data

resources.

Instructions: It is recommended that the Chief Juvenile Probation Officer complete the cost

benefit analysis worksheet in conjunction with a team of department representatives (i.e.

supervisor, line juvenile probation officer, and support staff). Each representative will perceive

the items on the worksheet differently from their work experience perspective and will be able to

provide insight as to possible implications. It is important to fully explore these implications (i.e.,

costs and benefits) from each representative’s point of view, drawing upon lessons learned from

other counties and other jurisdictional experiences. If the department is just beginning to

embark on an EBP implementation process it may not be able to quantify some of the costs or

benefits. If this is the case, the department is urged to contact a Point of Contact (POC) to

request technical assistance.

As noted, it is likely not practical for most counties to quantify costs or benefits uniformly. An

urban county may view what is “costly” very differently than a rural county and vice versa. A

department whose caseload sizes are small might be able to transfer cases or staff more readily

than one with large caseloads. Therefore, the worksheet provides a subjective measure (“Low,

Moderate, High, or Not Applicable”) for both cost and benefit. The department will need to

determine what these terms mean to them. To assist in this interpretation, a brief description for

Low, Moderate, High or Not Applicable is provided below. Departments should consider only

local costs when completing the worksheet. While they may not be able to identify precise

dollar or time costs, it is important that departments enter an EBP planning process with full

awareness of what is involved and how it might impact resources.

The following table provides a definition of Cost and Benefit for purposes of completing the

cost/benefit worksheet.

JJSES | STAGE ONE

11

Cost (Human, Fiscal, and Data) Benefit (to Mission Outcomes) N/A

Low: Can be done with minimal

additional or existing resources

Description: The change will

have minimal or no impact

on a person(s) workload or

department budget

Low: Minimal or no real benefit to the

department’s mission will be achieved if

this activity is conducted

Description: The benefit is minimally

associated with the mission, or cannot

be articulated with clarity

Is perceived

to be not

applicable to

your

department’s

efforts to

implement

evidence-

based

practices.

Moderate: Will take some

additional resources, but current or

future resources or restructuring can

be used without requiring a

significant resource acquisition or

burden

Description: Can readily acquire

a grant or shift existing human

or fiscal resources

Moderate: Benefits can be articulated

but are not significantly contributory to

public safety/risk reduction or are

abstract and hard to quantify

Description: There is a

distinguishable benefit to the mission

and the benefit slightly outweighs

the cost

High: Will require a concerted effort

to bring on additional resources

Description: Will require the

securing of additional funding

from a source that has other

competing interests and/or the

approval of one or more

stakeholders to move

resources

High: Clearly, this activity will greatly

aid in achieving BARJ/JJSES outcomes

Description: It is very clear that the

benefit far outweighs the cost,

appears to be achievable, and would

deliver a message that the agency is

adopting an EBP approach

The cost benefit analysis team should gain a consensus regarding the cost and benefit of

implementing various EBPs. After the worksheet is completed, a discussion should ensue as to

whether the department is prepared to proceed with JJSES and, if so, how to plan for the

upcoming costs. If it is determined that the department is not prepared to proceed with

implementation of the JJSES, a plan should be developed to address what needs to occur

before the department is able to move forward with implementation.

JJSES | STAGE ONE

12

Stage One - Readiness (Prepare for JJSES Implementation)

Objective

Recommended Activities Costs

(low, moderate or high)

Benefit (low, moderate or

high) or N/A

Human

Fiscal

Data

The department

will have assessed

its organizational

readiness for EBP

and adequately

addressed any

shortcomings so

as to improve the

likelihood that a

change initiative

regarding JJSES

will succeed

-Conduct Likert readiness to change

survey

-Develop a response plan around

any item on the survey that showed

a significant gap score

-Secure a firm commitment in the

form of, for example, a letter of

support from leadership for JJSES

Statement of Purpose

The benefit should

be rated high if the

department has a

number of staff-

perceived issues

that are affecting

staff energy and

commitment (e.g.,

communication,

incompatibility with

mission, etc.)

A high knowledge

level about

evidence based

practices among

probation staff and

stakeholders will

be obtained

-Conduct intro to EBP training for

probation staff

-Train stakeholders, service

providers and victim advocates on

EBP

-Conduct pre/post EBP tests to

measure knowledge gained

This should be

rated high.

Without the

necessary

foundational

knowledge by all,

progress will not

be made.

The department

will develop a

detailed plan to

move ahead with

EBP

-Set up an action plan workgroup

(diagonal slice of department) and

complete an 18 month action plan

This should be

rated moderate. It

is very useful but

progress can be

made without it.

Begin process of

stakeholder

engagement

-Provide ongoing EBP education and

information to stakeholders

-Acquire a “letter of intent” to work

with the department on EBP from

key service providers

This should be

rated high (if

stakeholders are

reluctant to accept

JJSES changes;

moderate if less

reluctant).

JJSES | STAGE ONE

13

What is needed to accomplish Stage One?

Human

Resources

________________________________________________________________

______________________________________________________

Fiscal

Resources

________________________________________________________________

______________________________________________________

Data

Resources

________________________________________________________________

______________________________________________________

Stage Two – Initiation (Put in Place Assessment and Case Planning Processes)

Objective Recommended Activities

Costs (low moderate or high)

Benefit (low, moderate or

high) or N/A

Human Fiscal Data

Engage and

motivate the

juvenile

-Train all direct service staff on

Motivational Interviewing (MI) consistent

with recommended MI Training

protocols

-Conduct booster sessions and quality

assurance processes

High benefit. MI

is a core skill

needed to

improve

outcomes

Assess juveniles

for risk/need per

county policy

and at

recommended

intervals using

actuarial

screening and

assessment

instruments

Conduct

specialized

screening and

assessments, if

needed

-Train staff and utilize actuarial

screening and assessment instruments.

The instruments may include:

1. YLS/CMI

2. MAYSI~2

3. Detention Risk Assessment

-Assess juveniles per county policy and

at recommended intervals using

actuarial screening and assessment

instruments

-Establish a process to ensure

assessment inter-rater reliability

-Determine if additional screening and

assessment instruments are needed

(e.g., sex offender, domestic violence,

reading, learning disability, responsivity,

etc.) and, if so, provide staff training

and/or develop referral protocols with

providers of specialized assessments

High benefit.

Risk/need

assessments

(and the other

actuarial tools)

provide the

foundation by

which key

decisions are

made. The

same is true for

inter-rater

reliability

processes.

The additional

screening and

assessment

instruments are

of moderate

benefit.

JJSES | STAGE ONE

14

Develop a

comprehensive

diversion policy

for low risk

offenders

-Educate staff and stakeholders on the

“Guide to Developing Pre-Adjudication

Policy and Practice in PA”

-Establish protocols to divert

appropriate low risk offenders away

from the juvenile justice system

-Establish minimal supervision levels for

low risk offenders who remain involved

the juvenile justice system

High benefit.

Diversion

prevents

unnecessary

penetration into

the system and

retains probation

services for

higher risk

youth.

Complete case

plans on

medium and

high risk youth,

meeting SMART

guidelines

Address at least

the top three

criminogenic

needs prior to

discharge

Engage and

involve family in

case plan

-Train direct service staff on case

planning utilizing recommended case

plan template

-Create and distribute criminogenic

need response guidelines

-Conduct case audits

-Train staff on family involvement

strategies

High benefit.

Research has

shown that the

use of case

plans improves

outcomes. It

also provides

the department

with a means of

determining

whether the

criminogenic

needs are being

addressed.

Develop

professional

alliance

-Train direct service staff on

professional alliance (done in

conjunction with core competency

training)

-Administer the DRI-R, “Working

Alliance Inventory,” or exit survey

High benefit.

Without

professional

alliance, risk

reduction cannot

occur. The use

of the DRI, WAI,

or exit survey is

moderate

benefit.

Establish

caseload/

commensurate

with risk

-Develop policies to achieve caseload

size and supervision requirements

consistent with level of risk

High benefit.

JJSES | STAGE ONE

15

Prepare

supervisors for a

teaching and

coaching role

regarding EBP

-Train supervisors on effective coaching

and leadership on EBP with an

emphasis on the two competencies of

professional alliance and case planning

-Supervisors lead booster sessions on

the core competencies of professional

alliance and case planning

-Establish a “Transfer of Learning”

process

-Revise/align relevant policies and

human resource procedures to coincide

with stage changes

High benefit

(most of the

activities). One

of the key

reasons EBP

has not taken

hold in probation

departments is

because

supervisors have

not had the

knowledge or

comfort to

support and

guide staff.

Analyze key

data to monitor

progress

-Collect EBP process and outcome data

measures on Stage Two and distribute

in user friendly way

High benefit.

You cannot

know if you are

meeting

intended

outcomes

without

collecting the

data.

What is needed to accomplish Stage Two?

Human

Resources

_______________________________________________________________________

_______________________________________________________________________

Fiscal

Resources

_______________________________________________________________________

_______________________________________________________________________

Data

Resources

_______________________________________________________________________

_______________________________________________________________________

JJSES | STAGE ONE

16

Stage Three – Behavioral Change (Put in Place Processes That Reduce Rearrest)

Objective Recommended Activities Costs

(low, moderate or high)

Benefit (low, moderate or high)

or N/A

Human

Fiscal

Data

Youth learn skills

that help them stay

out of the justice

system

-Train staff on the skill practice and

rewards/sanctions competencies

-Train staff on the skill building

tools

-Train staff on cognitive behavioral

interventions

-Establish/expand cognitive

behavioral programming if needed

(recommend minimum of two CBT

groups - one for boys/one for girls)

-Develop strategies and

techniques to support gains made

by youth through cognitive

behavioral interventions (e.g.

Aggression Replacement Training,

Thinking for Change)

-Establish a QA process that

includes direct observation or tape

review of staff interactions

High benefit. While

community service

providers can

greatly assist in

achieving risk

reduction the

research is clear

that the one-on-one

sessions with

probation staff can

likewise achieve

these outcomes. In

addition, probation

staff can reinforce

the skills taught by

service providers.

Established a

structured decision

making process for

rewards and

sanctions

-Establish a planning process to

establish the research, values,

policies, and procedures for a

structured decision making

process (including stakeholder

collaboration)

Moderate benefit. It

definitely adds

value to outcomes

but perhaps slightly

less than the other

techniques noted

above.

Further engage

stakeholders

-Determine how court related

process may need altering (e.g.,

disposition report, admissions,

violations, discharges)

High to low benefit

depending on the

nature of existing

processes. If these

processes are in

alignment with

JJSES then the

JJSES | STAGE ONE

17

-Provide additional training if

needed

benefit would be

low; if not, it would

be high benefit.

Align community

based services with

EBP

-Conduct a service gap analysis

using risk/need information

-Enter into an arrangement to

improve and/or add contracted

services per the gap analysis

-Collaborate with providers

regarding SPEP scores and

program improvement plans

High to low benefit

depending on the

alignment of current

community based

services. High if

the effort would

improve services;

low if services are

largely in alignment

with EBP.

Prepare supervisors

for a teaching and

coaching role around

EBP

-Train supervisors on effective

coaching and leadership on skill

practice and rewards/sanctions

-Supervisors lead booster sessions

on the core competencies of skill

practice and rewards/sanctions

High benefit. See

above discussion of

supervisors’ role.

Establish an EBP

culture

-Provide artifacts (checklists,

posters, etc.) that support skill

practice and EBP activities

-Revise/align relevant policies and

human resource procedures to

coincide with stage changes

Moderate benefit.

Analyze key data to

monitor progress

-Collect EBP process and outcome

data measures on Stage Three

and distribute in user friendly way

High benefit. You

cannot know if you

are meeting

intended outcomes

without analyzing

the data.

What is needed to accomplish Stage Three?

Human

Resources

_______________________________________________________________________

_______________________________________________________________________

Fiscal

Resources

_______________________________________________________________________

_______________________________________________________________________

Data

Resources

_______________________________________________________________________

_______________________________________________________________________

JJSES | STAGE ONE

18

Stage Four – Refinement (to Solidify Department Use of Knowledge to Inform Decisions)

Objective Recommended Activities Costs

(low, moderate or high)

Benefit (low, moderate or high)

or N/A

Human

Fiscal

Data

Become an

enduring “Learning

Organization”

-Develop a plan to continuously

acquire and disseminate new

research findings

Moderate Benefit.

Refine existing

policies to ensure

alignment with EBP

-Ensure personnel are recruited

with the types of EBP

competencies the department

seeks - consistent with the 14

Professional Alliance Traits

-Ensure future department

training is directly linked to the

required EBP competencies

-Ensure mission statement,

policy and procedures, job

descriptions, performance

evaluations, promotions, and

staff rewards are linked to EBP

related performance (e.g.,

addressing criminogenic needs)

instead of process only (e.g.,

meeting contact standards)

High Benefit. Few

things are more

important than

ensuring that existing

staff attitudes/beliefs

and skills are in

alignment with the

tasks required under

EBP. A department

must also ensure that

its reward system and

other infrastructure

are consistent with

EBP or else staff will

tend to gravitate to

that which has

traditionally been

valued.

Ensure information

system capacity to

evaluate and

research local

practices

-Analyze information system for

needed improvements

-Develop and disseminate

scorecard for purpose of

individual, department, and

system wide feedback

(customized to the target

audience)

-Evaluate key programs for

effectiveness (possibly with local

University assistance)

High benefit. You

cannot know if you

are meeting intended

outcomes without

analyzing and

distributing the data.

JJSES | STAGE ONE

19

What is needed to accomplish Stage Four?

Human

Resources

_________________________________________________________________

_____________________________________________________

Fiscal

Resources

_________________________________________________________________

_____________________________________________________

Data

Resources

_________________________________________________________________

_____________________________________________________

JJSES | STAGE ONE

20

EBP Knowledge Pre/Post Test

Instructions: The EBP knowledge test is designed to be given prior to the Intro to EBP training

and immediately after the training. The same test is administered both times in order to

compare the number of questions answered correctly on the pretest to those answered on the

post test. Participants should be told to not put their names on the test, as it is anonymous. It is

more important that participants answer to the best of their ability and give a true picture of their

knowledge than for them to feel any pressure to “get it right.” The testing is designed to identify

- Whether the training had its intended impact (increased knowledge); and

- What knowledge gaps exist for which booster sessions may be necessary

Participants should be given the pretest before the trainer starts teaching. The facilitator or

trainer should explain the purpose for the testing and that it is anonymous. Test results should

be collected before the training begins. After the training ends, participants should be re-

administered the same test. At the end of the training, the facilitator or trainer should remind

participants to check the “post-test” box on the test to distinguish given answers from the

pretest. These post-tests should then be collected and the number of questions answered

correctly compared to those answered correctly on the pretest. As a group, scores should

improve after the training for each question that was answered correctly by less than 90% of the

pretest participants. The goal is to achieve an 80% or higher correct response rate for each test

question. If, after the pre and post-test results are analyzed, the scores do not meet the 80%

threshold the department should determine the best method of improving results. Some

possible responses include

Conduct a short EBP booster session(s) (e.g., 30 minutes)

Have staff read articles on the topic in need of improved knowledge

Have supervisors conduct coaching sessions individually or in staff meetings

Note: the pre and post-test should be administered as written. However,

under Part I: General, question B it asks the participant to indicate their job

category. This is provided to give the department an understanding of which

job functions might need additional assistance. You will need to fill in the

categories of job functions that match your department’s position titles.

JJSES | STAGE ONE

21

EBP Knowledge Pre/Post Test

This survey is anonymous; please do not include your name on the materials. Please select only one response per question.

Part I: General

A. Please indicate the point in time you are taking this survey:

a) Before the training (pre-training survey) b) After the training (post-training survey)

B. Your Job Category:

a) To be filled in: _______________________ b) To be filled in: _______________________ c) To be filled in: _______________________ d) To be filled in: _______________________

C. How long have you been employed by your current employer?

a) Up to one 1 year b) 1-3 years c) 4-10 years d) Over 10 years

Part II: Evidence­Based Practices For each of the questions below, please select one response only. 1. Which of the following interventions best reduces recidivism in the long term?

a) Boot camps b) Wilderness programs c) Escalating sanctions d) Cognitive behavioral programs

2. Which is not a criminogenic need?

a) Depression b) Anti­social peers c) Low victim empathy d) Unemployment

3. The lack of education is among the top four criminogenic needs.

a) True b) False

JJSES | STAGE ONE

22

4. Responsivity means: a) Being responsive to the needs of juveniles b) Using actuarial risk/need assessment tools c) Focusing on juvenile strengths d) Matching interventions to the individual traits of the juvenile

5. It is generally true that treatment can be effective, and therefore, it is more important to provide a lot of treatment than to be concerned with the type of treatment.

a) True b) False

6. Placing juveniles with low self-esteem in programs that increase their confidence is an effective way to reduce recidivism.

a) True b) False

7. It is generally true that most treatment does not work in reducing recidivism. Therefore, dispositions and case plans should focus on holding the juvenile accountable and restoring the crime victim rather than treatment.

a) True b) False

8. Which of the following programs works in reducing recidivism over the long term?

a) Gardening and horticulture b) Yoga c) Lectures designed to give insight d) AA e) None of the above

9. Which of the following program components contribute the least to reducing recidivism?

a) Juveniles rehearsing skills learned in programs b) Staff role modeling c) Positive reinforcement d) Programs designed to provide insight

10. Which is most important in lowering recidivism?

a) Lower workload for staff b) Providing staff with training and skills c) Getting more resources d) Use of external control

11. Mixed gender treatment groups:

a) Are effective for role modeling and controlled exposure to mixed gender issues b) Tend to produce more harmful than positive results c) Tend to produce neither harmful nor positive results d) None of the above

JJSES | STAGE ONE

23

12. Evidence-based practices are not used in the fields of: a) Medicine b) Nursing c) Corrections d) Plumbing e) None of the above

13. It is important to give juveniles positive reinforcement and feedback when they are pro­social if your goal is risk reduction.

a) True b) False

14. Swift and certain responses to acts of noncompliance are important if your goal is risk reduction.

a) True b) False

15. What is the number of positive reinforcing statements that must be given before an individual will take in a corrective comment?

a) 1-2 b) 2-4 c) 4-6 d) 7 or more

16. The use of a correctional institution for higher risk juveniles is effective in changing future illegal behavior after release if the confinement conditions are severe enough so that the juvenile doesn’t want to return.

a) True b) False

17. Placing a juvenile who has used or been involved with drugs in drug education classes is an effective method of reducing drug use in the future.

a) True b) False

18. Putting lower risk offenders in a cognitive behavioral class will most likely:

a) Help settle the class down by providing positive role models for the higher risk offenders b) Increase the lower risk offenders' recidivism rates

JJSES | STAGE ONE

24

EBP Knowledge Pre/Post Test

Answer Key

1. D

2. A

3. False

4. D

5. False

6. False

7. False

8. E

9. D

10. B

11. B or C

12. E

13. True

14. True

15. C

16. False

17. False

18. B

JJSES | STAGE ONE

25

Organizational Readiness Survey

Instructions: The organizational readiness survey was derived from the research of Rensis Likert. He is one of the founders of business management theory, a psychologist, and best remembered for his research in organizational management and the development of the Likert Survey Scale. Likert divided management styles into four systems which provide the overall structure for the Likert Organizational Climate Survey. The four systems described by Likert are: 1. Exploitive authoritative system In this type of management system, the job of employees/subordinates is to abide by the decisions made by managers and those with a higher status than them in the organization. The non-management staff do not participate in decision-making. The organization is concerned simply about completing the work. The organization will use fear and threats to make sure employees complete the work set. There is no teamwork involved. 2. Benevolent authoritative system Just as in an exploitive authoritative system, decisions are made by those at the top of the organization and management. However employees are motivated through rewards (for their contribution) rather than fear and threats. Information may flow from subordinates to managers but it is restricted to "what management wants to hear". 3. Consultative system In this type of management system, non-management staff are motivated by rewards and a degree of involvement in the decision making process. Management will constructively use employee ideas and opinions. However, involvement is incomplete and major decisions are still made by senior management. There is a greater flow of information (than in a benevolent authoritative system) from staff to management. The information from staff to manager is often incomplete and euphemistic. 4. Participative system Management has complete confidence in their employees. There is significant communication and staff are fully involved in the decision making process. Employees comfortably express opinions and there is lots of teamwork. Teams are linked together by people, who are members of more than one team. Likert calls people in more than one group "linking pins". Employees throughout the organization feel responsible for achieving the organization’s objectives. This responsibility is motivational, especially as staff are offered rewards for achieving organizational goals which they have participated in establishing. An organization needs to be high functioning in a limited number of areas in order for a change effort to be successful. Without attending to these areas, agencies are more likely to experience perfunctory change “on paper” instead of actual modified staff activities. The fourth system, participative, is described as the ideal system for human-concerned organizations. In this system, leadership has confidence in their staff, personnel at all levels feel real responsibility for organizational goals, there is strong communication, and a substantial amount of cooperative teamwork.

JJSES | STAGE ONE

26

The organizational readiness survey, then, is a tool to determine staff perception as to which of the four management systems the department operates. It does this by comparing what the staff member thinks is true of an ideal organizational environment versus what currently exists. Where there are large gaps, the department will want to develop a plan to address them. The survey should be administered anonymously. Carefully designed collection procedures should be put in place to ensure anonymity. Some departments use survey monkey, others use an outside agency, yet others use a “drop box.” The technique employed should be determined based on the level of concern around anonymity and department history around trust and use of surveys. The surveys should be compiled in such a way as to identify “gap scores.” That is, the questions have a range of four answers. The largest gaps (which occur when the respondent indicates that the first choice represents the current situation and the fourth choice represents the idea situation) should be attended to first. To quantify gap scores, the first answer is given a numerical value of 1, followed by 2, 3, and then 4. Gaps that approach two points (or higher) should be attended to.

JJSES | STAGE ONE

27

Organizational Readiness Survey

For the following questions, address both your perception of the current situation in the organization, as well as your perception of what the ideal for this organization would be. Please select one response only for each the current situation and the ideal situation by circling your perspective of the “current” and “ideal” situation. Example: How much confidence and trust is shown in staff members by leaders in the department?

Current Situation

Ideal Situation

1. How much confidence and trust is shown in staff members by leaders in the department?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________

1

Virtually None

2

Some

3

A Substantial Amount

4

A Great Deal

1

Virtually None

2

Some

3

A Substantial Amount

4

A Great Deal

1

Virtually None

2

Some

3

A Substantial Amount

4

A Great Deal

1

Virtually None

2

Some

3

A Substantial Amount

4

A Great Deal

JJSES | STAGE ONE

28

2. How free do staff members feel to talk to leaders about their job?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 3. How often are staff members' ideas sought and used constructively?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 4. How predominant is the use of fear, threats, or punishment?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________

1

Not Very Free

2

Somewhat Free

3

Quite Free

4

Very Free

1

Not Very Free

2

Somewhat Free

3

Quite Free

4

Very Free

1

Seldom

2

Sometimes

3

Often

4

Very Frequently

1

Seldom

2

Sometimes

3

Often

4

Very Frequently

1

Very Frequently

2

Often

3

Sometimes

4

Seldom

1

Very Frequently

2

Often

3

Sometimes

4

Seldom

JJSES | STAGE ONE

29

5. How predominant is the use of rewards and incentives?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 6. Where is the responsibility felt for achieving the department's goals?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 7. How much cooperative teamwork exists in the department?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________

1

Seldom

2

Sometimes

3

Often

4

Very Frequently

1

Seldom

2

Sometimes

3

Often

4

Very Frequently

1

Mostly at the Top

2

Top and Middle Management

3

Line Level

4

At All Levels

1

Mostly at the Top

2

Top and Middle Management

3

Line Level

4

At All Level

1

Very Little Cooperative Teamwork Exists

2

Cooperative Teamwork Occurs on Situational

Basis

3

Cooperative Team work Occurs Among Select

Groups of Staff

4

Ongoing and Consistent Cooperative Teamwork Exists Among All Staff

1

Very Little Cooperative Teamwork Exists

2

Cooperative Teamwork Occurs on Situational Basis

3

Cooperative Teamwork Occurs Among Select

Groups of Staff

4

Ongoing and Consistent Cooperative Teamwork

Exists for All Staff

JJSES | STAGE ONE

30

8. What is the usual direction of information flow?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 9. How is downward communication accepted/received by staff members?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 10. How accurate is upward communication?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________

1

From Top to Bottom

2

From Bottom to Top

3

Sideways (Only Within Same Hierarchal Level)

4

In All Directions

1

From Top to Bottom

2

From Bottom to Top

3

Sideways (Only Within Same Hierarchal Level)

4

In All Directions

1

Negatively

2

With Suspicion, but not necessarily

negative

3

Neutrality

4

With a Receptive Mind

1

Negatively

2

With Suspicion, but not necessarily

negative

3

Neutrality

4

With a Receptive Mind

1

Usually Inaccurate

2

Often Inaccurate

3

Often Accurate

4

Almost Always Accurate

1

Usually Inaccurate

2

Often Inaccurate

3

Often Accurate

4

Almost Always Accurate

JJSES | STAGE ONE

31

11. How well do leaders in the department understand the problems faced by staff

members?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 12. At what level are decisions affecting the entire department made?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 13. Are staff members involved in decisions related to their work?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________

1

Poorly

2

Not Very Well

3

Well

4

Very Well

1

Poorly

2

Not Very Well

3

Well

4

Very Well

1

Always at the Top without Staff Input

2

Always at the Top/Middle without

Staff Input

3

At the Top/Middle with Staff Input

4

Throughout the Organization

1

Always at the Top without Staff Input

2

Always at the Top/Middle without

Staff Input

3

At the Top/Middle with Staff Input

4

Throughout the Organization

1

Almost Never

2

Occasionally Consulted

3

Generally Consulted

4

Fully Involved

1

Almost Never

2

Occasionally Concsulted

3

Generally Consulted

4

Fully Involved

JJSES | STAGE ONE

32

14. How are the department's goals established?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 15. How much covert resistance to the department's goals is present?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________ 16. Are there informal groups of staff resisting the direction/goals of the department?

Current Situation

Ideal Situation

Comments:___________________________________________________________________________________________________________________________________________________________________________________________________________________________

1

Orders are Issued Without Asking for Input

2

Orders are Issued With Limited Input Invited

3

Management Decides With Staff Paricipation

4

By Group Action (Except in Crisis Situations)

1

Orders are Issued Without Asking for Input

2

Orders are Issued With Limited Input Invited

3

Management Decides With Staff Participation

4

By Group Action (Except in Crisis

Situations)

1

Strong Resistance

2

Moderate Resistance

3

Some Resistence At Times

4

Little or No Resistance

1

Strong Resistance

2

Moderate Resistance

3

Some Resitance At Times

4

Little or No Resistance

1

Yes, Most of the Time

2

Frequently

3

Sometimes, but Infrequently

4

No, or Almost Never

Yes, Most of the Time

2

Frequently

3

Sometimes, but Infrequently

4

No, or Almost Never

JJSES | STAGE ONE

33

17. How is data most commonly used in the department?

Current Situation

Ideal Situation

Comments:______________________________________________________________________________________________________________________________________________________________________________________________________________

1

To Admonish/Discipline/

Punish Staff

2

To Identify Areas in Need of Improvement

3

To Take Constructive Corrective Action If

Needed

4

To Reward/Incentivize Staff

1

To Admonish/Discipline/

Punish Staff

2

To Identify Areas in Need of Improvement

3

To Take Constructive Corrective Action If

Needed

4

To Reward/Incentivize Staff

JJSES | STAGE ONE

34

Action Planning

Instructions: The department can use any of a number of action planning templates to

develop their EBP action plan. The attached templates are available for those who do not have

an existing, preferred form. It is recommended that the department complete an eighteen

month action plan after the other Stage One activities have been completed. The following are

recommended steps:

One Assemble an action planning team made up of no less than five staff members

and no more than fifteen. Make sure that the team represents a cross section of

staff functions including but not limited to, management, intake, supervision staff,

and service coordinators. Subcommittees may be formed.

Two Assign a chairperson. The chair should be someone that the other members

respect, knows how to facilitate a meeting, has good follow through, is objective,

is inclusive, and does not have an “agenda.”

Three Develop clear expectations of the group including, for example, length of time it is

to meet, objectives, communication protocol, decision making authority, etc. In

some departments, a committee charter is developed.

Four Create a staff input plan. If all staff cannot participate in the action planning

workgroup, determine the best method to gain staff input. Some ways might

include surveys, focus groups, topic specific invitees to meetings,

subcommittees, or asking for input on draft documents.

Five Determine, based on the JJSES Monograph, what goals, objectives and activities

should be accomplished in the next eighteen months that will ensure that the

department is achieving its JJSES objective. The plan should include the

person(s) taking the lead on each activity and timelines.

Six Use JJSES technical assistance to develop the plan, as needed.

Seven Write the plan and disseminate it to the entire department.

JJSES | STAGE ONE

35

Action Plan Templates

Example A: Goal 1: Utilize validated screening and assessment instruments to identify risk and need of juvenile offenders and to develop case plans that match the appropriate level of intervention.

What should be done?

Objectives

How will it be done?

Strategies or Activities

Who will take the lead?

Date objective to be done?

How will we know it’s done?

Outcome Measure A:

B:

C:

JJSES | STAGE ONE

36

Example B:

Goal: To increase juvenile and family and family engagement through strength-based techniques.

Objective 1:

Action Item Responsibility Deadline Completed

a)

b)

c)

d)

Objective 2:

Action Item Responsibility Deadline Completed

a)

b)

c)

d)

Objective 3:

Action Item Responsibility Deadline Completed

a)

b)

c)

d)

Goal:

Objective 1:

Action Item Responsibility Deadline Completed

a)

b)

c)

d)

Objective 2:

Action Item Responsibility Deadline Completed

a)

b)

c)

d)

Objective 3:

Action Item Responsibility Deadline Completed

a)

b)

c)

d)

JJSES | STAGE ONE

37

Example C:

Action Plan Template

Purpose: To create a “script” for your JJSES efforts and support implementation. Directions: 1. Use this form as a template, develop an action plan for each goal identified.

2. Modify the form as needed to fit your unique context. 3. Distribute copies of each action plan to the members of the department. 4. Keep copies handy to bring to meetings to review and update regularly. You may

decide to develop new action plans for new phases of your JJSES efforts.

Goal: Identify and target proven and effective interventions based on criminogenic risk and needs of juvenile offenders. Results/Accomplishments:

Action Steps

What Will Be Done?

Responsibilities

Who Will Do It?

Timeline

By When?

(Day/Mo)

Resources

A. Resources Available

B. Resources Needed (financial, human, political & other)

Potential Barriers

A. What individuals or organizations might resist?

B. How?

Communications Plan

Who is involved? What methods?

How often?

Step 1:

A.

B.

A. B.

Step 2:

A.

B.

A. B.

Step 3:

A.

B.

A. B.

Step 4:

A.

B.

A. B.

Evidence Of Success (How will you know that you are making progress? What are your benchmarks?) Evaluation Process (How will you determine that your goal has been reached? What are your measures?)