Reforming California’s Juvenile Reforming California’s Juvenile Corrections System: An Corrections System: An Evidence-Based Change Evidence-Based Change

Strategy*Strategy*

James M. Byrne, Ph.D. Professor James M. Byrne, Ph.D. Professor

Department of Criminal JusticeDepartment of Criminal Justice

University of Massachusetts, Lowell University of Massachusetts, Lowell

*Plenary presentation at the Transfer of Knowledge Workshop, “Parole Re-Entry: A New Vision”, sponsored by The Division of Juvenile Justice Parole Operations, Nov. 1,2005 Pomona, California

What is Evidence-based What is Evidence-based Practice?Practice?

It is the development and implementation of programs based It is the development and implementation of programs based on a systematic review of “what works”on a systematic review of “what works”

There are three basic approaches to Evidence-based There are three basic approaches to Evidence-based practice:practice:

Strategy 1: Conduct a comprehensive review of Strategy 1: Conduct a comprehensive review of allall available research on a particular topic:available research on a particular topic:

Examples:Examples: The systematic reviews conducted by the Campbell The systematic reviews conducted by the Campbell

Collaboration Crime and Justice Group (Sherman et. al, Collaboration Crime and Justice Group (Sherman et. al, 2005; Sherman, et. al, 1997)2005; Sherman, et. al, 1997)

The systematic reviews using meta-analytic methods The systematic reviews using meta-analytic methods including experimental and quasi-experimental research including experimental and quasi-experimental research (Gendreau, et. al, 1990)(Gendreau, et. al, 1990)

What is Evidence-based What is Evidence-based Practice? (continued)Practice? (continued)

Strategy 2: Examine only a subset of all available research Strategy 2: Examine only a subset of all available research studies, using randomized field experiments as the “Gold studies, using randomized field experiments as the “Gold Standard”, e.g. Farrington and Welsh’s recent review of all Standard”, e.g. Farrington and Welsh’s recent review of all randomized experiments (2005)randomized experiments (2005)

Strategy 3: Conduct a nonscientific review, simply say Strategy 3: Conduct a nonscientific review, simply say “evidence based”, and then offer your own listing of best “evidence based”, and then offer your own listing of best practices.practices. Reexamine/reposition scientific reviewsReexamine/reposition scientific reviews Only include a subset of all available research, often supporting Only include a subset of all available research, often supporting

either liberal or conservative ideology (Farabee, 2005; Cullen, either liberal or conservative ideology (Farabee, 2005; Cullen, 2002)2002)

No specific identification of review procedures, inclusion/exclusion No specific identification of review procedures, inclusion/exclusion criteria, etc.criteria, etc.

What is Meta-Analysis?What is Meta-Analysis?

Meta-Analysis is a statistical technique designed to Meta-Analysis is a statistical technique designed to synthesize empirical relationships across a large number of synthesize empirical relationships across a large number of studies.studies.

Meta-Analysis allows us to determine both the Meta-Analysis allows us to determine both the SizeSize and and DirectionDirection of effects across studies. of effects across studies.

Meta-Analysis techniques can be translated into summary Meta-Analysis techniques can be translated into summary statistics that public policy makers can easily understand statistics that public policy makers can easily understand (e.g. a 10% recidivism reduction effect for all correctional (e.g. a 10% recidivism reduction effect for all correctional interventions using multi-systemic strategies) interventions using multi-systemic strategies)

Sources:Sources: Wilson (2001); Rossi, Freeman, and Lipsey (2001) Wilson (2001); Rossi, Freeman, and Lipsey (2001)

When Should Meta-Analysis be When Should Meta-Analysis be Used?Used?

Meta-Analysis should only be used to summarize Meta-Analysis should only be used to summarize research findings from methodologically rigorous research findings from methodologically rigorous evaluation designs.evaluation designs.

Meta-Analysis should not be used when there are Meta-Analysis should not be used when there are only a small number of studies (10 or less) being only a small number of studies (10 or less) being included in the review.included in the review.

Meta-Analysis is most appropriate for studies Meta-Analysis is most appropriate for studies examining the relationship between clearly examining the relationship between clearly defined independent and dependent variables.defined independent and dependent variables.

What Review Criteria are Used What Review Criteria are Used in The Campbell in The Campbell

Collaborative’s Systematic Collaborative’s Systematic Reviews?Reviews?

Study Inclusion Criteria For Systematic ReviewsStudy Inclusion Criteria For Systematic Reviews

The scientific methods scale ranks evaluation studies from 1=weakest to 5=strongest on The scientific methods scale ranks evaluation studies from 1=weakest to 5=strongest on overall internal validity:overall internal validity:

What Works:What Works: For a program to be classified as working, there For a program to be classified as working, there must be a minimum of two level 3 studies with significance tests must be a minimum of two level 3 studies with significance tests showing effectiveness and the preponderance of evidence in the showing effectiveness and the preponderance of evidence in the same direction.same direction.

What Does Not Work:What Does Not Work: For a classification of not working, For a classification of not working, there must be a minimum of two level 3 studies with significance there must be a minimum of two level 3 studies with significance tests showing ineffectiveness and the preponderance of tests showing ineffectiveness and the preponderance of evidence in the same direction.evidence in the same direction.

What is Promising: What is Promising: For the classification of promising, at least For the classification of promising, at least one level 3 study is required with significance tests showing one level 3 study is required with significance tests showing effectiveness and preponderance of evidence in support of the effectiveness and preponderance of evidence in support of the same conclusion.same conclusion.

What is Unknown: What is Unknown: Any program not classified in one of the Any program not classified in one of the threethree above categories is considered to have unknown effects.above categories is considered to have unknown effects.

Source:Source: Welsh and Farrington, (2003: 169-170) Welsh and Farrington, (2003: 169-170)

Criteria for Assessing The Criteria for Assessing The Quality of Evaluation Studies:Quality of Evaluation Studies:

Level 5: Well-designed randomized field experimentLevel 5: Well-designed randomized field experiment

Level 4: Randomized field experiment of moderate Level 4: Randomized field experiment of moderate quality; well-designed quasi-experimentquality; well-designed quasi-experiment

Level 3: Quasi-experimental designs (moderate Level 3: Quasi-experimental designs (moderate quality)quality)

Level 2: Quasi-experimental designs (minimal quality)Level 2: Quasi-experimental designs (minimal quality)

Level 1: Non-experimental designsLevel 1: Non-experimental designs

How do Systematic Reviews How do Systematic Reviews (and Meta-Analyses) help us (and Meta-Analyses) help us answer the following public answer the following public

policy question:policy question:

Should current sentencing and Should current sentencing and corrections policies – both institution-corrections policies – both institution-based and community-based – focus on based and community-based – focus on offender rehabilitation or offender offender rehabilitation or offender control?control?

The evidence The evidence in favor of rehabilitationin favor of rehabilitation is is found in systematic reviews of correction found in systematic reviews of correction research that estimate that the provision of research that estimate that the provision of treatment (in sufficient dosages and treatment (in sufficient dosages and duration) is cost-effective and results in duration) is cost-effective and results in modest offender change (10% reduction)modest offender change (10% reduction)

The evidence The evidence opposed to rehabilitationopposed to rehabilitation is found in these same systematic reviews, is found in these same systematic reviews, which reveal that the vast majority of which reveal that the vast majority of individual research studies do not find individual research studies do not find statistically significant differences between statistically significant differences between experimental and control groups in experimental and control groups in recidivism.recidivism.

Current evidence-based Current evidence-based reviews highlight the reviews highlight the limitations inherent in limitations inherent in offender-based change offender-based change

strategies:strategies: Only Only incremental, incremental, short-term changes in offender short-term changes in offender behavior should be expected from the full behavior should be expected from the full implementation of evidence-based practices in implementation of evidence-based practices in adult and juvenile correctionsadult and juvenile corrections

If we are interested in long-term offender change, If we are interested in long-term offender change, we need to focus our attention on the community we need to focus our attention on the community context of offender behaviorcontext of offender behavior

There is a growing body of research on the need There is a growing body of research on the need to integrate individual and community-level to integrate individual and community-level change strategies (Sampson, et. al. 2005; Bursik, change strategies (Sampson, et. al. 2005; Bursik, 2005; Carr, 2003)2005; Carr, 2003)

The Need for Reform In The Need for Reform In California’s Juvenile California’s Juvenile Corrections System:Corrections System:

The Youth Incarceration RateThe Youth Incarceration Rate in California is among the in California is among the highesthighest in the country in the country

Institutional costsInstitutional costs represent a significant proportion (75%) represent a significant proportion (75%) of all juvenile corrections expenditures (400 million in FY ’04)of all juvenile corrections expenditures (400 million in FY ’04)

InstitutionalizationInstitutionalization is having a negative effect on both is having a negative effect on both juvenile offenders and on the communities in which juvenile juvenile offenders and on the communities in which juvenile offenders resideoffenders reside

Recycling/Churning:Recycling/Churning: Juvenile offenders released from CYA facilities are being Juvenile offenders released from CYA facilities are being

rearrested and returned to prison/CYA at unacceptably rearrested and returned to prison/CYA at unacceptably high rateshigh rates

Control-based community supervision strategies have Control-based community supervision strategies have failed to produce the desired recidivism reduction effects, failed to produce the desired recidivism reduction effects, either short-term or long-termeither short-term or long-term

Proposed Juvenile Correction Expenditures in California (400.2 Million Dollars)

Parole Services and Community

Corrections 9%

Education Services 10%

DJJ Board 1%

Administration 7%

Distributed Administration -7%

Institutions and Camps 66%

Institutions and Camps

Parole Services andCommunity Corrections

Education Services

DJJ Board

Administration

DistributedAdministration

Institutions and Camps $310,525

Parole Services and Community Corrections $41,209

Education Services $44,522

DJJ Board $3,251

Administration $31,040

Distributed Administration ($30,347)

Total $400,200

Proposed Juvenile Correction Expenditures

Source: Nadel-Hayes and Macallair, 2005: 32

Three Basic Organizational Three Basic Organizational Change Strategies:Change Strategies:

Empirical Rational Strategies:Empirical Rational Strategies: based on the based on the notion that information/education can lead to notion that information/education can lead to behavior changebehavior change

Normative Reeducation Strategies:Normative Reeducation Strategies: based on based on the notion that attitudes/values must change the notion that attitudes/values must change beforebefore behavior will change behavior will change

Power Coercive Strategies:Power Coercive Strategies: based on the based on the notion that formal, coercive, sanction-based notion that formal, coercive, sanction-based strategies can induce behavior changestrategies can induce behavior change

Common Characteristics of Common Characteristics of Successful Normative Successful Normative Reeducative Change Reeducative Change

Strategies:Strategies: At The Individual Level:At The Individual Level:

Cognitive RestructuringCognitive Restructuring Emphasize Positive Lifestyle Change Emphasize Positive Lifestyle Change (Employment, (Employment,

Marriage, Relocation)Marriage, Relocation) Focus on Informal Social ControlsFocus on Informal Social Controls

At The Organizational Level:At The Organizational Level: Leadership, Partnership, and OwnershipLeadership, Partnership, and Ownership Evidence-Based Strategic Plan Evidence-Based Strategic Plan Focus on Culture Change Focus on Culture Change (Moral Performance)(Moral Performance) New Ways of Doing Business: New Ways of Doing Business: Performance MeasurementPerformance Measurement

The Challenges for Community The Challenges for Community Corrections Managers Involved Corrections Managers Involved In Large-Scale Organizational In Large-Scale Organizational

ChangeChange1.1. DesignDesign a reform strategy that is a reform strategy that is defensibledefensible – –

based on “best practices”/evidence-based based on “best practices”/evidence-based reviewsreviews

2.2. Implement Implement a reform strategy that is a reform strategy that is compatible compatible with the values, skills, and capacity with the values, skills, and capacity of line staff (and Mgt.)of line staff (and Mgt.)

3.3. Evaluate Evaluate a reform strategy from the outset, a reform strategy from the outset, based on the premise that “what gets measured based on the premise that “what gets measured gets done”gets done”

Resistance to Change in Resistance to Change in Community Corrections:Community Corrections:

A TypologyA Typology ConformistsConformists – Accept organization’s goals and follow – Accept organization’s goals and follow

standard policies and proceduresstandard policies and procedures

RitualistsRitualists – Reject organization’s goals but still follow – Reject organization’s goals but still follow policies and procedures, often at minimal compliance policies and procedures, often at minimal compliance levellevel

Artful DodgersArtful Dodgers – May or may not accept organization’s – May or may not accept organization’s goals but do not follow existing policies and procedures goals but do not follow existing policies and procedures (catch me if you can)(catch me if you can)

Loose CannonsLoose Cannons – May or may not accept organization’s – May or may not accept organization’s goals but often directly challenge leadership on policies goals but often directly challenge leadership on policies and procedures (you can not touch me)and procedures (you can not touch me)

A Research Agenda for California’s A Research Agenda for California’s Division of Juvenile Parole OperationsDivision of Juvenile Parole Operations

1.1. Survey line staff on attitudes – Focus on organizational Survey line staff on attitudes – Focus on organizational change initiatives/culture/resistance to changechange initiatives/culture/resistance to change

2.2. Conduct workload study based on newly defined roles Conduct workload study based on newly defined roles and responsibilities for line staff and responsibilities for line staff andand management management

3.3. Examine the reliability and validity of classification Examine the reliability and validity of classification system (e.g. inter-rater reliability; prediction accuracy)system (e.g. inter-rater reliability; prediction accuracy)

4.4. Monitor compliance with new policies and procedures Monitor compliance with new policies and procedures using a clearly articulated performance measurement using a clearly articulated performance measurement systemsystem

5.5. Evaluate Evaluate bothboth the implementation and impact of the implementation and impact of specific strategies (and programs) using level 3 (and specific strategies (and programs) using level 3 (and higher) research designs (i.e. experiments and quasi-higher) research designs (i.e. experiments and quasi-experiments)experiments)

NIC’s Eight Principles for The Utilization NIC’s Eight Principles for The Utilization of Evidence-Based Practices in of Evidence-Based Practices in

Community Corrections:Community Corrections:1.1. Assess Actuarial Risk/NeedsAssess Actuarial Risk/Needs2.2. Enhance Intrinsic MotivationEnhance Intrinsic Motivation3.3. Target InterventionsTarget Interventions

a.a. Risk Principle:Risk Principle: Prioritize supervision and treatment resources Prioritize supervision and treatment resources for higher risk offendersfor higher risk offenders

b.b. Need Principle:Need Principle: Target interventions to criminogenic needs Target interventions to criminogenic needsc.c. Responsivity Principle:Responsivity Principle: Be responsive to temperament, Be responsive to temperament,

learning style, motivation, culture, and gender when assigning learning style, motivation, culture, and gender when assigning programs.programs.

d.d. Dosage:Dosage: Structure 40-70% of high-risk offenders’ time for 3-9 Structure 40-70% of high-risk offenders’ time for 3-9 monthsmonths

e.e. Treatment:Treatment: Integrate treatment into the full Integrate treatment into the full sentence/sanction requirementssentence/sanction requirements

4.4. Skill Train with Directed Practice (use Cognitive Behavioral Skill Train with Directed Practice (use Cognitive Behavioral treatment methods)treatment methods)

5.5. Increase Positive ReinforcementIncrease Positive Reinforcement6.6. Engage Ongoing Support in Natural CommunitiesEngage Ongoing Support in Natural Communities7.7. Measure Relevant Processes/PracticesMeasure Relevant Processes/Practices8.8. Provide Measurement Feedback Provide Measurement Feedback

In-Prison Meeting

with

Community Meeting

with

Police/Community Reintegration

Team

Housing CommunityJob

Family Victim

Outreach

Sign Behavioral Contract

Reentry Partnership ContinuumReentry Partnership Continuum

Classification for Risk,

Treatment, and Control

Move Target Population to

Select Facility

Treatment in Prison

Change

Education

Job

Family

Drug/

AlcoholTreatment

In-Community

Change

Education

Job

Family

Drug/

Alcohol

Surveillance ● Services ● Outreach

Structured

Reentry Institution

6 months

Community/Reintegration

The moral performance of institutional and The moral performance of institutional and community corrections needs to be community corrections needs to be

examined along with cost, and offender examined along with cost, and offender change.change.

Redefining The Criteria for A Successful Juvenile Corrections

System

Defining The Moral Performance of Defining The Moral Performance of Prisons:Prisons:

RelationshipsRelationships:: RespectRespect HumanityHumanity RelationshipsRelationships TrustTrust SupportSupport Social Structure:Social Structure: Power/authorityPower/authority Social relationsSocial relations

Regime:Regime: FairnessFairness OrderOrder SafetySafety Well-beingWell-being Personal DevelopmentPersonal Development Family ContactFamily Contact DecencyDecency

Other:Other: MeaningMeaning Quality of lifeQuality of life

Source Adapted From: Allison Liebling (2005)

Cullen’s “Liberal but not stupid” Cullen’s “Liberal but not stupid” approach approach to crime controlto crime control

(1) This approach is informed by a general belief that (1) This approach is informed by a general belief that people’s choices are bounded by social constraints, people’s choices are bounded by social constraints, including constraints that are rooted in social inequalities—including constraints that are rooted in social inequalities—with many such inequalities tied to economic processes with many such inequalities tied to economic processes that may produce wealth but also have untoward, if not that may produce wealth but also have untoward, if not unfair, consequences.unfair, consequences.

(2) Knowing that harsh living conditions are implicated in (2) Knowing that harsh living conditions are implicated in crime, it is irresponsible not to pursue social welfare crime, it is irresponsible not to pursue social welfare policies that ameliorate these disadvantages.policies that ameliorate these disadvantages.

(3) Within the criminal justice system—a system that will (3) Within the criminal justice system—a system that will affect millions of lives annually—it is irresponsible not to affect millions of lives annually—it is irresponsible not to pursue policies that improve the plight of offenders, pursue policies that improve the plight of offenders, victims, and potential future victims.victims, and potential future victims.

Cullen’s “Liberal but not stupid” Cullen’s “Liberal but not stupid” approach approach to crime control (continued)to crime control (continued)

(4) For those with progressive sentiments, the key issue is (4) For those with progressive sentiments, the key issue is not not nonintervention nonintervention but rather the but rather the quality of the quality of the intervention intervention that takes place in the criminal justice system.that takes place in the criminal justice system.

(5) Beyond the values that inform policy choices, criminal (5) Beyond the values that inform policy choices, criminal justice interventions should not be based on prejudice, justice interventions should not be based on prejudice, custom, or fashion but on “what works”—on what the data custom, or fashion but on “what works”—on what the data show are the most effective policies to pursue.show are the most effective policies to pursue.

(6) Liberal welfare approaches to crime control that are (6) Liberal welfare approaches to crime control that are revealed to be ineffective should not be defended but revealed to be ineffective should not be defended but relinquished.relinquished.

Concluding CommentsConcluding Comments

The focus of the The focus of the nextnext wave of adult and juvenile wave of adult and juvenile corrections programs should be on addressing the corrections programs should be on addressing the problems of problems of both both high-risk offenders and high-risk high-risk offenders and high-risk communitiescommunities

At minimum, correctional interventions should be At minimum, correctional interventions should be designed to designed to do no harmdo no harm, either to offenders or , either to offenders or communitiescommunities

It is not just adult and juvenile offenders that need to It is not just adult and juvenile offenders that need to change;change;

It is the It is the moral performancemoral performance of corrections – both of corrections – both institutions and community corrections – that needs to institutions and community corrections – that needs to change as well.change as well.