washington state’s experience with research-based juvenile justice programs may 19, 2005 robert...
TRANSCRIPT
Washington State’sWashington State’sExperience With Research-Based Experience With Research-Based
Juvenile Justice ProgramsJuvenile Justice Programs
May 19, 2005
Robert Barnoski, Ph.D.
(Barney)
Washington State Institute for Public Policy
www.wsipp.wa.gov
2
Washington State Institute for Public PolicyWashington State Institute for Public Policy
Created in 1983 by the state Legislature
Mission: Carry out non-partisan research on projects assigned either by the legislature or the Institute’s Board of Directors
8 legislators 4 higher education provosts or presidents 4 state agency directors
3
What We Will Talk About Today
This presentation describes Washington State's experiences implementing research-based programs
Some History
Outcome Evaluations
Quality Assurance
Cost Benefit
Cost Benefit Meta-Analysis
4
Juvenile Justice in Washington State
• Determinant sentencing since 1977; based on prior record and current offense
• Two systems: county vs. state– Each county runs juvenile court, detention, and probation– State runs juvenile correctional institutions, parole, and
distributes state funds to counties (Juvenile Rehabilitation Administration – JRA)
• Juvenile courts also manage non-offender cases: dependency, youth-at-risk, truancy, etc.
5
History of Legislation1995 Funding of intensive probation program.1996 Preliminary outcomes are not good.
1997 Determine if research-based programs exist and can be implemented in Washington State juvenile courts.Eliminate parole but establish intensive parole in JRA.
1998 Switch funding from intensive probation to research-based programs in juvenile courts.
Re-establish parole in JRA.
2002 Preliminary research-based program outcomes are positive and point to competent delivery.Parole and intensive parole outcomes are not good.
2003 Develop adherence and outcome standards for research-based programs.JRA moves toward research-based and Functional Family Parole (FFP).
Cost-benefit analyses of prevention and intervention programs beyond juvenile offenders.
6
1997: Community Juvenile Accountability Act (CJAA) Legislature said it would provide funding if research-based
programs could be identified
State, juvenile court administrators, and the Institute met regularly to identify “research-based” programs
Six programs were identified, four were recommended
1998: Legislature funded the programs ~$3.3 million/year
1999: Programs started
2001: Evaluation groups filled
2003: Evaluation completed
Initiation of Research-Based Effort in Washington State
7
1. Find research-based programs scientifically shown to work
2. Develop an assessment to identify the most appropriate program for each youth
3. Implement quality assurance to ensure services are delivered as designed
4. Conduct a valid outcome evaluation
Four Things Were Needed to Implement Research-Based Programs
8
Functional Family Therapy (FFT)
Aggression Replacement Training (ART)
Coordination of Services
Multi-Systemic Therapy (MST)
Multidimensional Therapeutic Foster Care
Adolescent Diversion
First Thing: Research-Based Program List
9
Functional Family Therapy($2,100 per youth)
• Blueprint Program: University of Colorado’s Center for the Study and Prevention of Violence
• Structured family-based intervention to enhance protective and reduce risk factors in the family
• FFT is a three-phase program Motivate family toward change Teach family how to change a specific critical problem Help family generalize their problem-solving skills
• Trained therapists have caseloads of 10 to 12 families
• Involves about 12 visits during a 90-day period
10
Aggression Replacement Training($700 per youth)
• Group training of 10 youth by two instructors
• Classes meet for one hour three times per week for 10 weeksAggression cycle Skill developmentMoral reasoning
• Training, not therapy, that uses guided discussion, modeling, role play, and homework
11
Coordination of Services($400 per youth)
• Educational program for low-risk juvenile offenders and their parents. The goals: Describe the consequences of delinquent behavior Stimulate goal setting Review the strengths of youth and family Explain what resources are available
• Participants given vehicle to open lines of communication and make shifts in thinking
• Community groups present participants with information concerning services they provide
12
Pre-screen for level of risk – more intensive service to higher risk youth and families; minimal for low risk
Comprehensive assessment only for higher risk youth – 10 life domains of risk and protective factors
Motivational interview to engage youth and family
Re-assess dynamic factors for progress – individual and groups
Common language for talking about youth and families
Second Thing: Build the Washington Juvenile Court Assessment
13
Mapping Problem to Intervention for Moderate to High Risk Youth
Problem Domain Intervention
Family (Primary) Functional Family Therapy (FFT), Multi-Systemic Therapy (MST), and Mentoring
No Family Multi-Dimensional Treatment Foster Care
Aggression Aggression Replacement Training (ART)
Impulsive Dialectical Behavior Therapy (DBT) or ART
Mental Health DBT or Family Integrated Treatment (FIT)
Sexual Behavior ? - ART, DBT, FIT
Alcohol/Drugs ? - ART, DBT, FIT
Attitudes ART, DBT
Skills ART, DBT
School, Employment, Free Time, Relationships
14
Case Management:Sequence to Self Reliance
If family has problems:Family =>
Engage & motivate family => Specific Problems
=> GeneralizationIf no family problems:Engage & motivate youth
=> Attitudes & Skills => Specific Problems
=> Generalization
15
Role of Probation in RehabilitationRole of Probation in Rehabilitation• Assess youth’s strengths and weaknesses
Assign youth to right caseload (low risk caseload)Assign youth to right program(s)Engage and motivate youth and family
– Courts: motivational interview and change cycle – JRA: Functional Family Probation
Support and help youth and family generalize what is learned during research-based program
Deflect enabling undesirable behaviorMonitor changes in risk and protective factors to see if approach is
making a difference and adjust
• Does not make job easier, but might make it more effective
16
Assessment Proliferation
• Must be something to WSJCA, since it is being used in at least a dozen states (including New York, Illinois, Florida, Utah, N. Dakota) and multiple courts – some in California
• Service providers particularly seem to like it
• We encourage continual refinement and improvement of the assessment; capitalizing on what has been done
• It is evolving to be a systematic collection of relevant social file information
17
Third Thing: Quality Assurance Infrastructure
• Steering committee
• Contract with program designers
• Statewide program experts
• Regional consultants
• Initial training
• On-going consultation, feedback, and training
• Assessment of competent program delivery
18
• A “rigorous” test Not enough funding for everyone, so “Waiting List” youth
assigned to a control group by court staff Includes completers and non-completers
• Youth’s eligibility for program is based on risk assessment Level of risk for re-offending Profile of risk factors
• Tests if probation plus a research-based program reduces recidivism more than probation alone
Fourth Thing: Evaluation Design
19
Control Group (N=313) Individual Therapists (N=387) Group Mean (Average)
18-M
ont
h U
nadj
uste
d F
elon
y R
ecid
ivis
m P
erc
ent
age
17
12
55
47
42
14
34
18
23
31 28
14
26
17
33
23
14
33
22
17 17
11
0
C 1 2 3 4 5 6 7 M 8 9 10 11 12 13 M 14 15 16 17 18 M 19 20 21 22 23 24 25 M
Not Competent
Borderline
Competent
Highly Competent
8
18 20
43
47
63
26
FFT ResultsFFT Results
20
6%
25%
49%
27%
7%
50%
7%
19%
45%
Misdemeanor and Felony Recidivism
Felony Recidivism Violent Felony Recidivism
18-M
onth
Rec
idiv
ism
Per
cent
age
Control (N=525) Not Competent (N=203) Competent (N=501)
ART ResultsART Results
21
19%
3%
17%
1%
Felony
12-M
onth
Rec
idiv
ism
Per
cent
age
Control (N=171) COS (N=171)
Adjusted 12-Month Recidivism
Misdemeanor and Felony
Coordination of Services Results
22
• FFT delivered competently Reduces 18-month felony recidivism by 38 percent Benefit to cost ratio of $7.69
• ART delivered competently Reduces 18-month felony recidivism by 24 percent Benefit to cost ratio of $12.60
• Coordination of Services Reduces 12-month felony recidivism by 57 percent Benefit to cost ratio of $13.58
• MST No findings because of problems implementing the
Institute’s evaluation design
Findings From Initial Studies
23
A Clear Lesson…
The key to reducing recidivism with research-based programs is
competently delivering the service.
24
Parole
Intensive Parole
Local Parole
Local Commitment
Group Homes
------------------------------------------
Basic Training Camp
Dialectic Behavior Therapy
Mentoring Program
Family Integrated Transitions
Additional JRA Programs We Were Asked to Evaluate
25
-$12,478-$4.95$2,098-$10,379Regular Parole (v. not having parole)
-$5,992$0.00$5,992$0Intensive Parole
-$1,500$0.00$1,500$0Intensive Probation Supervision
$5,073$1.78$6,471$11,544Mentoring
$5,128$13.58$408$5,536Interagency Coordination
$8,805$12.60$759$9,564Aggression Replacement Training
$14,315$7.69$2,140$16,455Functional Family Therapy
$19,247$3.15$8,968$28,215Family Integrated Transitions
$22,364n/a-$7,586$14,778Washington Basic Training Camp
$31,243$38.05$843$32,087Dialectical Behavior Therapy
Benefits Minus Costs
Benefits per
Dollar of Cost
CostsBenefits
Measured Benefits and Costs Per YouthCost Benefit Estimates for Washington State Programs as of September 17, 2004
26
Benefits and Costs of Prevention and Early Intervention Programs for Youth
September 2004
Steve Aos et al. Washington State Institute for Public Policy
www.wsipp.wa.gov
2003 Legislature Starts to Repeat Process for Other Intervention and
Prevention Programs
27
• Question: Are there Question: Are there “research-based”“research-based” programs programs or policies with a or policies with a “real world”“real world” ability to: ability to:
• Reduce crime,• Lower substance abuse,• Improve educational outcomes,• Decrease teen pregnancy,
• Reduce teen suicides?• Lower child abuse or neglect, or
Research Questions & MethodsResearch Questions & Methods
• The six outcomes• The quality of a study’s research design• Whether the program is “real world”
• We screened evaluations written in English for:We screened evaluations written in English for:
• We computed effects (meta-analytically)We computed effects (meta-analytically)
• We then calculated We then calculated monetarymonetary benefits and costs benefits and costs
28
Summary of Benefits and Costs (2003 Dollars)Summary of Benefits and Costs (2003 Dollars)Dollars Per Youth (PV lifecycle) BenefitsBenefits CostsCosts B B -- C C
Early Childhood Education $17,202 $7,301 $9,901
Nurse Family Partnership $26,298 $9,118 $17,180
Functional Family Therapy $16,455 $2,140 $14,315
Life Skills Training $746 $29 $717
Seattle Soc. Dev. Project $14,246 $4,590 $9,837
Guiding Good Choices $7,605 $687 $6,918
Multi-D Treat. Foster Care $26,748 $2,459 $24,290
Intensive Juv. Supervision $0 $1,482 -$1,482
Big Brothers/Sisters (all costs) $4,058 $4,010 $48 (taxpayer costs only) $4,058 $1,283 $2,775
Selected FindingsSelected Findings
Multi-Systemic Therapy $14,996 $5,681 $9,316
29
Managing Competent Program Delivery
Invest in increasing skills, not accountability
• Pick good people
• Provide required training
• Enhance skills
• Retain only competent providers
30
• Written knowledge test and feedback after initial training
• Follow-up skill training and consultations
• Certification at end of probationary period
Initial Provider Training
31
• Form regional support teams
• On-site direct observation by an expert, including audio- or video-taping to improve competence
• Case-by-case consultations
• Give feedback
Continually Enhance Provider Skills
32
Please place a check in the box that best describes what happened in this session. YES▼
NO ▼
1. Was a positive climate established through welcoming students?
2. Were group norms reviewed, emphasizing positive participation?
3. Were all ACT concepts covered to this point reviewed?
4. Were issues from the last anger control training session reviewed?
5. Did all youth complete the hassle log(s)?
Skill Feedback
33
• Establish process for corrective action
• Corrective action taken when needed
• Building morale; taking pride that we are good!
Ensure Competent Providers Are Retained
34
• Youth eligible, assigned, and completing programs (75%)
• Changes in targeted risk and protective factors
• Program provider competency ratings
• Meeting expected recidivism rates
Annual Legislative Accountability
35
“Let’s get going on one of these programs.”
Some of You May Be Thinking…
-or-
“We already do a great job, so why would we be interested in this?”
36
• Understand what is currently being done
• Understand what is being done in research-based programs
• Examine the differences Are you already doing itDoes it seem like an improvementWhat needs to be tested
• Implement quality assurance for current practice
• Have competent researcher do valid outcome evaluations
If You Are Not Comfortable With the Research Approach, a Possibility…
37
Essential Ingredients
• Infrastructure for program fidelity• Funding and de-funding contingent
on continued outcomes
• Leadership• Legislation• Funding pilot research-based programs • De-funding ineffective programs
• Juvenile court involvement (administrators, supervisors, and staff)
• Local provider involvement• Skilled independent
researchers
• Assessment • Research-based programs
• Valid outcome evaluations
• Cost-benefit analysis