mark a. greenwald laura moneyham director of...

8/21/2015 1

Mark A. GreenwaldDirector of Research and

Data Integrity

Laura MoneyhamAssistant Secretary for Residential Services

2

Discussion Topics

• Risk Assessment

• Importance of Service Matching

• The Disposition Matrix

• Continuum of Services Mapping

• Standardized Program Evaluation Protocol (SPEP)

Risk Assessment – Then and Now

8/21/2015 3

Intuitive

1st Generation

Actuarial & Static

2nd Generation

3rd Generation

4th Generation

Dynamic

3rd Generation

4th Generation

Case Management

4th Generation

Protective & Responsivity

4th Generation

5th Generation:In Development

Using Predictive Analytics to Improve Risk Assessment

8/21/2015 5

Modeling Metrics to Outcomes

NEW RECIDIVISM VARIABLE_ Total Convictions_Adj_New Defmition

Node 0 Catego!Y % n

• NO 77.679 47440 • YES 22 .3 21 13632

Total 1 00 .000 61 072

I L:: JJIS Total number of adjudicated charges pnor to program

00: = 2 .000 )> 2 .000

I I Node 2 Node 1

% catego!:Y % n 83.1 84 36344 16.816 7 3 47

Total 71 .540 43691

• NO • YES

Total

63.840 11 096 36.160 6285 28 .460 17381

I L:: JJIS Total number of charge s pnor to program

< = 13.000 )> 13.000

I I Node 3 Node 4

Catego!Y % Category %

• NO 70 .308 6178 • NO 57.226 4918 • YES 29.692 2609 • YES 42.774 3676

Total 14.388 8787 Total 14.072 8594

I '-= JJIS Number of Other Felony Prior Charges

Node 5 Node 121 6 Catego!:Y % Catego!Y %

• NO 57 .330 4908 • NO 30.303 • YES 42_670 3653 • YES 69_697

Total 14 .018 8561 Total 0 .054

I L= Age at Intake

<= 16 638 )> 16 638

I I Node 6 Node 525

Catego!:Y % Catego!:Y % n

• NO 50.639 1 782 • NO 61.999 3126 • YES 49 .361 1737 • YES 38.001 1916

Tota l 5 .762 3519 Total 8 .256 5042 "1± I L=

10 23

33

JJIS Total number of charges pnor to program

<= 26 000 )> 26 000

I I Node 526 Node 946

Catego!:Y % n Catego!:Y %

• NO 64_942 2297 • NO 55_083 • YES 35.058 1240 • YES 44 .917

Total 5.792 3537 Total 2_464

L±

Risk Assessment at the Florida Department of Juvenile Justice

Prevention Assessment Tool (PAT)

Positive Achievement Change Tool (PACT)

Residential Positive Achievement Change Tool (R-PACT)

8/21/2015 6

Sample PACT Overview Report

8/21/2015 7

PACT - Overview Report

Name: Chopper Test DOB: 5/1/1990 DJJID: 804266

Created By: FL State Acct Administrator

Created Date: Aug 9 2006 1:23PM

Last Modified By: FL State Acct Administrator

Last Modified Date: Aug 10 2006 1:49PM

Risk Factors

25% 50%

Overall Level of Risk toRe-Offend: Low

Record of Referrals Risk Score: 3

Social History Risk Score: 4

Static and Dynamic Combined

Protective Factors

75% 100% Domain 0% 25% 50% 75%

GA : History of Relationships

SA: Employment History

7A: Family History

10: Attitudes/Behaviors

3A: School History

7B: Current Uving Arrangements

SA: Alcohol and Drug History

1: Record of Referrals

3B: Current School Status

4A: Historic Use of Free Time

48: Current Use of Free Time

SB: Current Employment

GB: Current Relationships

SB: Current Alcohol and Drugs

9A : Mental Health History

9B: Current Mental Health

PACT Validation StudiesChris Baird, Theresa Healy, Kristen Johnson, Andrea Bogie, Erin Wicke Dankert, and Chris Scharenbroch (2010). “A Comparison of Risk Assessment Instruments in Juvenile Justice”. U.S. Department of Justice Report # 244477.

Michael T. Baglivio (2009). The Assessment of Risk to Recidivate Among a Juvenile Offending Population. Journal of Criminal Justice, 37; 596-607.

Michael T. Baglivio, and Katherine Jackowski (2013). Examining the Validity of a Juvenile Offending Risk Assessment Instrument Across Gender and Race/Ethnicity. Youth Violence and Juvenile Justice, 11:1; 26-43

Kristin Parson Early, Gregory A. Hand, and Julia L. Blankenship (2012). “Validity and Reliability of the Florida PACT RISK and Needs Assessment Instrument; A Three-Phase Evaluation”. Justice Research Center, Inc. Contract P2085.

Ira M. Schwartz, Peter York, Mark A. Greenwald, and Ana Ramos-Hernandez (2015). The Application of Predictive Analytics and Machine Learning to Risk Assessment in Juvenile Justice: The Florida Experience. (Forthcoming)

8

Matching Services…

10

A Graduated Sanctions Model

Diversion

Teen Court

Probation

Intensive PS

Day Treatment

Residential Placement

C/R Day Treatment

Probation

Intensive PS

Redirection

Redirection

26

EBP Mantra

The right service

For the right kid

At the right time

In the correct dosage

12

JJSIP Components

Comprehensive Strategy

Structured Decision Making

Evaluation Standardized Program Evaluation Protocol (SPEP)

13

Tiers of Evidence

The lowest form is anecdotal evidence; stories, opinions, testimonials, case studies, etc.

The highest form is empirical evidence –research, data, results from controlled studies, etc.

We do not want to norm an entire system on anecdotal outliers…

14

5 Principles of Effective Intervention

Principle InterventionRisk: Target high-risk offenders.Need: Treat risk factors associated with

offending behavior.Treatment: Employ evidence-based and

research-proven treatment approaches and interventions.

Responsivity: Tailor treatments to meet special needs.

Fidelity: Monitor implementation quality and treatment fidelity.

15

Targeting High-Risk Offenders

Risk Level and Treatment Recidivism Outcomes

Level of TreatmentStudy Risk Level Minimal Intensive

O’Donnell et al. (1971) LowHigh

.16

.78.22.56

Baird et al. (1979) LowHigh

.03

.37.10.18

Andrews & Kiessling (1980)

LowHigh

.12

.58.17.31

Bonta et al. (2000) LowHigh

.15

.51.32.32

16Source: D.A. Andrews & James Bonta (2003). The Psychology of Criminal Conduct. 3rd Edition. Cincinnati, Ohio, Anderson Publishing Company.

17Source: Michael T. Baglivio (2013). The Risk Principle. Florida Department of Juvenile Justice, Bureau of Research and Planning

Q) ...... ro

0::::

E V')

> ""0 u Q)

0::::

Recidivism Rate for all Low Risk to Re-offend Youth by Placement Type

30

7S

.... ..... ~ 20

15 ~ ....

~ 10

5

0

~ .....

/

_...... ~

~All Low Risk Youth

Note: Data from 2012 Comprehensive Accountability Report (CAR) final files Recidivism rate for I DDS significantly lower than all other placement types for the low risk sample.

Diversion and I DDS significantly lower than Probation Supervision. Probation Supervision, CBIS, Probation Enhancement rates statistically equivalent. Probation, CBIS, and Probation Enhancement rates significant ly lower than Day Treatment, Redirection, Residential, and PCP. Day T reatment, Redirection, Residential, and PCP recidivism rates are statistically equivalent.

18Source: Michael T. Baglivio (2013). The Risk Principle. Florida Department of Juvenile Justice, Bureau of Research and Planning

Recidivism Rate for Low Risk Youth by "Needs" Level by Placement Type

Note: Data from 2012 Comprehensive Accountability Report (CAR) final files

~Low Risk "Not Identified as High Needs"

- Low Risk "Higlh Needs"

"High Needs" defined as youth greater than 1 standard deviation above the mean on t lhe Social History Score subcomponent of the PACT. Statistically significant differences found iin the recidivism rates for low risk "high needs" youth versus youth

not identified as such for the following Placement Types: Diversion, I DDS, Probatiion Supervision, with llow risk "high needs" youth having significantly higher recidivism rates .. Differences in recidivism rates for Probata ion Enhancement, Day Treatment, Redirection, Residential, and Post Commitment Probation were not significant.

Matched Low-Risk YouthResidential Probation Difference

Recidivism Pre-matching 26.7% 16.1% 10.6%*

Post-matching 26.7% 16.9% 9.8%*

19

• 28,681 Probation youth (low-risk)• 1,726 Residential youth (low-risk)

• Matched on:• Age at 1st arrest• Current drug/alcohol use• Expulsion/drop out• Violent felony• Felony• Antisocial peers/gang association• County• Race/ethnicity• Gender

Source: Analysis conducted by the Florida Department of Juvenile Justice, Office of Research and Data Integrity

Lipsey’s 2009 Meta-analysis “Interventions applied to high-risk

delinquents…produced larger recidivism reductions than when those interventions were applied to low-risk delinquents” (p.23)

“There was no indication that there were juveniles whose risk level was so high that they did not respond to effective interventions” (p.23)

20Source: Mark W. Lipsey (2009). The Primary Factors that Characterize Effective Interventions with Juvenile Offenders: A Meta‐Analytic Overview. Victims & Offenders, 4:2; 124‐147.

Common Risk Factors Predict Delinquency (The Big Eight)

1. Antisocial Attitudes 2. Antisocial Peers3. Antisocial Personality Patterns (impulsivity, low

self-control, risk taking)4. History of Antisocial Behavior5. Problems at School/Work6. Problematic Family Circumstances7. Problematic Leisure Activities/use of free time8. Substance Abuse

21

Need Principle: Why Dynamic Priority Domains?

Research shows a 38% reduction in recidivism when case plans contained interventions matched to assessed criminogenic needs for high risk youth. (Luong, D., & Wormith, J.S. (2011).

The absence of interventions to address a domain that was ranked medium risk or higher was associated with an 82% increase in likelihood of recidivism. (Luong, D., & Wormith, J.S. (2011).

22

91

50

0

10

20

30

40

50

60

70

80

90

100

No InterventionFor High RankedNeedBaseline

31

50

0

10

20

30

40

50

60

InterventionsMatched toNeedsBaseline

Disposition Recommendation Matrix Is a structured decision making tool that assists

with matching youth to the appropriate level of service/supervision

Is based on a matrix of risk to reoffend (PACT) and the presenting offense

Consists of graduated sanctions – The intensity of services increases as the risk level and offense severity increases

24

Key Points of the Disposition Matrix Low-risk offenders remain in the community

with minimal supervision

Moderate-risk offenders typically placed in more structured community programs, with intensive probation supervision for higher risk youth

Residential placement reserved for the highest risk offenders after community-based alternatives have been exhausted

25

26

Florida Department of Juvenile Justice Disposition Recommendation Matrix (Staff should begin with the least restrictive setting within a particular disposition category. See Structured Decision-Making guidelines.)

Most Serious Presenting Offense

Civil Citation Eligible1

Low Risk to Reoffend

Level 1

Level2 or 3a

Level2 or 3a

Level 2 or 3a-b

PACT Risk Level to Reoffend Moderate Risk to Reoffend

Level 1

Level2 or 3a

Level 2 or 3a-b

Level 2, 3a-c, or 4

Moderate-High Risk to Reoffend

Level 2 or 3a-c

Level 3a-c or 4

Level 3a-c, 4, or 5

Level 3a-c or 4

Level 3a-c or 4

Level 3a-c, 4, or 5

- Eligibility for civil citation is outlined in F_S 985_12_ Youth deemed ineligible for civi l citation (based on community standards) should be reviewed under the "Minor" offense category based on the PACT risk level to reoffend_ 2- All misdemeanor offenses_ 3 - Felony offenses that do not include violence_ 4- Violent felony offenses (do not include misdemeanor assault and battery which are captured under "Minor")_

l evel1 -Alternatives to Arrest l evel 2 - Diversion & Non-DJJ Probation l evel 3 - Community Supervision l evel 4 - Non-Secure Residential Commitment

(3a)- Probation Supervision level 5- Secure Residential Commitment (High & Maximum Risk Programs) (3b)- Probation Enhancement Services (ART, LifeSkills, etc_) (3c) - Day Treatment, MST, FFT, Minimum Risk Commitment

Disposition Matrix Validation

38,117 youth released in FY10-11.

Below (n=691) Optimum (n=27,916) Appropriate (n=7,322)Above (n=2,188)

Holds true for males, females, across race/ethnicity, and for all risk levels of youth.

27Source: Michael T. Baglivio, Mark A. Greenwald, and Mark Russell. (2014). Assessing the implications of a structured decision‐making tool for recidivism in a statewide analysis: A disposition matrix for court recommendations made by juvenile probation officers. Criminology and Public Policy, 14:1, 5‐49.

Continuum Mapping

Identify the available services within each county

Map the identified available services according to service category within each county

Identify the target population for each categorized service according to levels of the Disposition Recommendation Matrix

29

Data Collection

30

County Service Mapping Report

31

http://www.djj.state.fl.us/research/delinquency-data/services-continuum-report/

The Second JJSIP Component

33

Meta-Analysis: Dosage

Group 5 Service (Score=30) Cognitive-Behavioral Therapy

○ Target Weeks=15; Target Hours=45 ○ Qualifying Supplemental Services: None (automatic 5 points added to

score)

Group 4 Service (Score=25) Group Counseling


score) Mentoring

○ Target Weeks=26; Target Hours=78 ○ Qualifying Supplemental Services: Behavioral Contracting/Management

Behavioral Contracting; Contingency Management ○ Target Weeks=24; Target Hours=72 ○ Qualifying Supplemental Services: Mentoring, Mixed Counseling

(individual, group, family, and/or vocational), Remedial Academic Program

34

Dosage (cont. 2) Group 2 Service (Score=10)

Restitution; Community Service ○ Target Weeks=12; Target Hours=60 ○ Qualifying Supplemental Services: None (automatic 5 points added to

score) Remedial Academic Program

○ Target Weeks=26; Target Hours=100 ○ Qualifying Supplemental Services: Job -Related Services (work

experience, job preparation, and/or job training)

Group 1 Service (Score=5) Individual Counseling


score) Job-Related Training

○ Vocational Counseling Target Weeks=20; Target Hours=40 Qualifying Supplemental Services: Remedial Academic Services

○ Job Training Target Weeks= 25; Target Hours=400 Qualifying Supplemental Services: Remedial Academic Services

○ Work Experience Target Weeks=26; Target Hours=520 Qualifying Supplemental Services: Remedial Academic Services

35

37

Delinquency Arrests in Florida1

1 Source: Florida Department of Juvenile Justice, Office of Research and Data Integrity; FY 2014-15 data is preliminary. Official delinquency arrest figures will be released in the 2015 Delinquency Profile Report (October, 2015).

110,493

97,144

85,44878,345

74,871

0

20,000

40,000

60,000

80,000

100,000

120,000

FY 2010-11 FY 2011-12 FY 2012-13 FY 2013-14 FY 2014-15

38

Changes in Delinquency Arrests by Offense Seriousness, FY 2013-14 and FY 2014-151

Offense Category

Sum of Offenses FY

2013-14

Sum of Offenses FY

2014-15Difference % Change

Felony 25,775 25,542 -233 -1%

Misdemeanor 34,786 31,809 -2,977 -9%

"Other" 17,784 17,520 -264 -1%

Sum of Total 78,345 74,871 -3,474 -4%

1 Source: Florida Department of Juvenile Justice, Office of Research and Data Integrity; FY 2014-15 data is preliminary. Official delinquency arrest figures will be released in the 2015 Delinquency Profile Report (October, 2015).

391 Source: Florida Department of Juvenile Justice, Office of Research and Data Integrity.

0%

10%

20%

30%

40%

50%

60%

FY 2010-11 FY 2011-12 FY 2012-13 FY 2013-14 FY 2014-15

PACT Risk to Reoffend for Youth Disposed to Commitment by Percentage of Youth Committed

(FY 2010-11 through 2014-15)1

Low Moderate Mod-High High

84% of current commitments involve moderate-high or high risk youth

72%

40

Operational Capacity for Residential Services at Onset of Fiscal Year

6,258

7,018

7,140

7,065

6,572

6,118 6,010

4,8524,488

4,136

3,6063,177

2,5142,131 2,154

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Res

iden

tial C

omm

itmen

t Bed

s

Fiscal Year

Source: Florida Department of Juvenile Justice, Residential Services

ReferencesDonald A. Andrews and James Bonta (2003). The Psychology of Criminal Conduct. 3rd Edition. Cincinnati, Ohio, Anderson Publishing Company.

Michael T. Baglivio, Mark A. Greenwald, and Mark Russell. (2014). Assessing the implications of a structured decision-making tool for recidivism in a statewide analysis: A disposition matrix for court recommendations made by juvenile probation officers. Criminology and Public Policy, 14:1, 5-49.

Michael T. Baglivio, Katherine P. Jackowski, Mark A. Greenwald, and James C. Howell. (2014). Serious, Violent, and Chronic Juvenile Offenders: A Statewide Analysis of Prevalence and Prediction of Subsequent Recidivism Using Risk and Protective Factors. Criminology and Public Policy, 13:1, 83-116.

Mark A. Greenwald, and Michael T. Baglivio (2015). Analysis of Serious, Violent and Chronic Delinquency in Florida. Florida Department of Juvenile Justice, Office of Research and Data Integrity.

Mark W. Lipsey (2009). The Primary Factors that Characterize Effective Interventions with Juvenile Offenders: A Meta-Analytic Overview. Victims & Offenders, 4:2; 124-147.

41

Other Department Resources

The Office of Research and Data Integrity:http://www.djj.state.fl.us/research

The Juvenile Justice System Improvement Project (JJSIP) in Florida:http://www.djj.state.fl.us/research/latest-initiatives/juvenile-justice-system-improvement-project-(jjsip)

Delinquency Profile Report:http://www.djj.state.fl.us/research/delinquency-data/delinquency-profile

Delinquency Briefings / Special Topics Research:http://www.djj.state.fl.us/research/fast-facts/delinquency-briefings

Delinquency in Florida’s Schools Research:http://www.djj.state.fl.us/research/reports/research-reports/delinquency-in-schools

42

mark a. greenwald laura moneyham director of...

Documents