predictive analytic approach to achieving youth stability...

Achieving Better Outcomes

for Transitioning Youth

Predictive Analytic Approach

to Achieving Youth Stability

(PAAYS)

Elizabeth Wynter

Leslie Leip

Ira Schwartz

September 10, 2015

Florida Department of Children & Families Summit

Achieving Better Outcomes

for Transitioning Youth

How are transitioning youth faring in your

community?

How do you know how they are faring?

Do transitioning youth have the services

they need?

How do you know what services are most

important for good outcomes?

Will Extended Foster Care impact these

outcomes?

TIL History- Broward County In 2002, Broward County established a TIL Task Force

made up of community leaders, funders, and providers.

In 2003, United Way & Community Foundation hired a

consultant to develop an action plan.

In 2006, United Way & Community Foundation

combined funds to hire a TIL Coordinator to work with

the community to build an integrated system of care.

In 2007, CSC in partnership with the Jim Moran

Foundation expanded their Future Prep to provide life

coaches to youth ages 15 to 21.

In 2009, the Junior League’s legacy project, a youth

resource center (FLITE Center), opened its doors to

provide youth a single hub for services and supports

In 2013, the FLITE Center, recognized as a national

model, was established as a nonprofit agency.

TIL History- Broward County

Integrated system of care built by

leveraging and shifting resources and

developing partnerships with housing

developers, workforce alliance,

The system of care includes 188 housing

units, life coaches, mentors, employment

services, educational assistance, and a

youth resource center.

System-wide outcomes are being tracked by

the FLITE Center

TIL Outcomes- Broward County

Outcomes 2011

(n=757)

2012

(n=749)

2013

(n=757)

2014

(n=767)

Obtained high

school diploma or

GED

35% 38% 39% 41%

Parenting 19% 20% 20% 17%

On probation 8% 9% 9% 9%

Incarcerated 5% 3% 3% 2%

Experienced

homelessness in

the last 3 months

3% 2% 4% 3%

Predictive Analytic Approach

to Achieving Youth Stability

In 2013, ChildNet was 1 of 18 agencies awarded a

federal grant to find answers these questions.

The research project entitled, Predictive

Analytic Approach to Achieving Youth Stability

(PAAYS), worked to identify youth most at-risk of

homelessness and to design an approach to

reduce homelessness among youth/young adults

The objective of the project was together, mine,

and analyze data on youth to better understand

their protective and risk factors.

Administration of Children & Families

(ACF)

The purpose of the ACF grant was to build on

a framework for intervening with youth who

are in foster care or have experienced some

time in care, including youth age 14 and

older, and are most likely to have a

challenging transition to adulthood, including

homelessness and unstable housing

experiences.

The framework for considering how to

structure interventions for youth/young adults

who have or have had involvement with the

foster care system and are facing

homelessness was developed by the U.S.

Interagency Council on Homelessness (USICH).

USICH Framework

Protective Factors

• Family Cohesion & Support

•School Engagement

•Employment

•Survival Skills

•Positive Connections

•Positive Future Expectations

•Good Decision-making Skills

•Self-esteem

•Self-efficacy

•Good Health

•Stable placement

Risk Factors

• Trauma

• PTSD

• Other Behaviors

Resulting from

Trauma

• Sexual Risk

Behaviors

• Family Problems

• Criminal/Delinquent

Behavior

• Substance Abuse

• Running Away

Phase I

Overview of PAAYS – Phase I

As the Community-Based Care lead agency responsible for overseeing the child welfare system in Broward and Palm Beach Counties, ChildNet clearly recognizes the need for planning and refining the system of care for youth and young adults to prevent homelessness.

In order to achieve better outcomes for youth and young adults at risk of homelessness, PAAYS focuses on:

Anticipating the challenges that children will bring with them when they enter the child welfare system;

Rethinking the structure of services delivered throughout the system; and

De-scaling practices that are not achieving results while concurrently scaling up evidence-based interventions.

Predictive Analytic Approach to

Achieving Youth Stability (PAAYS)

PAAYS PHASE I

GOAL: Transform the system of care through

data driven strategies in order to reduce the

incidence of homelessness among youth with

foster care history

OBJECTIVES: Gather, mine, and analyze data on

youth to better understand them

Entering care between the ages of 14 to 17

Exiting care at age 18, and

Former foster care young adults, ages 18 to

21, who have experienced homelessness.

PAAYS PHASE I

A three pronged approach was used to

gather and assess data on these 3 cohorts of

youth/young adults.

Risk and protective factors were

collected using validated tools.

Multisystem data was mined and

analyzed to identify predictive factors

for successful adulthood.

Existing service data was amassed to

pinpoint gaps in the system of care.

YOUTH RISK & PROTECTIVE

FACTORS Starting in January 2014 through March 2015,

Assessment Clinicians interviewed 243 youth in

the 3 cohorts.

Youth entering care between the ages of 14

to 17 were assessed using the validated Child

& Adolescent Needs & Strengths (CANS) tool

that was aligned with the USICH risk and

protective factors

Young adults ages 18 to 21 who were

transitioning out of care or who had

experienced homelessness were assessed

using the Adult Needs & Strengths

Assessment (ANSA) tool that was aligned with

the USICH risk and protective factors


FACTORS

The CANS survey as of March 15, 2015, was

completed with 129 youth ages 14 to 17.

Protective factors- Most youth possessed

strengths in language, physical health, self-

efficacy, and self-esteem.

Risk factors- Majority of youth had

experienced disruptions in

caregiving/attachment losses, neglect, and

adjustment to trauma.

-10%

-5%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

55%

60%

65%

70%

75%

80%

85%

90%

95%

100%

PAAYS Cases

Distribution of PAAYS Youth Age 14-17, N=129CANS-Lite assessment tool

Percent Protective Factors - Percent Risk Factors = Risk Typology

35% to 75% = At-risk

76% or more=Low Risk

Less than 34%=Risky

Mean Score =55%

Standard Deviation = 21%

This group is reassessed

every 3 months.This group is reassessed

every 6 months.


every 12 months.


FACTORS FINDINGS- CANS

Key Findings:

CANS assessments identified

disruptions in caregiving/attachment

losses as the main risk factor for the

youth age 14-17

Employment and school achievement

were identified as the main

protective factors that need to be

improved.

-10%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%Distribution of PAAYS Youth Age 18-21, N=114

ANSA-Lite assessment tool Percent Protective Factors - Percent Risk Factors = Risk Typology

41% to 76% = At-risk

77% or more = Low Risk

41% or less = High Risk


every 6 months.This group is reassessed

every 12 months.


every 3 months.

PAAYS Cases

Mean Score =59%

Standard Deviation = 18%


FACTORS FINDINGS-ANSA

Key Finding:

ANSA assessments identified

disruptions in caregiving/attachment

losses as the main risk factor for the

youth age 18-21, and service

permanence and relationships were

identified as the main protective

factors that need to be improved.

Summary of

Findings for

the 3

Cohorts

DATA for PREDICTIVE ANALYTICS

OVERVIEW

The objective of mining historical data from a

multiple systems was to identify factors that

hinder successful adulthood among youth that

have exited the system and to identify pathways

to success.

In October of 2014, data were pulled for youth

and young adults, ages 14 to 23, from FSFN, DJJ,

School Board, FLITE Center, and the CANS and

ANSA surveys.

In total there were 2,554 unique cases with more

than 14,000 data elements.

735 cases with complete data sets were used for

the analytics.

DATA for PREDICTIVE ANALYTICS

DATA PROCESSING

Risk factors for young adults to be having a criminal

history, poor educational performance, lack of housing

stability, lack of employment experience, and risky

behaviors.

Youth were considered high risk if they had 3 to 5 of

these indicators and low risk if they had 0 to 2 of the

indicators.

Of the cases analyzed, 70% of the youth showed low risk

and 30% were considered high risk.

The descriptive data elements included citizenship,

employment, work/life skills, education status,

placements, removals, most recent service type,

correctional services history, and adoption services

history.

PREDICTIVE ANALYTICS

ANALYTIC PROCESS

In total, elements from 735 cases of youth

ages 18 to 23 were analyzed

Cases of youth ages 14 to 17 will be analyzed

next.

While the conclusions on services and

interventions that work are limited due to the

small sample size, the predictive models were

found to have 72% to 88% accuracy ratings.

Pathways to successful adulthood included

placement stability, remaining in care, and

employment experience.

Number of Placements

>1

Risk=

Number of Placements

<=1

Successful Adulthood

92% not at risk

Employed

Citizenship

Overall Model

Accuracy =80%

Overal l Pathways to


Unemployed 56% at risk

NoCitizenship

63% not at risk

74% at risk

At risk = of unsuccessful adulthoodNot at risk = of unsuccessful adulthood

31

Not Employed

Employed


66% not at risk

Work/Life Skills

Strength

Overall Model

Accuracy =73%

Pathways to Successful Adulthood for

Cit izens

65% not at risk

Work/Life Skills Need

63% not at risk

100% at risk

100% at risk

Unknown Employment

Status

Work/Life Skills

Unknown


32

Aging Out

Not Aging Out


88% not at risk

Unemployed

Correctional Services

Overall Model

Accuracy = 76%


Cit izens

Employed 61% not at risk

NOCorrectional

Services

76% at risk

Withdrawn from

Education

NotWithdrawn

from Education

56% at risk

54% not at risk


33

More than One

Placement

No More than One

Placement


91% not at risk

Overall Model

Accuracy = 88%


Non-Cit izens

84% at risk


34

PREDICTIVE ANALYTIC FINDINGS

YOUTH SERVICES SURVEY

The online survey collected service

information and opinions of its usefulness

from 178 youth who had participated in

the CANS or ANSA surveys.

Questions included their living

arrangement, religious/spiritual

activities and use of therapy, life coach,

employment, and educational services.

YOUTH SERVICES SURVEY Key Findings

28% were connected to a life coach; 74%

of the youth working with a life coach

found it useful.

Only 8% received education services.

No youth reported being referred to or

receiving employment services.

YOUTH ENGAGMENT Using a peer advocate approach, youth were kept

engaged through social media and monthly phone

calls or text messages.

An initial focus group with the youth unearthed their

desire to create a YOU MATTER community to talk

about every day issues.

They created a flag and a creed. Some months more

than 50 youth came to the monthly meeting and kept

the conversation going in the parking lot after the

session ended.

YOU MATTER EVENTS

Feedback Question Some Responses

I learned something from this event. "That I am cared for so much."

"Yes, I learned that I actually do Matter & I have people

that really care about me."

YOU MATTER events make me feel a "They always do, nothing new."

part the of the group. "I love You Matter."

"Yes, it helps me to meet new people. Overall I had a

great time. And I would love to come to all the events."

"We have group discussions and everyone gets to be

heard."

Would you attend another event "Its' a learning experience. I always get information and

organized by the YOU MATTER team? learn something new/or get refreshed."

"We have fun every time."

Based on the findings from

Phase I, we know:

Youth need a permanent connection

Youth need education services

Youth need employment services

Youth want to be part of the process,

intervention, evaluation

Phase II intervention should focus on

strengthening protective factors – employment,

education, and emotional well-being – while

maximizing stability, permanency, and service

permanence.

PHASE II RECOMMENDATIONS

Based on the key findings, the following elements

were included in the Phase II proposal:

1. Maintain youth engagement strategies to stay

connected youth/young adults.

2. Scale up the use of evidence-based programs to

support transitioning youth.

3. Implement an evidence based education

intervention.

4. Implement an evidence based employment

intervention.

5. Implement an evidence-based intervention

focused on emotional well-being.

6. Support permanency and sustain placements.

THANK YOU

predictive analytic approach to achieving youth stability...

Documents