decision support algorithms with communimetrics

Decision Support Algorithms with Communimetrics

JOHN S. LYONS, PHD and APRIL D. FERNANDO, PHD

Center for Innovation in Population Health, University of Kentucky

TCOMCONVERSATIONS.ORG | @PRAEDFOUNDATION

CommunimetricsBackground and Philosophy


The Story of HelpingThings happen in people’s live and sometimes these events lead someone to believe that receiving care from a professional might be helpful.

Adults recognize that care might help. Individuals with limited means must rely on public systems for support.

This is where the story in the system begins …

Possible referral sources for adults:• Internal recognition of value• Family and Friends• Other Medical Professional

Possible referral sources for children:• Parents/Guardian• School staff or Teacher• Other Medical Professional

Understanding the Business of Helping: The Hierarchy of Offerings

Gilmore & Pine, 1997

Products

.

Produced for a retail market

Sushi

Transformations

Helping people change in some notable way.

Teaching people how to fish

Experiences

Purchasing a memory.

Sport FishingCommodities

Raw Materials

Fish

Services

Having someone apply a product for you.

Seafood Restaurant

Communimetrics: Understanding a Person’s Story

The individual shares their story.

The care provider listens to their

story.

The other story tellers share their

perspectives.

The stories are combined and a

single narrative is agreed upon.

01

02

03

04

Person-Centered Care Requires New Metrics

Clinimetric CommunimetricClassicalTest

ItemResponse

1 2 3 4 5 6 7 8

\

History of Measurement Theories

Communimetrics

§ Native Naturalism (Reality Theory) rather than British Empiricism

§ Non-arbitrary—every number has immediate meaning

§ Culturally and developmentally informed—the measure of a story

§ Based on qualitative approaches to synthesizing complex phenomenon—modified grounded theory

§ Post triangulation rather than pre-triangulation measurement

Scenario 1: Youth is distressed and the parent is minimizing the situation. With treatment the youth feels better and the parents come to realize the youth’s mental heath needs

0123456789

10

Catastrophizing Youth Minimizing Parent

AdmitTransition

Scenario 2. Parent is catastrophizing and youth is minimizing. With treatment the youth understand his her mental health needs better and the parent sees progress

0123456789

10

Minimizing Youth Catastrophizing parent

AdmitTransition

The problem with means of single perspectives—the average of two clinically successful treatment episodes equates to no effect

0

1

2

3

4

5

6

Youth Perspective Parent Perspective

AdmitTransition

01 Items are selected because they are relevant to service/case planning.

02 Each item uses a 4-item rating scale that translates into action.

03Rating should describe child/youth, not the child/youth in services.

6 Key Principles

04Consider culture and development before determining ratings.

05The ratings are agnostic as to etiology; it’s about the What, not the Why.

06Use a 30-day window in considering what is relevant to children, youth and their families.

RELEVANCE

ACTION LEVELS

CLIENT FOCUS

CULTURE & DEVELOPMENTTHE “WHAT”

30-DAY WINDOW

Action LevelsS t a t u s a n d I m p a c t

3

2

1

0No evidence of need. No need for action.

Significant history of need, or possible need that is not interfering with functioning.

Watchful waiting, additional assessment.

Need interferes with functioning. Action/intervention required.

Need is dangerous or disabling.Immediate/intensive action required.

Well developed centerpiece strength. Easily accessible by individual; essential for planning.

Useful strength. Evident and can be accessed by individual; useful for planning.

Strength identified. Requires building in order to be useful for individual or planning.

No strength identified. Considerable effort or building to create and develop strength.

Individuals & Family Program System

Decision SupportCare Planning

Effective PracticesEBPs

Outcomes Management

Quality Improvement

ROW 6

TCOM Grid of Tactics

EligibilityStep-Down

Resource ManagementRight-Sizing

Provider Transitions & Celebrations

ProgramEvaluation

Provider Profiles Performance/

Contracting

Case Management Integrated Care

Supervision

CQI/QAAccreditation

Program Redesign

TransformationBusiness Model

Design

Decision Support and CommunimetricsA Person-Centered Approach


Decision Making and Decision Support

Decision Support Uses of CommunimetricMeasuresCommunimetrics is actively used in decision support within child welfare settings in the United States for each of the following key decisions. Many of these applications have been operations for years; some for more than a decade.

o Placement type and intensity

o Level of care

o Case management intensity

o Case rates

o Service Packages

o Evidence-based Treatments

o Safety and Risk


TCOMCONVERSATIONS.ORG | @PRAEDFOUNDATIONTo date, not all child serving systems using the CANS have implemented decision support algorithms.

Person-Centered Decision Support• Some approaches to level of care use service receipt or history to inform

level of care.• These approaches make the assumption that all such decisions are

perfectly indicated clinically.

• We know that, in reality, all sorts of factors actually influence service receipt in reality including availability, racial and cultural factors, etc.

• Thus using service receipt or history in decision support simply institutionalizes these biases into ongoing decisions.

• Compliance is simply an indirect indicator of prior service receipt.


https://zoom.us/rec/play/-qGYol1YAYAawVU_ANCXsOFsOZjZfz2MtsvezTeKZLCl4w4FTH4m7-ogh2vJdIy7KYWM6IUEjNUo_jfE.--7iDI7T-OiNFlpi?continueMode=true&_x_zm_rtaid=G5g83OCeQ_SRinr0uFCL3g.1599071050241.5b24fc6ff52d727a8fda6a6e8b674c5e&_x_zm_rhtaid=757

https://zoom.us/rec/play/-qGYol1YAYAawVU_ANCXsOFsOZjZfz2MtsvezTeKZLCl4w4FTH4m7-ogh2vJdIy7KYWM6IUEjNUo_jfE.--7iDI7T-OiNFlpi?continueMode=true&_x_zm_rtaid=G5g83OCeQ_SRinr0uFCL3g.1599071050241.5b24fc6ff52d727a8fda6a6e8b674c5e&_x_zm_rhtaid=757

Percent of hospital admissions that were low risk by racial group Adapted from Rawal, et al, 2003

0%5%

10%15%20%25%30%35%40%45%50%

1998 1999 2000 2001 2002

% of

Low

Risk

Adm

ission

s White

AfricanAmerican

Hispanic

Most existing measures use total scores with cut-offs for decision support applications. These cut-off approaches can be problematic at the margins because very small differences in a total score can lead to very different decisions.

In Larry P v. Riles (1984), for example, norm-based decision support using IQ in schools was deemed discriminatory in CA.


Communimetric algorithms and other measures’ approaches to decision support

Because of the item level reliability and action level format, patterns of actionable needs are used to generate algorithms using boolean (branching logic) models.

These models are far more intuitive clinically and therefore more defensible as they are easy to describe and the differences between individuals at different levels are always meaningful.


Communimetric algorithms and other measures’ approaches to decision support

Decision Making as a Consensus Process

Customization of Algorithms• Unlike most existing measures, the algorithms use a flexible TCOM framework to think

about decision support rather than an ‘off-the-shelf’ logic that applies the same standards everywhere.

• In reality different systems are different and to generate meaningful change you have to start where the system is currently functioning.

• For example, the original algorithm for residential treatment placement in Illinois was VERY low. This still resulted in a 30% decline in residential placements. Once the system was effectively evolved, then the algorithm could be adjusted to reflect the evolving system’s performance.

• As such, the approach allows for the customization of decision models by jurisdiction (county) to reflect different cultural contexts.


The Utility of Communimetric Decision Support AlgorithmsEach of the following States have successfully used communimetricdecision support algorithms for LOC decisions in Child Welfare for more than a decade. There have been no problems and notable system improvement:

◦ Indiana

◦ New Jersey

◦ New York

◦ Wisconsin

◦ Tennessee

A number of other states and counties have used and continue to use communimetric decision support algorithms for shorter periods of time.


Statewide CANS Algorithm ExamplesNew Jersey, I l l inois and Tennessee


New Jersey’s System of CareThe f irst statewide implementation of a cross-systems

(comprehensive) version of the CANS


In New Jersey• Despite the number of children and youth served in their system of

care tripling over the past decade [Figure A]

• The actual number of youth placed in residential treatment has been reduced by over one third during the same period [Figure B]

• CANS algorithms were used to support this process and research using the CANS informed the design of the step down process

• At the same time nearly 1/3 of detention centers have been closed along with all state hospitals for children and youth.



Figure A. New Jersey’s Children’s System of Care Expansion: 2008-2016


Figure B. New Jersey Children’s System of Care:

Number of Youth in Residential Care (2010-2016)

Illinois Department of Children and Family ServicesThe state with the f irst use of a CANS Algorithm applied within a

Team Decision Making process


In a series of published studies, Brian Chor and his colleagues studied the Illinois CANS decision support algorithm as used in their team decision making process. They found…

• Following the CANS level of care recommendation predicted greater clinical improvement as compared to not adhering to the recommendation. Chor, et al (2012) Children and Youth Services Review

• Serving children below the CANS recommended level of care resulted in less improvement in functioning, and serving them at a higher than recommended level of care resulted in reduced rates of improvement as compared to adhering to the CANS recommendation. Chor, et al (2014) Administration and Policy in Mental Health

• The CANS algorithms out-performed the placement decisions made by the child-family team Chor, et al (2013) Child Abuse and Neglect


Communimetrics (Lyons, 2009)• Following the placement recommendations of the CANS decision

support algorithm within a Team Decision Making process was associated with more stable placements over time. [Figure C]

• Placing children below the CANS recommended level of care placement results in the second most stable placement history.

• Placing children in a higher than recommended level of care was associated with dramatically less stable placements over time.



Figure C. Survival analysis of time to placement disruption for children/youth whose placement matches CANS recommendations (Match= 0, green), those whose placed is at a lower intensity than recommended (match= -1, blue) and those whose placement is more intensive than recommended (match= 1, brown).

Tennessee Department of Children’s ServicesThe f irst state in which case workers completed the CANS

themselves



Figure D. In an independent replication, Epstein, et al also found that following a CANS algorithm was associated with more stable placements than not following that algorithm on the initial placement in child welfare. Epstein, et al (2015) Residential Treatment for Children and Youth

0.00

0.25

0.50

0.75

1.00

0 100

200

300

400

500

600

700

800

900

1000

1100

1200

1300

1400

1500

1600

1700

1800

1900

Time (Days)

Prop

ortio

n of

Und

isrup

ted

Reco

rds

Strata A

Strata B

4336 1848 944 580 381 253 176 124 81 57 40 22 13 6 2 1 1 0 0 0

9604 5132 2628 1683 1111 759 537 379 264 176 119 86 56 41 27 20 16 8 5 1Strata B

Strata A

Numbers at risk

Tennessee implemented the CANS as a result of a law suit

• The TN DCS implementation was the first to train case workers to complete the CANS. Training and support was provided to case workers to help them build the necessary skill set through Vanderbilt University’s Center for Excellence.

• TN State leadership credits the use of the CANS as part of the reason that they are now out from under this lawsuit as they went from being ranked as one of the least effective systems to one of the more effective systems during the decade after the CANS was fully implemented.


In Summary• The CANS is widely used in decision support applications as described

above including Placement and Level of Care (and case rates/service packages).

• The CANS has been successfully used by a number of jurisdictions for more than a decade without incident and with sustained successful system change.

• Although many have been tried, no other cross-sector decision support approach has experienced this level of sustained success in child welfare and behavioral health.


Decision Support AlgorithmDevelopment Process


Decision Support Algorithm Development• A description of the jurisdiction’s Level of Care structure and criteria is provided to

Praed.

• Identify youth in the current jurisdiction’s Level of Care structure and review their current CANS ratings (if available).

• Jurisdiction establishes the decision options – define each level and provide any child specific information about each.

• Items from CANS mapped to the jurisdiction’s defined levels and new algorithm is developed.

• Jurisdiction identifies a Panel of Experts to review the new algorithm recommendations. The panel of experts, along with Praed convene to discuss the initial algorithm recommendations.


Algorithm Variables Non-Algorithm Variables

Acuity # of Assessments

Low 7890

Med 2438

High 5093

Plots for Low, Med, High Acuity by Variable Type

decision support algorithms with communimetrics

Documents