1 on track advanced topics getting the most out of your outcomes data eric hamilton, m.s. vice...

1

On Track Advanced Topics

Getting the Most Out of Your Outcomes Data

Eric Hamilton, M.S.Vice President of Clinical Informatics, ValueOptions

Jeb Brown, Ph.D.Director, Center for Clinical Informatics

2

Overview

1. What can outcomes data add to my practice?

2. A closer look at the Client Feedback Form

3. What is MyToolkit and what do all those numbers mean?

4. Putting data into practice

5. Discussion

3

What can outcomes data add to my practice? • Do you always know which patients

struggling in therapy?• Do some of your clients stop therapy before

you can establish a good relationship?• Can you show clients evidence of their

progress compared to benchmarks?• Can you show prospective clients,

colleagues, or managed care companies evidence of your clinical outcomes?

4

How good are clinicians at identifying clients at risk for treatment failure?

• Lambert and colleagues compared clinician predictions to predictive algorithms using early client self-assessment data

• Only 3 out of 550 cases predicted to have poor outcome• Of the 40 that had a poor outcome, only 1 had been predicted

by a clinician• Algorithms based on early assessments identified 85% of

poor outcome cases – but also identified some that did well

0

100

200

300

400

500

600

Positive No Change Deteriorated

TherapistPredictedOutcome

ActualtreatmentOutcome

Source: Hannan, et al (2005), JCLP

Number of cases

5

What do the predictive algorithms mean?

• Statistical model that compares early client progress to a “typical profile,” adjusting for case characteristics such as intake severity

• Poor early progress does not “predict” a likelihood of failure – rather it is an indicator of heightened drop-out risk

• “Off-track” clients who remain engaged generally get good outcomes – it just might take them longer

0

5

10

15

20

25

30

35

40

45

Intake 3 6 9 12 15 18+

Weeks

WA

-Adu

lt S

core

Actual score

85% percentile

Predicted Score

15th percentile

Clinical cuttoff

Global DistressScore

6

Client Feedback Form (CFF) - Adult

Item Groups • Global Distress: 1-10

– Core scale– Sensitive to change over time– Depression, anxiety, social functioning

• Risk of self-harm: 5– Risk indicator

• Substance use: 11-13– Risk indicator

• Work productivity: 14-15 – Indicator of functioning

• Therapeutic alliance: 16-18– Support tool for therapist

• Background items: 19-20– For case-mix analysis and identification of co-

morbidities

7

Tips for Using the CFF

• Print multiple copies before you need them• Have available for the member to complete

before the session begins• Show you value the data – when giving the

CFF and when it is returned• Review scored results online before the

next session • Measure early and measure often!

8

CFF Results: What do all those numbers mean?

Access On Track results

9

Outcomes based on the most recent CFF

High scores in red

10

You decide which variables

to display

11

Results: Key things to remember• “Effect Size” is a way of looking at the combined

results of cases with 2+ data points, compared to normative data

• Links at the top of each column take you to more information

• All scores are presented as means– Range is from zero to four

• Higher severity results appear in Red• GDS = Global distress score (Questions 1-10)• Change Score = Difference in the mean GDS score

from first assessment to the most recent• Benchmark Score = How much better or worse the

change score is compared to norms

12

Interpreting Adult Global Distress Scores• Scores of 0 to 1.5

– Typical of community (non-treatment) samples

• Scores of 1.6 to 2.5– Moderate distress– Only 20% of community samples would be expected to score

in this range; about half of individuals seeking treatment score in this range

• Scores of 2.6 to 4.0– Severe distress– 25% of individuals seeking MH services score in this range

13

The Trajectory of Change Graph

Case is “off-track” compared to the benchmark projection

14

Putting Data Into Practice

• Most cases don’t require a lot of review of the data, look for the high risk results

• Talk with your clients about their results, especially when they share something new or surprising

• An “off-track” cases does not mean a big change is needed – some encouragement to stay engaged may be all that is needed

• Be sensitive to small changes in the alliance questions

• If your effect size is low at first, keep measuring and reviewing the data

• Alliance items are a great place to look for clues for improving effectiveness

15

Questions and Discussion Resources for Questions

– Frequently Asked Questions

On the web site, near bottom of the ValueOptions page– Technical/Data/Web:

Email to datacenter@clinical-informatics.com

– General comments or questions: Email to OnTrackOutcomes@valueoptions.com or Call On Track Customer Service 866-476-9796