predictive modeling: a key tool for decision-making in care … · 2019. 5. 22. · financial 30.2%...
TRANSCRIPT
![Page 1: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/1.jpg)
Predictive Modeling: a Key Tool for Decision-Making in Care
Management at Johns Hopkins HealthCare
Linda Dunbar, Vice President, Care ManagementMartha Sylvia, Health Data Analyst, Care Management
![Page 2: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/2.jpg)
JOHNS HOPKINS H E A L T H C A R E
23,000
122,000
45,000
![Page 3: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/3.jpg)
JHHC Care Management
HIV/AIDS
Cardiovascular/DiabetesTeleWatch
Guided Care
Children with Special Needs
Partners with Mom Rehabilitation
End of Life Care
End Stage Renal DiseaseBehavioral Health
Substance Use
Complex Medical Needs
![Page 4: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/4.jpg)
Historical Population Case Finding
• Target diagnoses with financial prioritization– Total dollars– Medical Loss Ratio
• Referrals– Utilization management– Providers– Health department– Outreach department
• Focused committees
![Page 5: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/5.jpg)
Motivation for Change
• Increasing Evidence that threshold-based models are inadequate for case finding and can lead to misallocation of resources, inefficiencies, and missed opportunities (Cousins MS, Shickle LM, Bander JA (2002), An introduction to predictive modeling for disease management risk stratification, Disease Management, 5(3), 157-67.)
• Evidence that Adjusted Clinical Groups Predictive Model (ACG-PM) and similar predictive models perform better than threshold-based models (ACG Virtual Library: Version 5.0 (December 2001) ACG Software Documentation/Users Manual. www.acg.jhsph.edu.
![Page 6: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/6.jpg)
Stakeholders in the Decision to Implement ACG-PM
• Care Management Administration• JHHC Administration• Medical Directors• Disease and Case Management Staff• Finance • Information Systems• Decision Support
![Page 7: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/7.jpg)
Assessing Our Medicaid Population
44.3%42.9%ACG-PM Method
24.5%30.2%Financial Method
Positive Predictive Value or Detection Rate
Sensitivity or Capture Rate
Analysis was performed using ACG beta-testing version 6. CY2001 predicted enrollees were compared to CY2002 actual high cost enrollees.
![Page 8: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/8.jpg)
Financial Selection Method 2001
Top 2% acgPM in 2001 Predicted for 2002
Top 2% Actual Costs CY 2002 N=1450
N=4318
N=1027
N=534
N=810
N=1195
N=1344
N=871
N=5850 N=3032
N=3131
![Page 9: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/9.jpg)
ACG-PM to Select Medicaid Enrollees with Substance Use Problems(SUP)
• All Claims for the determined time period inputted into the ACG grouper software
• ACG grouper software assigned a probability score to every enrollee that represented their probability of being in the top 5% of high utilizers in the next year(s)
• Algorithm using diagnoses and ACG-PM score used to select enrollees for intervention
![Page 10: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/10.jpg)
Selection AlgorithmMedicaid currently enrolled,
Age >= 21, geographic criterian=14,624
Positive for substance use using ICD-9 and CPT criteria, exclusions removed
n=3123
Ranking on ACG-PM ScoreTop 400 chosen for intervention
ACG-PM range = 0.39 to 1.00
![Page 11: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/11.jpg)
Characteristics of the SUP/High ACG-PM Enrollees
• Compared to low-risk SUP enrollees, high-risk SUP enrollees identified by ACG-PM– Had higher prevalence of 52 chronic medical conditions– Had a higher average number of medical conditions– More hospital admissions– More hospital days– More ER visits– Higher pharmacy costs– Higher total Costs
![Page 12: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/12.jpg)
High-risk SUP Enrollees had More Admissions and ED Visits than Low-risk SUP Enrollees
Admissions and Emergency Department Visits per Thousand (annualized)p<0.0011,2
307
1314
4714
1772
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
Admits per Thousand (annualized) ED Visits per Thousand (annualized)
Visi
ts p
er T
hous
and
Low-Risk n=1985
High-Risk n=400
1 p calculated using Mann-Whitney-U (non-parametric). Did not meet assumption of equal variances necessary for t-test.2 n=1985 represents number of members claims data available for out of the total n=2085.
![Page 13: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/13.jpg)
Total Costs PMPMp<0.0011,2
$ 5 8 7
$ 2 , 0 8 4
$0$250
$500$750
$1,000$1,250
$1,500$1,750$2,000
$2,250$2,500
Low -Risk n=1985 High-Risk n=400
1 p calculated using M ann-Whitney-U (non-parametric). Did not meet assumpt ion of equal variances necessary for t-test.2 n=1985 represents number of members claims data available for out of the total n=2085.
High-risk SUP Enrollees had Higher Total Costs than Low-risk SUP Enrollees
![Page 14: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/14.jpg)
Mean Number of Medical and Psychosocial Conditionsp<0.0011,2,3
2.18
5.29
1.52
6.81
1.16 1.01
0
1
2
3
4
5
6
7
8
9
10
Number of Medical Conditions Number of PsychosocialConditions
Number of Medical andPsychosocial Conditions
Mea
n nu
mbe
r of C
ondi
tions
Low-Risk n=1919
High-Risk n =400
1 52 chronic medical conditions and 7 psychological conditions w ere analyzed using Expanded Diagnostic Clusters (EDCs, w hich are part of the ACG toolkit). These conditions w ere chosen because of their high cost and amenability to intensive clinical intervention2p calculated using Mann-Whitney-U (non-parametric). Did not meet assumption of equal variances necessary for t-test.3 n=1919 represents number of members disease categories w ere assigned for out of the total n=2085.
High-risk SUP Enrollees had More Medical and Psychosocial Conditions than Low-risk SUP
Enrollees
![Page 15: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/15.jpg)
High-risk SUP Enrollees had a Higher Prevalence of Selected High Cost Medical Conditions than Low-
risk SUP EnrolleesDisease Prevalence of Selected Medical Conditions
p < 0.0011,2,3
3% 2%
9%
2% 2%
16%
5%2%
6%
12%
5%
27%
20%
28%
20%
11%
43%
19%15% 15%
30% 29%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Ische
mic
Heart
Dise
ase
Conge
stive
Hea
rt Fa
ilure
Lipid
Diso
rder
s
Diabete
s w C
ompli
catio
ns
Chron
ic Liv
er D
iseas
eLo
w Bac
k Pain
Perip
hera
l Neu
ropa
thy
Cereb
rova
scul
ar D
iseas
e
Obesit
y
Asthm
a
COPD
Low -Riskn=1919
High-Riskn=400
152 chronic medical conditions and 7 psychological conditions using Expanded Diagnostic Clusters (EDCs, w hich are part of the ACG toolkit). These conditions w ere chosen because of their high cost and amenability to intensive clinical intervention2 p calculated using chi-square3 n=1919 represents number of members disease categories w ere assigned for out of the total n=2085.
![Page 16: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/16.jpg)
Integrating ACG-PM into Daily Operations
• Intensive staff education• Disseminating information • Disease management database
enhancements• Clinical screener role• Clinical screener toolkit
![Page 17: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/17.jpg)
Database Enhancements
![Page 18: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/18.jpg)
Database Enhancements
![Page 19: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/19.jpg)
Clinical Screener: A New Nursing Role
• Registered Nurse• Managed care and clinical experience• Proactive screening of enrollees identified
by ACG-PM as potential high utilizers • Continued referral screening• Assessment and program referral
![Page 20: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/20.jpg)
Clinical Screener Toolkit
• ACG-PM: Predictive Modeling• Diagnoses• Utilization• Clinical indicators: lab and radiology results• Clinical assessment: telephone contact• Disease/case management amenability
assessment
![Page 21: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/21.jpg)
ACG Lookup Database
![Page 22: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/22.jpg)
ACG Lookup Database
![Page 23: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/23.jpg)
ACG Lookup Database
![Page 24: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/24.jpg)
Referral from Screener to Disease/Case Manager
![Page 25: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/25.jpg)
Referral from Screener to Disease/Case Manager
![Page 26: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/26.jpg)
Referral from Screener to Disease/Case Manager
![Page 27: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/27.jpg)
In Summary…
• Predictive Modeling– An important part of an overall strategy
• One of many screening tools
– Improves CM and DM targeting• Our research shows that predictive model does identify high-
risk cases amenable to Care Management interventions
– Must be employed with appropriately educated staff• Clinical Screener Role• Disease and Case Managers must understand output of
predictive models
![Page 28: Predictive Modeling: a Key Tool for Decision-Making in Care … · 2019. 5. 22. · Financial 30.2% 24.5% Method Positive Predictive Value or Detection Rate Sensitivity or Capture](https://reader036.vdocument.in/reader036/viewer/2022071104/5fde7824e1fc7333c3264dc3/html5/thumbnails/28.jpg)
Future Directions….
• Children with Special Health Care Needs– Performance of model with children– Part of a screening strategy that would include
instruments that are sensitive to children and families
• HIV/AIDS population– Performance of model– Part of a screening strategy