using rwd in the form of ehr data to predict adverse ...• 1.04% of study population had a bleed...
TRANSCRIPT
Using RWD in the Form of EHR Data to Predict Adverse Events and OutcomesPHUSE US Connect 2020Monday March 9th, 2020
Adverse Drug Events: Huge costs to Patients and Providers Alike
1 in 6senior hospital admissions is due to an ADE
Annually,
1 million patient admissions nationally with an estimated cost of more than
$14 billion
More than 90% of 30-day readmissions due to ADEs are preventable.1
2
Research Need
1. Dalleur, O., et al (2017). 30-Day Potentially Avoidable Readmissions Due to Adverse Drug Events. Journal of Patient Safety, [Online, published ahead of print]. Available from: https://journals.lww.com/journalpatientsafety/
• Patient-level risk scoring can improve patient outcomes and prevent costs to the health system
• Limited scale of prior predictive modeling of ADE risk for anticoagulant users
• Edj’s machine learning modeling excels at decision support and resource allocation, making it a natural fit for this problem
Early Risk Detection Can HelpResearch Impact
3
Objective
Dataset
Adverse Event PredictionObjectives and Data
4
• Increase patient safety by predicting serious bleeding event risk and risk factors for individual patients undergoing anticoagulant therapy
• Evaluate risk on a 90-day horizon • Drive reduction in readmission rate with risk factors identified by machine
learning
• 39,058 patients on anticoagulants from a health system touching about two million lives
– Time span: 2017-2018• 1.04% of study population had a bleed event in 3-month window• Received first-party EHR data and integrated proprietary SDoH factors
Results
Key Findings
Adverse Event PredictionResults and Key Findings Snapshot
5
• Edj queried and compiled diagnoses for all encounters (inpatient, ER, outpatient, ambulatory) for serious bleeding events.
• Edj built a highly predictive model well-suited for resource allocation– Model’s highest ADE risk group has an event 10x more often than the total
population
• Edj’s model produces risk and anti-risk factors at both a population level and an individual level
• Edj generated significant predictive lift (8%) by combining proprietary SDoH data with health system EHR data.
• Describes real events in a health system
• First-hand data from service providers
• Modeling fit to the local population
• No third-party data costs
Why Model with EHR Data?Benefits of EHR Data as RWD
6
• Event-level data source with clear time component
– Can be updated near real-time
• Patient-level data allows for predictions by individual
• Easier to integrate back into the EHR for provider decision support
In-House Data Granularity Integration
To model with EHR data, we:• Perform extensive ETL/Data Processing
– Hundreds of person-hours of data exploration and cleaning to prepare for modeling stage
• Creatively engineer predictive features– Created more than 1000 features for ADE model– Example: Calculating change in a patient’s vitals
over various time periods• Added a temporal component to static model types
Preparing EHR Data for InsightsData Cleaning and Feature Engineering
7
Modeling ResultsUsing RWD in the Form of EHR Data to Predict Adverse Events and Outcomes
Instant Risk Detection and ContextModeling Results
9
Highly Accurate and InterpretableRisk Scores for Each Patient
• Predicts risk of a bleed event within a three-month window based on EHR and non-clinical factors
• Promotes optimizing care continuum resource allocation by risk score
Population-level and Patient-level Risk and Anti-Risk Factors
• Details “the why” for a risk score, making it truly a decision support tool
• Individual-level risk factors identified, driving personalized risk-prevention plans
Features
10
• CPT and ICD codes– Indicator vectors for high occurring individual codes– Rolled up versions of individual codes– Bag of words representations of code descriptions
• Medications– Indicator vectors of generic name and pharmacy class
• Labs and vitals– Summary stats of labs and vitals over past two years, as well as rate of change over time.
• Demographics– Indicator vectors for age, sex, smoking status, etc.
• Various encounter information– Frequency of visits, service locations, discharge disposition, insurance type
Modeling Results
• AUC (Area Under the Curve) is a score for how accurate a model’s predictions are in aggregate.
- It ranges from 0 to 1- Also referred to as a C-
Statistic• The random forest model, using
EHR data coupled with SDoH data, provides strong predictive accuracy within the three-month prediction window.
- Area under curve metric (AUC = .8) indicates that the model distinguishes effectively between patients with bleed event and no bleed event.
11
Evaluating Predictive PowerModeling Results
• There is a low target event rate with an overall bleed rate of .0104.
• As your risk score increases, the actual bleed rate for patients scored in that strata is increasing.
• The scores from the model output are conducive to allocating resources to higher risk patients.
Model PerformanceModeling Results
12
• At a high level, feature importance tells how much of a given prediction is coming from that feature.
– A feature being important doesn’t imply a positive or negative linear relationship.
– These features are important in a series of rules rather than being important in isolation.
Key Features of ImportanceModeling Results
13
• Key features of importance in determining an individual’s risk score include:– Procedure descriptions, diagnosis descriptions– Medications– Vitals and general health– Engagement with the health system
Sample Patient-Level Risk FactorsModeling Results
14
Why Do Social Determinants Matter?
• A 2016 study determined that only 16% of a health outcome could be explained by clinical care.1
• The remainder of the health outcome could be explained by social determinants of health (SDoH)
– Physical environment – Healthy behaviors – Socioeconomic conditions
• To complete the picture of the patient, Edj integrates proprietary SDoH data into clinical modeling
Modeling Results
151. Hood, C. M., Gennuso, K. P., Swain, G. R., & Catlin, B. B. (2016). County Health Rankings:
Relationships Between Determinant Factors and Health Outcomes. American Journal of Preventive Medicine, 50(2), 129–135. doi: https://doi.org/10.1016/j.amepre.2015.08.024
Key SDoH factors for this model included:• Percent of never delinquent transactions (i.e. bills paid)
– Possibly an indicator for drug adherence• Life insurance score
– Credit-based measure of mortality risk• Credit score
SDoH Features Drive Better PredictionsModeling Results
16
8 percent increase in predictive power
Q&AUsing RWD in the Form of EHR Data to Predict Adverse Events and Outcomes
