ami treatment cost & predictive mortality analysis · ami (acute myocardial infarction –...

AMI TREATMENT COST & PREDICTIVE MORTALITY ANALYSIS

Steve SteklenskiAdvisory Consultant Dell EMC [email protected]

Shiliang WangPrincipal Data ScientistHCL America [email protected]

2016 EMC Proven Professional Knowledge Sharing 2

Table of Contents Introduction .................................................................................................................................................. 4

AMI (Acute Myocardial Infarction) Heart Attack ...................................................................................... 4

Objectives of This Study ................................................................................................................................ 5

Figure 1: AMI-related Clinical Diagnoses .............................................................................................. 5

Figure 2: AMI Patient Count by ICD9 and Financing Source (Payer Plan) ............................................. 6

Figure 3: Average Cost of AMI Treatment by ICD9 Diagnosis and Number of Patients ....................... 6

Figure 4: Variation of AMI Treatment Cost (Logarithmic) by ICD-9 Diagnostic Code ........................... 7

Figure 5: AMI Treatments by Ethnic Group ........................................................................................... 7

Current and Related Guidelines and Research ......................................................................................... 8

Figure 6: AMI Treatment Guidelines (Source: Ryan, et al, Management of Acute Myocardial

Infarction, Journal of American College of Cardiology, November 1996) ............................................. 8

Data & Methodology .................................................................................................................................... 9

Data Used in This Study ............................................................................................................................ 9

Figure 7: Variables Analyzed (Most Relevant Highlighted) ................................................................... 9

Methodologies ........................................................................................................................................ 10

Using Random Forest decision trees to predict in-hospital mortality ................................................ 10

Figure 8: Hospital Survival and Death rates ........................................................................................ 10

HLMM (Hierarchical Linear Mixed Model) .............................................................................................. 10

Figure 9: Logarithmic Transformation Applied to AMI Treatment Cost ............................................. 11

Figure 10: Density plots for AMI treatment cost and logarithm transformed AMI cost ..................... 11

Results ......................................................................................................................................................... 12

Comparison of AMI treatment cost after standardization ..................................................................... 12

Figure 11: AMI Treatment Cost Averages for Each State, Before Standardization ............................. 13

Figure 12: Baseline Average of AMI Treatment Costs for Hospitals, After Standardization ............... 13

Figure 13: Baseline Average of AMI Treatment Cost for States, After Standardization ..................... 14

Baseline average of AMI treatment cost for states after standardization ............................................. 14

Developing a model to predict in-hospital mortality .......................................................................... 14

Figure 14: Risk factors for AMI in-hospital mortality – Procedure 1 delay in days, financing source,

and number of chronic diseases .......................................................................................................... 15

Figure 15: Random forest model predicts AMI in-hospital mortality with ~90% accuracy ................. 15


Discussion of Results / Conclusions ............................................................................................................ 16

Summary ................................................................................................................................................. 16

Future Directions .................................................................................................................................... 16

References .................................................................................................................................................. 17

Disclaimer: The views, processes or methodologies published in this article are those of the authors.

They do not necessarily reflect Dell EMC Corporation’s views, processes or methodologies.


Introduction

AMI (Acute Myocardial Infarction) Heart Attack

AMI (Acute Myocardial Infarction – commonly known as Heart Attack) is a leading acute disease in the

US in terms of both treatment cost and mortality. It consists of 20 separate medical diagnoses (Figure 1).

It represents an estimated 2.5 million inpatient treatment visits annually, with an aggregate treatment

cost of $175 billion, representing an average cost of $69k per visit. AMI is also responsible for one in

four deaths annually (134,000), with an inpatient mortality rate of 5.27%.

While the etiology of AMI largely remains elusive, the diagnostic causes are diverse (Figure 1). The AMI

treatment cost varies drastically by different diagnostic codes. For example, the cost for treating ICD-9M

code 410.10 is 3.5 times of that for 410.20 on average (Figure 2). Fortunately, our data showed that the

diagnostic code – 410.71 is the most common AMI for in-patient treatment. At the same time, AMI

treatment cost varies greatly by lots of other factors such as disease severity, complications, chronic

diseases, procedures, treatment options, labor, and material cost in different regions.

A prominent example of cost variation, for example, is the 5000 times difference of treatment cost

observed from patients with the same primary diagnostic code (Figure 3). In addition to those variation

causes, the significant cost difference was observed to be affected by differences in payer plans and

demographic factors (Figures 2 and 5, based on analysis of 2011 national sampled data as described

below). Because of the broad array of factors causing treatment cost variation, it is a big challenge for

insurance companies and healthcare providers to compare the cost and benchmark their performance

directly. Yet, it is of great interest for these stakeholders to have this information so that they can

choose different services and improve their performance.

Not only does AMI present significant costs for insurance companies and government plans, it also

contributes a big percentage of death tolls in the US. Hospital resources, performance, disease severity,

treatment options, and time all affect AMI mortality rate. Given the significance of AMI, the Centers for

Medicare & Medicaid Services (CMS) have built a model to standardize mortality for each hospital and

predict mortality for each individual case3. However, no existing model has been established to

accurately predict in-patient mortality and identify significant factors contributing to in-patient mortality

with different insurance plans and a wide range of chronic conditions that patients may have.


Objectives of This Study The objectives of this study were to identify key predictors of mortality and develop a risk- and quality-

adjusted ranking of hospital costs associated with AMI inpatient treatment. These objectives address

key challenges associated with AMI, which include significant variation in treatment costs and outcomes

associated with inpatient episodes of care. Specifically, this study targeted the following objectives:

1. Identify the key drivers of variation in treatment cost and survival rates for AMI. 2. Develop a standardized method to compare AMI treatment cost and generate a national

hospital ranking list for the treatment cost on the basis of risk-adjustment. 3. Develop a predictive model for inpatient short term survival of AMI.

Figure 1: AMI-related Clinical Diagnoses

410.00 AMI ANTEROLATERAL,UNSPEC

410.01 AMI ANTEROLATERAL, INIT

410.10 AMI ANTERIOR WALL,UNSPEC

410.11 AMI ANTERIOR WALL, INIT

410.20 AMI INFEROLATERAL,UNSPEC

410.21 AMI INFEROLATERAL, INIT

410.30 AMI INFEROPOST, UNSPEC

410.31 AMI INFEROPOST, INITIAL

410.40 AMI INFERIOR WALL,UNSPEC

410.41 AMI INFERIOR WALL, INIT

410.50 AMI LATERAL NEC, UNSPEC

410.51 AMI LATERAL NEC, INITIAL

410.60 TRUE POST INFARCT,UNSPEC

410.61 TRUE POST INFARCT, INIT

410.70 SUBENDO INFARCT, UNSPEC

410.71 SUBENDO INFARCT, INITIAL

410.80 AMI NEC, UNSPECIFIED

410.81 AMI NEC, INITIAL

410.90 AMI NOS, UNSPECIFIED

410.91 AMI NOS, INITIAL


Figure 2: AMI Patient Count by ICD9 and Financing Source (Payer Plan)

Figure 3: Average Cost of AMI Treatment by ICD9 Diagnosis and Number of Patients


Figure 4: Variation of AMI Treatment Cost (Logarithmic) by ICD-9 Diagnostic Code

Figure 5: AMI Treatments by Ethnic Group

Unknown

White

Black

Hispanic

Asia or Pacific

Islander Native American

Other


Current and Related Guidelines and Research

As mentioned above, AMI consists of 20 diagnostic categories, for which various treatment protocol

guidelines are proscribed. As a result this study anticipated findings in variations of care, based on

variation in diagnosis and treatment regimen. Figure 6 summarizes current AMI Treatment Guidelines.

Figure 6: AMI Treatment Guidelines (Source: Ryan, et al, Management of Acute Myocardial Infarction, Journal of American

College of Cardiology, November 1996)

The previously cited Yale study prepared for CMS, in its methodology report3, highlights significant

findings associated with the evaluation of AMI treatment and outcomes. The Yale study included

evaluation of AMI treatment and 30-day outcomes, including post-discharge care and readmissions, as

well as adjusted cost.

However, while the Yale study attempted to adjust cost on the basis of excluding non-AMI related

expenses, it did not provide comparative risk adjustment for random effects, something our study has

included at both the regional and institutional level, resulting in a more robust risk adjustment

methodology as a basis for outcome costs.


Data & Methodology

Data Used in This Study

This paper presents findings based on the analysis of data made available by the Healthcare Cost and

Utilization Project (HCUP), sponsored by the Agency for Healthcare Research and Quality (AHRQ), a

department of the United States Health & Human Services (HHS). The data set used is the National

Inpatient Sample (NIS), which is collected annually on more than 7 million hospital stays, incorporating

data from state, hospital, private data organizations, and the US federal government. The NIS represents

a 5% sample of available data.

Available NIS data used in this analysis, reflective of data typically provided in a hospital discharge

abstract, include:

Primary and secondary diagnoses and procedures

Anonymized patient-level data including demographics

Payment sources and associated charges

Severity and co-morbidity measures

Listed below are the data variables considered in this analysis. Highlighted are those variables identified

as most significant based on the application of the random forest and hierarchical linear mixed model

statistical approaches. Details of the approaches are outlined in the following section.

Figure 7: Variables Analyzed (Most Relevant Highlighted)


Methodologies

Using Random Forest decision trees to predict in-hospital mortality

Random Forest techniques (R package - RandomForest) were used to predict AMI survival (binary

outcomes modeling) and identify the key risk factors associated with AMI, such as diagnosis code and

procedure delays. In summary, a sample of AMI records was randomly drawn from all 127,016 records

to develop a decision tree with around 2/3 of the sample. Five predictive variables were randomly

chosen to start building a tree. Then the best variable which minimized genie index was picked to split

the node for each sample. The tree was then tested by the remaining 1/3 of the sample and the mean

error was monitored and recorded. This random sampling process was repeated 1000 times and a tree

built for each random sample. The final prediction was made by the aggregation of all 1000 trees with

the majority of votes.

Bootstrap sampling was used to correct the imbalance data set where the mortality only is about 5.27%

of the sample (Figure 8). The significant mortality predictors were identified by the mean prediction

error through permutation and plotted by variable importance plot in RandomForest package. The top

30 significant mortality risk factors were evaluated by their medical meaning. The accuracy of prediction

was accessed by the test samples recorded.

Figure 8: Hospital Survival and Death rates

*Survival rate displayed in green, death rate displayed in red

HLMM (Hierarchical Linear Mixed Model)

The Hierarchical Linear Mixed Model (HLMM) algorithm was used to standardize healthcare treatment

comparisons and estimate the baseline of AMI cost at the hospital and state level. The untransformed

AMI treatment cost was assessed by the density plot and is highly right skewed. The logarithm

transformation (Figure 9) was applied to AMI treatment cost, and the normality distribution was

achieved after the logarithm transformation (Figure 10). Random Forest techniques were again

leveraged to select significant predictors to model treatment cost. Disease severity-related specific cost

was modeled as fixed effects and both hospital and regional costs were modeled as nested random

effects.


Figure 9: Logarithmic Transformation Applied to AMI Treatment Cost

Figure 10: Density plots for AMI treatment cost and logarithm transformed AMI cost

HLMM model selection was implemented by approximated Z-testing, and nested F-testing. A risk-

adjusted cost ranking of hospitals was calculated as the sum of overall mean, random effect of region

(state), and random effect of each hospital. All data was saved in SQL Server and the access was

achieved by R package ODBC.

𝒍𝒐𝒈(𝑪𝒐𝒔𝒕) = 𝟏 + ∑ 𝒙𝒊

𝒑

𝒊=𝟏

+ 𝟏|𝒔𝒕𝒂𝒕𝒆/𝒉𝒐𝒔𝒑𝒊𝒕𝒂𝒍

𝑯𝒐𝒔𝒑𝒊𝒕𝒂𝒍̂ = 𝒐𝒗𝒆𝒓𝒂𝒍𝒍̂ + 𝑹𝑬𝒐𝒇𝑺𝒕𝒂𝒕𝒆̂ + 𝑹𝑬𝒐𝒇𝑯𝒐𝒔𝒑𝒊𝒕𝒂𝒍̂

𝒙𝒊, risk-factors, procedures, insurance, comorbidity

AMI Treat Cost

Log10(AMI Treat Cost)

Density

Density


Results

Comparison of AMI treatment cost after standardization

A huge variation of AMI treatment cost has been reported in the news. To compare the cost in a

standardized way, a hierarchical linear mixed model has been applied in this study. CMS has modeled

hospitals as a random effect to account for variation among different hospitals. Contrasting with the

mixed model used by CMS, our model treats both state and hospitals as hierarchical nested random

effects, enabling identification of variation among both regions and hospitals.

The original 67 cost predictive variables were further selected by random forest algorithm. The 25 cost

predictive variables were finally selected for hierarchical linear mixed model based on variable

importance score of permutation. The model selection was performed using approximate Z tests and

nested F test. Based on p values from Z test and F test, PL_NCHS2006 and ZIPINC_QRTL were removed

from the fixed effect list. The random effects were extracted for both states and hospitals.

The baseline averages of AMI treatment cost for each state and hospital were calculated by the formula

described in the methodology section. In summary, those baseline averages are standardized AMI cost

after adjusting for disease-related factors such as the number of complications, the number of major

procedures, different chronic diseases, and the cost difference associated with different states and

hospitals. The latter factor may imply the labor, material cost, and/or disease severity nature in different

states and hospitals. Those baseline averages are shown in Figures 11, 12, and 13. For example, Nevada

has the highest AMI treatment cost ($125.7K/visit). After standardization, the baseline average in

Nevada ranks 4th in the country ($32.6K/visit at the baseline) while New Jersey and California rank 1st

and 2nd, respectively.

These baseline averages allow insurance companies and healthcare providers to compare the cost in a

standardized level. This result agrees with the fact that the labor and material costs in New Jersey and

California are clearly higher and disease severity in Nevada is usually higher. The disease severity

variations in Nevada clearly drive the unstandardized AMI cost higher in the state.


Figure 11: AMI Treatment Cost Averages for Each State, Before Standardization

Figure 12: Baseline Average of AMI Treatment Costs for Hospitals, After Standardization


Figure 13: Baseline Average of AMI Treatment Cost for States, After Standardization

Baseline average of AMI treatment cost for states after standardization

Developing a model to predict in-hospital mortality

AMI costs many Americans their lives by its acute and devastating nature. Most patients die either on

the scene where the patient experienced acute heart attack or on the way to the hospital. Even in

hospitals, the mortality is high compared with other diseases. To understand what factors contribute to

high in-hospital mortality and improve the quality of healthcare, a random forest model was developed

in this study. A total of 127,016 AMI records were sampled from the United States in 2011. Among those

records, 6697 patients died in hospitals (5.27%).

Many predictors were treated as categorical variables such as ICD 9–M diagnostic codes, race, gender,

chronic conditions, and different insurance plans. The decision tree building and testing processes were

described in the methodology section. The original 69 variables were put into random forest model and

the model will choose the most significant predictors by computing variable importance scores by

permutation.

Since the imbalanced data in which death cases constitute 5.27% where survivals constitute 94.73%, the

accuracy of the prediction of death is only 48% in the initial model. After application of the bootstrap

sampling technique, the accuracy for both death and survival predictions were improved to around 89%.

As shown in Figures 14 and 15, length of stay, diagnostic code –DX1, the number of chronic conditions,

mortality risk score assigned by doctors, and procedure delay in days for procedure 1, 8, and 4 are

significant factors in the prediction of mortality. The application of the random forest model provides a


first time identification of significant factors affecting in-hospital mortality, and can be used to predict

death ahead of time and improve the management of operating rooms in hospitals.

Figure 14: Risk factors for AMI in-hospital mortality – Procedure 1 delay in days, financing source, and number of chronic

diseases

MeanDecreaseAccuracy measures the accuracy difference between the observed data and the

permutated data set for the selected variable. Larger values indicate more importance to predicted

mortality.

Figure 15: Random forest model predicts AMI in-hospital mortality with ~90% accuracy


Discussion of Results / Conclusions

Summary

AMI is one of the most expensive and fatal chronic conditions treated within the US. This affects not

only patient survival but challenges healthcare institutions and professionals to provide quality, cost-

effective care. Large variations in diagnoses, treatments, and other factors result in varying levels of

survival and costs.

This paper outlines an approach (using the hierarchical generalized linear regression model) for

assessing comparative inpatient hospital case mixes, costs, and outcomes for AMI on a risk-adjusted

basis. This model is consistent with publicly reported quality measurement approaches used by CMS and

supporting studies.9

The study sample was based on a subset of publicly gathered and well-defined patient, treatment, cost,

and comorbidity data associated with patients incurring a primary discharge diagnosis of AMI. The

measured patient outcomes are based on risk-adjustments for comorbid conditions. AMI treatment cost

analysis, and the associated hospital cost ranking, include adjustment for geographic and random

effects. An inpatient morbidity prediction model, demonstrated to provide a 90% confidence interval,

permits institutions to identify those factors which are most likely to lead to near-time inpatient death

from AMI.

Our study analysis reveals substantial variations in AMI treatment costs, adjusted for quality and

outcomes, at an institutional level. These findings can be used by institutions to further analyze drivers

in quality care practices and outcomes. Payers and Accountable Care Organizations (ACO’s) can leverage

these findings to improve patient care quality by partnering with those institutions demonstrating the

highest quality and most cost-effective care.

Future Directions

While this study focused on inpatient outcomes and costs associated with AMI, the alignment of its

methodology with CMS analytical approaches makes it suitable for application to the analysis of other

acute disease states, such as heart failure.

With availability of additional, post-discharge data, further analysis can also be conducted into 30-day

outcome and quality measures. These measures can incorporate data related to follow up care, on a

geographically- and random effect-adjusted basis.

Analysis of data across multiple years, integrated with the ability to incorporate broader longitudinal

studies at a patient level (such as through de-identified patient level data, which can reflect detailed

patient and medication histories, as well as treatment histories for comorbid conditions), can provide

additional insights into not just the acute episode of care. These studies could provide improved insight

to long term treatment of key patient risk factors aimed at reducing levels of acute AMI episodes.


References 1. Hierarchical Modeling using HCUP Data, Report# 2007-01

2. Hierarchical Generalized Linear Models in the Analysis of Variations in Health Care Utilization;

Michael J. Daniels a & Constantine Gatsonis

3. CMS 30-Day Risk-Standardized Readmission Measures for Acute Myocardial Infarction (AMI),

Heart Failure (HF), and Pneumonia

4. ACC/AHA Guidelines for the Management of Patients With Acute Myocardial Infarction JACC -

1996

5. 1999 Update: ACC/AHA Guidelines for the Management of Patients With Acute Myocardial

Infarction: Executive Summary and Recommendations – 1999

6. Statistical Models and Patient Predictors of Readmission for Acute Myocardial Infarction

7. Classification and Regression by Random Forest; Andy Liaw and Matthew Wiener, 2002

8. Fitting Linear Mixed-Effects Models Using lme4; Douglas Bates, Martin Mächler, Benjamin M.

Bolker, Steven C. Walker, 2012

9. Statistical Issues in Assessing Hospital Performance, prepared for CMS by the Committee of Presidents of Statistical Societies, 2012

Dell EMC believes the information in this publication is accurate as of its publication date. The

information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” DELL EMC MAKES NO

RESPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS

PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS

FOR A PARTICULAR PURPOSE.

Use, copying and distribution of any Dell EMC software described in this publication requires an

applicable software license.

Dell, EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries.

ami treatment cost & predictive mortality analysis · ami (acute myocardial infarction –...

Documents