Comparison of different statistical methods to predict Intensive Care Length of Stay

Ilona VerburgNicolette de Keizer

Niels Peek

Dept. Of Medical InformaticsAcademic Medical CenterUniversity of Amsterdam

The Netherlands

ESCTAIC 2012,Timisoara

Background

Intensive Care Units (ICUs) assess their performance to improve quality and reduce costs

4-10-2012 2

Background

Background and objective

mortality

length of stay

Case mixEffectiveness of care

Efficiency of care

4-10-2012 3

Length of stay (mean) 10 days 5 days

Age (mean) 68 57

Medical vs surgical 80% medical 40% medical

admission type (%) 20% surgical 60% surgical

ICU Length of stay is influenced by case mix.

Example:

Background and objective

Background and objective

4-10-2012 4

Observed outcome

Predictive model

Expected outcome

ICU

Case mix

Com

pare

Case mix

4-10-2012 5

ObjectiveCompare the performance of different statistical regression methods to predict ICU LoS.

Background and objective

Background

Models exist to predict ICU mortality (example APACHE IV)

Few models exist to predict ICU Length of Stay (LoS)

No consensus about best modelling method

Data

4-10-2012 6

NICE registry

Dutch National Intensive Care Evaluation (NICE)

Registry of ICU admissions in the Netherlands (since 1996)

All admissions from (voluntary) participating ICUs (>90%)

Evaluating (systematically) the effectiveness and efficiency of ICUs in the Netherlands

Identifying quality of care problems

Quality assurance

Database

Data

4-10-2012 7

Exclusion criteria

APACHE IV exclusion criteria

elective surgery

Data

Patients admitted to ICUs participating NICE

2009 - 2011

84 ICUs

81,190 (86.1%) survivors

94,251 (42.4%) admissions

13,061 (13.9%) non-survivors

Included patients

Length of stay

4-10-2012 8

Distribution of Length of Stay in fractional days

ICU survivors (n= 81,190) ICU non-survivors (n= 13,061)

Median: 1.7 (days)Mean: 4.2 Standard deviation: 8.2Maximum: 326.6

Median: 2.4 (days)Mean: 5.9Standard deviation: 10.2Maximum: 139.0

9

ICU Length of Stay

Distribution of discharge time

Modeling ICU length of stay

Different methods to model ICU length of stay (in fractional days)

Ordinary least square (OLS) regression LoS and Log-transformed LoS

Most frequently used method in literature

4-10-2012 10

4-10-2012 11

Modeling ICU length of stay

Different methods to model ICU length of stay (in fractional days)

Ordinary least square (OLS) regression LoS and Log-transformed LoS

General linear models (GLM) Gaussian - difference with OLS is the log link function

Gamma - LoS time until discharge

- depending on chosen parameters positively skewed

Poisson - LoS count data

`-depending on chosen parameters positively skewed

- property: expectation = variance → overdispersion

•Negative binomial - count data

-depending on chosen parameters positively skewed

- generalisation of poisson

4-10-2012 12

Modeling ICU length of stay

Different methods to model ICU length of stay (in fractional days)

Ordinary least square (OLS) regression

LoS and Log-transformed LoS

General linear models (GLM) 4 different families

Gaussian

Gamma

Poisson

negative binomial

Cox proportional Hazard (Cox PH) regression

No assumptions on the shape of the distribution

Omits the need of transform the outcome

4-10-2012 13

Selection of covariates

Starting with large set of variables

Known relationship with LoS (literature)

Stepwise backwards elimination of variables

Modeling ICU length of stay

Included case mix

Demographics

Age

Gender

Admission type

Diagnoses (APACHE IV)

Severity of illness (APACHE IV severity-of-illness score)

Different comorbidities (21)

4-10-2012 14

Performance measures

Validation

Squared Pearson correlation = R2 =

Root Mean squared prediction error (RMSPE) =

Relative BIAS =

Relative mean absolute prediction error (MAPE) =

2

)ˆ()(

)ˆ,(

YY

YYCov

k

kk yEyn

21

kk

kk

kk

yn

yn

Eyn

1

11

kk

kkk

yn

yEyn

1

1

Good prediction

High ↑

Low ↓

Low ↓- or +

Low ↓

Validation

4-10-2012 15

Validation

Performance measures calculated on original data

Correcting for optimistic bias

100 bootstrap samples

4-10-2012 16

Covariates survivorsOLS reg los

OLS reg log los

GLM: gaussian

GLM: poisson

GLM: negative binomial

GLM: Gamma

Cox PH

chronic dialysis -1.04 -0.16 -0.25 -0.26 -0.28 -0.28 0.31cva 0.74 0.1 0.13 0.18 0.26 0.26 -0.3diabetes -0.34 -0.01 -0.07 -0.06 -0.04 -0.04 0.03resperatory insufficient 0.38 0.03 0.06 0.09 0.15 0.15 -0.11spline Aps (1) 5.55 0.64 1.74 1.65 1.61 1.61 -1.52spline Aps (2) 11.07 1.09 3.16 2.78 2.64 2.64 -2.57spline Aps (3) 15.98 0.99 2.07 2 2.08 2.08 -1.79

Results coefficients

Covariates non-survivorsOLS reg los

OLS reg log los

GLM: gaussian

GLM: poisson

GLM: negative binomial

GLM: Gamma

Cox PH

chronic dialysis 0.15 0.08cva -0.68 -0.18 -0.15 -0.12 -0.12 0.09diabetes 0.35 0.03 0.05 0.05 0.06 0.06 -0.05resperatory insufficient -0.51 -0.03 -0.11 -0.1 -0.09 -0.09 0.07spline Aps (1) -5.59 -0.43 -0.94 -0.84 -0.8 -0.8 0.7spline Aps (2) -6.08 -0.73 -1.09 -1.26 -1.53 -1.55 1.54spline Aps (3) -6.47 -0.84 -1.64 -1.76 -1.87 -1.88 1.83

  R2 RMSPE Relative BIAS Relative MAPE

OLS regression (LoS) 0.174 7.448 0.008 0.812

OLS regression (log(LoS)) 0.183 7.714 -0.400 0.674

GLM Gaussian 0.197 7.335 0.001 0.771

GLM Poisson 0.194 7.349 0.000 0.769

GLM Negative Binomial 0.186 7.388 0.005 0.773

GLM Gamma 0.184 7.407 0.005 0.773

Cox PH regression 0.097 9.002 -0.693 0.938

Results validation

4-10-2012 17

ICU survivors

Mean observed > mean expectedUnderestimation of mean LoS

  R2 RMSPE Relative BIAS Relative MAPE

OLS regression (LoS ) 0.107 9.618 0.005 0.891

OLS regression (log(LoS)) 0.107 10.213 -0.510 0.762

GLM Gaussian 0.134 9.462 -0.009 0.868

GLM Poisson 0.128 9.504 0.000 0.872

GLM Negative Binomial 0.12 9.545 -0.001 0.872

GLM Gamma 0.112 9.602 -0.001 0.877

Cox PH regression 0.075 11.388 -0.808 0.906

Results validation

4-10-2012 18

ICU non-survivors

Conclusion and discussion

GLM models shows best performance

Poorest performance found for Cox PH regression

Large relative bias was found for OLS regression of log-transformed LoS

4-10-2012 19

Difficult to predict ICU LoS

Influenced by admission and discharge policy

Seasonal pattern for admission and discharge time

Skewed to the right

Differences in performance between models not statistically tested

Conclusion and discussion

4-10-2012 20

Similar study for CABG patients (Austin et al.), with comparable results

Different patient type

Different distribution of length of stay

Future research

Different models for survivors and non-survivors

combining with mortality in one prediction

Statistical methods to predict ICU LoS

developing a model for benchmarking purposes

4-10-2012 21

Questions?

Thank you for your attention!

APACHE IV Exclusiecriteria

4-10-2012 22

• Age < 16

• ICU admission < 4 hours

• Hospital admission >365 days

• Died during admission

• Readmissions

• Admissions from CCU/IC other hospital

• No diagnose

• Burns

• Transplantations

• Missing hospital discharge