Comparison of different statistical methods to predict Intensive Care Length of Stay

Ilona VerburgNicolette de Keizer

Niels Peek

Dept. Of Medical InformaticsAcademic Medical CenterUniversity of Amsterdam

The Netherlands

ESCTAIC 2012,Timisoara

BackgroundIntensive Care Units (ICUs) assess their performance to improve quality and reduce costs

4-10-2012 2

Background

Background and objective

mortality

length of stay

Case mixEffectiveness of care

Efficiency of care

4-10-2012 3

Length of stay (mean) 10 days 5 days

Age (mean) 68 57

Medical vs surgical 80% medical 40% medicaladmission type (%) 20% surgical 60% surgical

ICU Length of stay is influenced by case mix.Example:

Background and objective

Background and objective

4-10-2012 4

Observed outcome

Predictive model

Expected outcome

ICU

Case mix

Com

pare

Case mix

4-10-2012 5

ObjectiveCompare the performance of different statistical regression methods to predict ICU LoS.

Background and objective

BackgroundModels exist to predict ICU mortality (example APACHE IV)Few models exist to predict ICU Length of Stay (LoS)No consensus about best modelling method

Data

4-10-2012 6

NICE registryDutch National Intensive Care Evaluation (NICE)Registry of ICU admissions in the Netherlands (since

1996)All admissions from (voluntary) participating ICUs (>90%)

Evaluating (systematically) the effectiveness and efficiency of ICUs in the NetherlandsIdentifying quality of care problemsQuality assurance

Database

Data

4-10-2012 7

Exclusion criteriaAPACHE IV exclusion criteriaelective surgery

DataPatients admitted to ICUs participating NICE2009 - 201184 ICUs

81,190 (86.1%) survivors

94,251 (42.4%) admissions

13,061 (13.9%) non-survivors

Included patients

Length of stay

4-10-2012 8

Distribution of Length of Stay in fractional days

ICU survivors (n= 81,190) ICU non-survivors (n= 13,061)

Median: 1.7 (days)Mean: 4.2 Standard deviation: 8.2Maximum: 326.6

Median: 2.4 (days)Mean: 5.9Standard deviation: 10.2Maximum: 139.0

9

ICU Length of Stay

Distribution of discharge time

Modeling ICU length of stay

Different methods to model ICU length of stay (in fractional days)

Ordinary least square (OLS) regression LoS and Log-transformed LoS Most frequently used method in literature

4-10-2012 10

4-10-2012 11

Modeling ICU length of stay

Different methods to model ICU length of stay (in fractional days)

Ordinary least square (OLS) regression LoS and Log-transformed LoS

General linear models (GLM) Gaussian - difference with OLS is the log link functionGamma - LoS time until discharge

- depending on chosen parameters positively skewedPoisson - LoS count data

`-depending on chosen parameters positively skewed- property: expectation = variance → overdispersion

•Negative binomial - count data -depending on chosen parameters positively skewed- generalisation of poisson

4-10-2012 12

Modeling ICU length of stay

Different methods to model ICU length of stay (in fractional days)Ordinary least square (OLS) regression LoS and Log-transformed LoS

General linear models (GLM) 4 different families Gaussian Gamma Poisson negative binomial

Cox proportional Hazard (Cox PH) regressionNo assumptions on the shape of the distributionOmits the need of transform the outcome

4-10-2012 13

Selection of covariatesStarting with large set of variablesKnown relationship with LoS (literature)Stepwise backwards elimination of variables

Modeling ICU length of stay

Included case mixDemographicsAgeGenderAdmission typeDiagnoses (APACHE IV)Severity of illness (APACHE IV severity-of-illness score)Different comorbidities (21)

4-10-2012 14

Performance measures

Validation

Squared Pearson correlation = R2 =

Root Mean squared prediction error (RMSPE) =

Relative BIAS =

Relative mean absolute prediction error (MAPE) =

2

)ˆ()()ˆ,(

YYYYCov

k

kk yEyn

21

kk

kk

kk

yn

yn

Eyn

1

11

kk

kkk

yn

yEyn

1

1

Good prediction

High ↑

Low ↓

Low ↓- or +

Low ↓

Validation

4-10-2012 15

ValidationPerformance measures calculated on original dataCorrecting for optimistic bias100 bootstrap samples

4-10-2012 16

Covariates survivorsOLS reg los

OLS reg log los

GLM: gaussian

GLM: poisson

GLM: negative binomial

GLM: Gamma

Cox PH

chronic dialysis -1.04 -0.16 -0.25 -0.26 -0.28 -0.28 0.31cva 0.74 0.1 0.13 0.18 0.26 0.26 -0.3diabetes -0.34 -0.01 -0.07 -0.06 -0.04 -0.04 0.03resperatory insufficient 0.38 0.03 0.06 0.09 0.15 0.15 -0.11spline Aps (1) 5.55 0.64 1.74 1.65 1.61 1.61 -1.52spline Aps (2) 11.07 1.09 3.16 2.78 2.64 2.64 -2.57spline Aps (3) 15.98 0.99 2.07 2 2.08 2.08 -1.79

Results coefficients

Covariates non-survivorsOLS reg los

OLS reg log los

GLM: gaussian

GLM: poisson

GLM: negative binomial

GLM: Gamma

Cox PH

chronic dialysis 0.15 0.08cva -0.68 -0.18 -0.15 -0.12 -0.12 0.09diabetes 0.35 0.03 0.05 0.05 0.06 0.06 -0.05resperatory insufficient -0.51 -0.03 -0.11 -0.1 -0.09 -0.09 0.07spline Aps (1) -5.59 -0.43 -0.94 -0.84 -0.8 -0.8 0.7spline Aps (2) -6.08 -0.73 -1.09 -1.26 -1.53 -1.55 1.54spline Aps (3) -6.47 -0.84 -1.64 -1.76 -1.87 -1.88 1.83

  R2 RMSPE Relative BIAS Relative MAPE

OLS regression (LoS) 0.174 7.448 0.008 0.812

OLS regression (log(LoS)) 0.183 7.714 -0.400 0.674

GLM Gaussian 0.197 7.335 0.001 0.771

GLM Poisson 0.194 7.349 0.000 0.769

GLM Negative Binomial 0.186 7.388 0.005 0.773

GLM Gamma 0.184 7.407 0.005 0.773

Cox PH regression 0.097 9.002 -0.693 0.938

Results validation

4-10-2012 17

ICU survivors

Mean observed > mean expectedUnderestimation of mean LoS

  R2 RMSPE Relative BIAS Relative MAPEOLS regression (LoS ) 0.107 9.618 0.005 0.891

OLS regression (log(LoS)) 0.107 10.213 -0.510 0.762

GLM Gaussian 0.134 9.462 -0.009 0.868

GLM Poisson 0.128 9.504 0.000 0.872GLM Negative Binomial 0.12 9.545 -0.001 0.872GLM Gamma 0.112 9.602 -0.001 0.877Cox PH regression 0.075 11.388 -0.808 0.906

Results validation

4-10-2012 18

ICU non-survivors

Conclusion and discussion

GLM models shows best performance

Poorest performance found for Cox PH regression

Large relative bias was found for OLS regression of log-transformed LoS

4-10-2012 19

Difficult to predict ICU LoS

Influenced by admission and discharge policySeasonal pattern for admission and discharge time

Skewed to the right

Differences in performance between models not statistically tested

Conclusion and discussion

4-10-2012 20

Similar study for CABG patients (Austin et al.), with comparable results

Different patient type

Different distribution of length of stay

Future research

Different models for survivors and non-survivors

combining with mortality in one prediction

Statistical methods to predict ICU LoS

developing a model for benchmarking purposes

4-10-2012 21

Questions?

Thank you for your attention!

APACHE IV Exclusiecriteria

4-10-2012 22

• Age < 16

• ICU admission < 4 hours

• Hospital admission >365 days

• Died during admission

• Readmissions

• Admissions from CCU/IC other hospital

• No diagnose

• Burns

• Transplantations

• Missing hospital discharge