www.postersession.com

)

Benchmarking Critical Care Outcomes: Using data to drive effectiveness and efficiency

Thomas L. Higgins MD MBAVice Chair for Clinical Affairs, Department of Medicine, Baystate Medical Center, Springfield MA

Professor of Medicine, Surgery & Anesthesiology, Tufts University School of Medicine

Resource Utilization Graph from Nathanson et al, Crit Care Med 2007; 35:1853

Project IMPACT Data 2005-06

Hospitals within control limits

Better than expectedMortality and LOS

Driving Change

• Normalized ratios can be created for any outcome: e.g.: ventilator days

• In this example, ventilator days are higher than predicted, indicating an opportunity for improvement

• Interventions could include education, institution of “daily wake-up”, attention to VAP and CLABSI, respiratory therapy protocols, or twice-daily weaning trials

Driving Change Using ICU Benchmarking Tools

• Morbidity and mortality– Evidence-based bundles / ordersets; CPOE, medication

scanning; alerts, early warning– Excess length-of-stay– Admission, discharge, triage policies– Open versus closed units– Ventilator weaning and sedation practices

• Ventilator-associated Pneumonia– Ventilator “bundle” of care including HOB elevation– Respiratory therapy equipment and change-out policies

• Catheter-related Bloodstream Infections– Attention to technique and tools– Operator training restrictions

Length of Stay ReductionMICU + SICU Patients, BMC, 2002-2012excludes Heart & Vascular (CVICU, CCU)

0.0

5.0

10.0

15.0

20.0

25.0

ICU LOS

Hosp LOS

Central Line Associated Blood Stream Infections (CLABSI)

Current ICU Benchmarking Tools

Summary

• Measuring ICU performance requires a balanced scorecard• Outcomes must be severity-adjusted

– Tools include APACHE, MPM, SAPS– Endpoints include mortality, LOS– Normalized ratios/benchmarking can drive change

• Readmission rates must also be severity-adjusted but once adjusted do not correlate with case-mix adjusted mortality or other quality measures, raising questions about CMS use of metric– Kramer et al, Crit Care Med 2013; 41:24-33

• Quality metrics also include CLABSI, VAP, complications and patient satisfaction

• Track employee engagement as well as family satisfaction• Academic institutions may also track research productivity,

teaching evaluations, publications

Other Domains of Interest• Clinical Quality

– Patient and family satisfaction – H-CAHPS Scores• Human Capital

– Engagement, turnover, morale – Gallup EmployeeSurvey • Financial Performance

– Revenue and Costs (Part A and Part B) – Income Statement

– Resource Utilization by provider – Premier Database• Academics: Research and Education –

– Grant funding, number of publications, faculty teaching evaluations (New Innovations)

Model n AUROC HLGOF, p

APACHE-II (1985) 5,815 0.86 nr

APACHE-IV (2002-3) 110,558 0.88 0.08

ICNARC (1995-2003) 216,626 0.87 <0.001

MPM0-II (1993) 12,610 0.84 0.62

MPM0-III (2001-4) 124,885 0.82 0.31

SAPS-II (1993) 13,152 0.86 0.104

SAPS-III (2002) 16,784 0.85 0.39

Standardized Mortality Ratio (SMR)

Observed Risk-Adjusted Mortality

SMR = Expected Risk-Adjusted Mortality

Values 2 SD > 1.0 may indicate poor performanceValues 2 SD < 1.0 indicated superior performance

Patient Diagnosis Predicted Actual

DKA 2% 0

Pneumonia 12% 0

Asthma 10% 0

Acute MI 24% 0

Septic Shock 30% 1

Pneumonia 12% 0

Heart Failure 15% 0

Septic Shock 30% 0

Ruptured AAA 65% 1

Heart Failure 15% 0

AVERAGE: 21.5 0.20

Example of calculating SMR for a hypothetical ICU

One patient each; 10 diagnoses

SMR for this ICU=

Observed (20%)

Predicted (21.5%)

= 0.93

Major Domains of Interest• Clinical Quality

– Standardized mortality rate (observed/expected)– ICU and hospital lengths of stay– Complications (CR-BSI, VAP, “never” events)– Patient and family satisfaction

• Human Capital– Engagement, turnover, morale

• Financial Performance– Revenue and Costs (Part A and Part B)– Resource Utilization by provider

• Academics: Research and Education

Critical Care Medicine in the US: Big business, and growing

• 93,955 CCM beds in 3,150 Hospitals (increasing 1%/yr 2000-2005)• 23.2 million patient days (10.6% increase over 5 years)• Cost per day: $3518 (30.4% increase over 5 years)• Total costs: $ 81.7 Billion (44.2% increase over 5 years)

– Critical Care accounts for 13.4% of hospital costs– 4.1% of national health expenditures– 0.66% of GDP (rate of increase = 3.6% per year)

• Halpern and Pastores, Crit Care Med 2010; 38:67-71• Hospital Mortality Rate for ICU patients: ~6 to 19%

– 13.8% in 124,855 patients, Project IMPACT (2001-2004)• Higgins et al, Crit Care Med 35:827, 2007

– 13.5% in 44,288 patients, APACHE-IV validation (2002-2003)• Zimmerman et al, Crit Care Med 34:1297, 2006

Accurate Risk Stratification Needed

• Mortality outcomes are highly dependent on presenting patient condition– Unadjusted results misleading

•Mortality rate for DKA <2%•Mortality rate for septic shock ~30%

• Case-mix thus affects unadjusted overall mortality rate• Adjusted data is required for internal Quality Improvement efforts• Risk stratification helpful (but not infallible) for individual patient

prognosis• Risk-adjustment models must meet criteria for discrimination (area

under ROC >0.80) and calibration (non-significant HL-GOF)

Who wants to know? Patients, Families, Physicians, Administrators, Insurers, the media…..

AUROC = area under receiver operating curve, ideally >0.80)

HLGOF = Hosmer-Lemeshow Goodness of Fit, ideally >0.05

Worse than expected resource utilization (Length of Stay)