Standardised Mortality Ratios& their monitoring

Paul Hawgood

Everything that you wanted to know about SHMI but were too afraid to ask!

Housekeeping

Join the conversation..

#SaferNHS

@AQUA_Inform

Agenda (Pre Coffee)Item TimeWelcome, introductions, housekeeping

9:30

Planned outcomes for the workshop 9:45Quiz 9:50An introduction to SMRsHow SHMI is calculatedDifferences between SMRs What to look for in the dataMyth busters

10:00-

11:30

Break

Agenda (Post Coffee)

Item TimeQ&A Session 11:30Quiz – yes, again! 11:45How do I engage with AQuA? 12:00How do I engage with my local trust / health economy/ social services?

12:15

Lunch 12:30

Outcomes• Increase your understanding of SMRs in

general and, specifically, SHMI and HSMR so that you can:– work with local acute Trusts on an informed basis– look to reduce mortality rates as a whole health economy

• Provide you with the knowledge to ask the right questions of providers.

• Place SMR in context/relation to other indicators of the quality of care.

It’s not about the data!“It’s true that cancer care has come on in leaps and bounds but we can’t look at our comparative survival rates and think anything other than we need to do more.”

Professor Sir Bruce Keogh, National Medical Director 24/3/2014

Content• My approach / house rules• Crude rate• Standardised Mortality Ratios [SMRs]• Different methodologies• SHMI in detail • Interpretation• Myth busters• Questions

Crude [mortality] rate x 100 x 100

• Population-based rates– CCG mortality rates– GP Practice-based disease prevalence

Problem?

Standardised Mortality Ratios• Standardising means “making adjustments”

– age, sex, severity of condition– admission type, deprivation, palliative care

• Indirect and direct standardisation• Calculates a new, standardised mortality

rate

• ‘Rate’, ‘Ratio’ or ‘Index’?

x 100

Damned lies“Essentially, all models are wrong but some are useful.”

George E.P. Box 1951

SHMI Oct 2011 to Oct 2012 (NW Trusts)

Methodologies

• 3 main methodologies in NHS• SHMI, HSMR & RAMI

– Describe – Similarities– Differences– Pros & Cons

Summary Hospital-Level Mortality Indicator

• SHMI• Published by NHS [HSCIC]• Published quarterly

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Oct-12 n n n n n n n n n n n nJan-13 n n n n n n n n n n n nApr-13 n n n n n n n n n n n nJul-13 n n n n n n n n n n n nOct-13 n n n n n n n n n n n nJan-14 n n n n n n n n n n n nApr-14 n n n n n n n n n n n nJul-14 n n n n n n n n n n n n

2011 2012SHMI issue date

Data used in the SHMI calculations

2013

Ja n

Summary Hospital-Level Mortality Indicator

• SHMI• Published by NHS [HSCIC]• Published quarterly• Key Differences

– 137 Trusts– Out-of-hospital deaths (30 days post

discharge)– Agnostic to Palliative Care

Hospital Standardised Mortality Ratio• HSMR• Published by Dr Foster Intelligence

– Hospital Guide

• Fore-runner to SHMI• Published annually / quarterly• Key differences

– All trusts– Explicit adjustment for deprivation– Palliative Care

Risk Adjusted Mortality Indicator

• RAMI• Published by CHKS• Published ??• Key differences

– HRG not first episode– Excludes spells with Palliative Care

Key DifferencesAttribute SHMI HSMR

Hospitals included Excludes specialist, MH & independent

All providers (SUS)

Basket of conditions No exclusions Basket of 56 diagnostic groups which relate to c. 80% of activity

Death attributed to… Trust that patient died in or was discharged from (30 days)

All trusts involved in the patient’s care during the ‘super-spell’

Out-of -hospital Includes deaths up to 30 days post discharge

Not included

Palliative Care No adjustment made Relative-risk weighting applied

Deprivation Proxy via co-morbidity [?] Relative-risk weighting applied

How SHMI is calculated

• SHMI Specification 1.14• Observed deaths• Expected deaths• Outliers

Observed deaths• HES data [from CDS; from the Trust]• Patients dying in hospital

– Discharge method = 4 (therefore, not Stillbirths)

– Excluding daycases, regular day attenders, regular night attenders [Classpat = 2,3 or 4]

• Patients that died within 30 days of discharge– ONS data; linked to HES data

How SHMI is calculated• SHMI Specification 1.14• Observed deaths• Expected deaths

Expected Deaths (1)• All discharges (apart from noted exclusions)• Split them up into all possible combinations

of risk factors (i.e. stratify)– How many groups?

Group Number

Admission Method 3

Age 21

Charlson Index 3

Gender 3

Year 3

CCS Group 140

Total 238,140

Expected deaths (2)

Bottom 10 and top 10 Intercept Values

Groups of Codes ICD10 Codes16,000

CCS Categories260

CCS Groups

140

Groups of Codes in practice

• Primary diagnosis of first episode [FFCE]• Unless primary diagnosis is an ‘R’ code; in

which case use primary diagnosis of 2nd FCE• Unless this is also an ‘R’ code; in which case

revert to the primary diagnosis of the FFCE !!!!

Charlson Co-morbidity Index

• All Secondary diagnoses of first episode [FFCE]

• Unless primary diagnosis is an ‘R’ code; in which case use secondary diagnoses of 2nd FCE

• Unless this is also an ‘R’ code; in which case revert to the secondary diagnoses of the FFCE !!!!

Outliers• Data are put into a funnel plot

Outliers• Data are put into a funnel plot• Poisson distribution• Over-dispersion [page 16]• 3 bands

Interpretation (SHMI)• 3 bandings (when compared to England

average)– Higher than expected– As expected– Lower than expected

Interpretation (SHMI)• 3 bandings (when compared to England

average)– Higher than expected– As expected– Lower than expected

• Trends

Interpretation (SHMI)• 3 bandings (when compared to England

average)– Higher than expected– As expected– Lower than expected

• Trends• What is driving a change?

– Observed or Expected– CCS Groups, week-end and the rest…

Observed or Expected?

Interpretation (SHMI)• 3 bandings (when compared to England

average)– Higher than expected– As expected– Lower than expected

• Trends• What is driving a change?

– Observed or Expected– CCS Groups, week-end and the rest…

Myth-busters• We’re in a highly deprived area• We have some data recording issues, that’s all• We provide high levels of Palliative Care• We have audited every death and found no

issues• We have a high proportion of [SHMI] deaths

that are out of hospital• CCS Groups are useful• The London effect

Deprivation (1)

• SHMI makes no adjustment for social deprivation– It might create the impression that a higher

death rate for those who are more deprived is acceptable and has the potential to remove from the SHMI some of the differences that it is designed to measure

Deprivation (2)

Deprivation (3)

Deprivation (4)

Data/coding issues (1)

Data/coding issues (2)

• R codes

Palliative Care (1)• SHMI: not adjusted for

– Included in published “Contextual” Data

• HSMR: adjusted for• RAMI: Spells discounted

Palliative Care (2)

Audit of deaths• Why do it?

– Learn lessons– Check coding

• Expected deaths based upon all discharges• CCS Group analysis

• PRISM2– Nick Black et al.– 2,000 case-note reviews, nationally

Out of hospital deaths (1)

Out of hospital deaths (2)• It is desirable to have fewer deaths

– It is desirable to have fewer deaths in hospital– It is desirable to have fewer deaths <30 days

of discharge

• Does the ratio of the 2 figures matter?

Out of hospital deaths (3)

Baseline Scenario 1 Scenario 2 Scenario 30

200

400

600

800

1,000

1,200

In-hosp Out hosp

25%

44% 9%

9%

Out of hospital deaths (4)• Factors causing a higher rate

– Patient discharged too early– Poor care in the community– Good hospice care / support to die at home

• Factors causing a lower rate– The opposite of the above– Good systems to prevent unnecessary

admissions, especially in those likely to die

• Desirable things could cause a higher rate or a lower rate – so, it doesn’t matter.

Out of hospital deaths (5)

CCS Groups• 140 Groups• Alerts• Relies on accurate coding• Identifies clinical areas to look at…• …or does it? • Congruence with death certificate

The London effect

Any questions?

Pack• Quiz – with answers• SHMI Specification• Latest quarterly mortality report• PRISM2 review form• Palliative Care coding – first 2 pages • ccvol7issue4 Palliative Care Coding.pdf

Portal• Avoidable mortality

– Amenable to healthcare– Preventable deaths

• Lessons Learned + checklist

Next time…• CUSUM alerts• VLADs• GP/CCG SHMI !!!• Severe Winter!!!

Next Steps …• What do you want from AQuA?• CCG co-design session 16th July (29th

July)• ? other masterclasses• Handouts & Links post session