providing actionable healthcare analytics at scale: understanding improvement measures

Providing actionable healthcare analytics at scale:

Understanding improvement measures

Thomas Woodcock, Improvement Science Fellow, Imperial

College London

NIHR CLAHRC Northwest London

Collaboration for Leadership in Applied Health Research and Care

Northwest London

The National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care (NIHR CLAHRC) Northwest London

is hosted by Chelsea and Westminster Hospital NHS Foundation Trust and academically led by Imperial College London, in partnership with North West London

Understanding Improvement

Measures

Dr Tom Woodcock Programme Lead Information Intelligence

Health Foundation Improvement Science Fellow


To achieve this: Develop a comprehensive systematic approach to

improvement which is transferable and generalizable across the NHS

3

Primary aims

Translate research into practice

Improve health outcomes and patient experience

Deliver value locally and to the wider NHS

Working in collaboration with: Patients, communities, health and care

services, healthcare professionals, academics, universities, third sector,

industry…



Northwest London



Measurement for Improvement


Solberg et al 1997

Three reasons for measurement


Measurement for Improvement

A few simple and specific measures,

linked to your objectives and aims, and

available in real time

to demonstrate whether changes are

making improvements.

NHS Institute for Innovation and Improvement


Measurement in, and of,

Improvement

• Are changes happening in the way health

care is delivered?

• Are these associated with improvements

for patients, carers & the public?

• Are these changes causally linked?

• How can we reproduce this improvement

elsewhere?


The measurement cycle

1 Decide aim

2 Choose measures

3 Confirm collection

6 Take appropriate action

5 Analyse & present

8 Repeat steps 4-6

4 Collect data

7 Review measures

5 Analyse & present

Adapted from a slide by Mike Davidge


What to measure?

Aim Concept Measure

Operational Definitions

Data Collection

Plan

Data Collection

Analysis Action

R. Lloyd. Quality Health

Care: A Guide to

Developing and Using

Indicators.


Aims

Measures

Ideas

(NOT Solutions)

How do we change?

Ideas -> Solutions

The Model for Improvement


Enumerative vs Analytic Study

On Probability As a Basis For Action, W E Deming, The American Statistician, Vol. 29 No. 4 1975, pp. 146-152

(and previously in ‘42 and ‘50)

“An enumerative study has for its aim an estimate of the number of units of a frame that belong to a specified class.

An analytic study has for its aim a basis for action on the cause-system or the process, in order to improve product of the future”


Enumerative vs Analytic Study

Analytical studies: a framework for quality improvement design and analysis, Lloyd P Provost BMJ Qual Saf 2011; 20 (Suppl. 1) doi:10.1136/bmjqs.2011.051557

“Because of the temporal nature of improvement, the theory and methods for analytical studies are a critical component of the science of improvement.”

Analogy: pond vs river


SPC: Origins

• Dr. Walter Shewhart: physicist & engineer,

Western Electric and Bell Laboratories,

1920s. Quality in manufacturing.

• Dr. W. Edwards Deming extended

Shewhart’s work, developing and

explaining applications (U.S. Then Japan

after WWII, worldwide in the 80s and 90s).

• Increasingly seen in healthcare – but

applied with varying degrees of rigor and

success.


Understanding variation

Everything we can measure will vary

Statistical Process Control separates out Special Cause

from Common cause (routine variation)

– Common: inherent to the process, none dominant

– Special: due to a specific reason, dominant

Distinguishing between the two is crucial for good decision

making

SPC provides tools to make this distinction

– run and control charts



Northwest London



An Example QI Initiative


An Example QI Initiative


Physical health and severe and

enduring mental illness

“Adults with serious mental illness represent the

greatest and least recognized health disparity...

reflected in a 13- to 30-year reduced life expectancy.

The primary cause... is cardiovascular disease

associated with disproportionately high rates of

obesity and tobacco use... the greatest current

challenge to providing appropriate mental health

services is not a knowledge gap, but an

implementation gap.” Bartels, Am J Psych, 2015


Don’t just screen- intervene


Linking project aims and activities…


Untangling systems and processes…


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Run Chart - Percentage of Admissions with BMI Recorded A MENTAL HEALTH WARD IN NORTHWEST LONDON

Percentage with BMI recorded

Baseline Median (21%)

Baseline Median - Extended

New process median (61%)

Baseline audit (247 admissions)

Intervention period (311 admissions)



Northwest London



Additional Examples


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

02-Jan 02-Feb 02-Mar 02-Apr 02-May 02-Jun 02-Jul 02-Aug 02-Sep 02-Oct 02-Nov 02-Dec

COPD Care Bundle - Hospital X Overall Compliance

Percentage Compliance

Average

Lower Control Limit


0%

5%

10%

15%

20%

25%

30%

Percentage EDattendances admitted

SeasonalisedAverage

Seasonalised LowerProcess Limit

Seasonalised UpperProcess Limit

Monthly % ED Attendances Admitted; England 2011-2014

0

0.1

0.2

0.3

0.4

0.5

0.9

0.92

0.94

0.96

0.98

1

1.02

1.04

1.06

1 2 3 4 5 6 7 8 9 101112

Ran

ge C

hart

Av

era

ge C

hart

Month

Average and Range Chart for Monthly Seasonal Factors

Average

Grand Average

Lower AverageLimitUpper AverageLimit



Northwest London



What next?


We conclude that

although informal theory is

always at work in

improvement, practitioners

are often not aware of it or

do not make it explicit.

The germane issue for

improvement practitioners,

therefore, is not whether

they use theory but

whether they make explicit

the particular theory or

theories, informal and

formal, they actually use.


Safer Clinical Systems

Evaluation findings (http://www.health.org.uk/publication/safer-clinical-systems-

evaluation-findings)

http://www.health.org.uk/publication/safer-clinical-systems-evaluation-findings











Northwest London



Plan

Explore

Design

Test and Scale Up

Spread

Measure

ment P

lannin

g

Support T

ool

Measure

ment fo

r

Impro

vem

ent S

upport T

ool

SP

C P

ublic

atio

n

Sta

ndard

s

WIS

H

Measure

Defin

ition

Pro

cess M

easure

ment T

rain

ing

(inc. Q

I4U

)

Technic

al s

upport


Measurement for Improvement -

Summary

• Goal is improvement – analytic study

• Methodology required is different – SPC

• Not an excuse to sacrifice rigour!

• QI methods support good practice – if used

correctly

• Further research required – how to reliably

achieve good measurement

• More support and guidance is needed for QI

teams

providing actionable healthcare analytics at scale: understanding improvement measures

Healthcare