qi talk time - hse.ie...– special cause of variation – assignable cause – unstable process –...

API, 2018

Measurement for Improvement Lloyd Provost 3rd July 2018

Connect Improve Innovate

Building an Irish Network of Quality Improvers QI TALK TIME

API, 2018

Speaker Lloyd P Provost:

An improvement advisor with Associates in Process Improvement who has worked worldwide to apply the science of improvement. His experience includes consulting and teaching in collaborative improvement, planning, management systems, measurement, planned experimentation, and statistical process control. Lloyd is also senior fellow of the Institute of Healthcare Improvement (IHI). Lloyd also works with IHI on their “Improvement Advisor Development Program”.

Lloyd holds has a Bachelor of Science in Statistics from the University of Tennessee and a Master of Science in Statistics from the University of Florida. He is the author of multiple papers relating to quality and measurement and co-author of three books on planned improvement including the Health Care Data Guide (Jossey-Bass, 2011).

API, 2018

Instructions • Interactive

• Sound:

Computer or dial in:

Telephone no: 01-5260058

Event number:840 185 291 #

• Chat box function

– Comments/Ideas

– Questions

• Keep the questions coming

• Twitter: @QITalktime

API, 2018

Lloyd P. Provost Associates in Process Improvement

[email protected]

Measurement for Improvement

3 July, 2018

API, 2018

Webinar Objectives

• Review the key ideas on measuring for improvement.

• Appreciate the importance of analysing measures using time series charts.

• Appreciate learning from special causes.

• Review some examples of the useful types of control charts in healthcare applications.

5

Reference: The Health Care Data Guide, Provost and Murray, 2012

MEASUREMENT FOR IMPROVEMENT

Project Measures: Overall results related to the project

aim (outcome and process

measures). Also Balancing

measures

PDSA Measures Quantitative data on the impact of a

particular change

Qualitative data to help refine the change

Subsets or stratification of project

measures for particular patients

Whatarewetryingtoaccomplish?

Howwillweknowthatachangeisanimprovement?

Whatchangecanwemakethatwillresultinimprovement?

ModelforImprovement

Act Plan

Study Do

From:TheImprovementGuide,AssociatesinProcessImprovement

Aim

Measures

Ideas

Act Plan

Study Do

19 6

API, 2018

March, 1997 The Joint Commission

Journal on Quality Improvement,

Vol 23, No 3.

We are increasingly realizing not

only how critical measurement is to

the quality improvement we seek

but also how counterproductive it

can be to mix measurement for

accountability or research with

measurement for improvement.

7

API, 2018

Percent of Patients Counseled on Smoking Cessation

0

20

40

60

80

100

Jan

06

M M J S N J-

07

M M

%

Percent of Smokers Who Have Not Smoked for Two

Months

0

10

20

30

40

50

60

Jan

06

M M J S N J-

07

M M

%

ArticleFree Gum or Patch

Support GroupE-mail BuddiesArticle

Free Gum or PatchSupport Group

E-mail Buddies

Data for Judgment vs. Improvement

HC Data Guide, p 29, 30 8

API, 2018

Data for Judgment vs. Improvement

Ave Patient Satisfaction Scores

60

70

80

90

100

Jan

06

M M J S N J-

07

M M

Percen

t

Scripting Rec Process Test w ait chg

Patient Satisfaction Percentile Ranking

60

70

80

90

100

Jan

06

M M J S N J-

07

M M

Percen

tile

Scripting Rec Process Test w ait chg

HC Data Guide, p 31 9

API, 2018

Guidelines for Collecting Data for Improvement

• A few key measures that clarify the aim of the improvement effort and make it tangible should be regularly reported throughout the life of the project.

• Be careful about over-doing process measures. A balance of outcome, process and balancing measures is important.

• Plot data visually on the key measures over time. •

• Make use of existing databases and data already collected for developing measures.

• Whenever feasible, integrate data collection for measurement into the daily work routine.


API, 2018

Three Categories of Measures

Outcome Measures: Voice of the customer or patient. How is the system performing? What is the result?

Process Measures: Voice of the workings of the system. Are the parts/steps in the system performing as planned?

Balancing Measures: Looking at a system from different directions/dimensions. What happened to the system as we improved the outcome and process measures? (e.g. unanticipated consequences, other factors influencing outcome)


API, 2018

Family of Measures for Improvement Project

Surgical Infection Rate

0.000

0.010

0.020

0.030

0.040

0.050

Mar A M J J A S O N D J

Rate

Median 0.026

Patient Satisfaction

75

80

85

90

95

100


Perc

ent V

ery

Good/E

x

Median 89

Average Cost per Case

6.4

6.6

6.8

7

7.2

7.4

7.6

7.8


Dolla

rs (

k) Median 7.2

Staff With SafetyCulture Score >4

0

10

20

30

40

50

60

70

80

90

100


Perc

ent

Median 44

Percent On-Time Antibiotic Use

40

50

60

70

80

90

100


Perc

ent

Percent Appropriate Antibiotic Selection

40

50

60

70

80

90

100


Perc

ent

Median 86.2

Median 89.9

Figure 2.27: Surgical Safety Family of Measures HC Data Guide, p 61-64

Process Measure

Process Measure

Process Measure

Balancing Measure

Balancing Measure

Outcome Measure

12

API, 2018

Monthly Measure – Goal = 90%

Month Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov

Measure 83 80 81 84 83 85 68 87 89 92 91

60

65

70

75

80

85

90

95

100

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar

Pe

rce

nt

Run Chart of Measure

Goal

Why a run chart? Why not just a table?


API, 2018

Average Before=8 hours delay Average After=3 hours delay

Need for Run Charts on Improvement Projects

HC Data Guide, p 16-17 14

API, 2018

QI Tools to Learn

from Variation in Data


160

180

200

220

240

260

280

300

320

LO

S (

min

ute

s)

Goal

Work-up done on floor

Bed ahead

Individual responsible

for bed control

Quick-look x-

rays

2/16/98 3/16 4/13 5/11 6/8

Week

Minimum Standard for QI Studies: Annotated Time Series for Outcome and Process Measures

API, 2018

Shewhart’s Theory of Variation

• Common Causes—those causes inherent in the system over time, affect everyone working in the system, and affect all outcomes of the system – Common cause of variation – Chance cause – Stable process – Process in statistical control

• Special Causes—those causes not part of the system all the time or do not affect everyone, but arise because of specific circumstances – Special cause of variation – Assignable cause – Unstable process – Process not in statistical control

HC Data Guide, p. 108 17

API, 2018

Special Causes—those causes not

part of the system all the time or do

not affect everyone, but arise because of specific

circumstances

API, 2018

Shewhart Charts

The Shewhart chart is a statistical tool used to distinguish between variation in a measure due to common causes and variation due to special causes

(Most common name is a control chart, more descriptive would be learning charts or system performance charts)


API, 2018

Why Not Use a Run Chart for Everything?

HC Data Guide, p. 118, 119 20

API, 2018

• Selection of a measure and a statistic to be plotted.

• A method of data collection: observation,

measurement and sampling procedures.

• A strategy for determining subgroups of

measurements (including subgroup size and

frequency).

• Selection of the appropriate Shewhart chart.

• Criteria for identifying a signal of a special cause.

The Method of Shewhart Charts

HC Data Guide p. 114 21

API, 2018

Using Shewhart Chart to Guide

Improvement Work


API, 2018

Learning from Special Causes

23

Conclusion: This case study highlights the utility of statistical

process control in the surveillance and investigation of CA-BSI.

API, 2018

A Special Cause in August

24 26 21 36 34 24 18 24 15 21 27 12 6 6 7 9 9 9 5 8 5 3 8 6 5 8 8 6 6 5 5 4 4 1 4 4 4 9

8340

8741

8546

9034

9080

8890

9324

10054

10450

10667

11007

10491

3861

4094

3827

3625

3706

3434

3439

3790

3757

3651

3705

3840

4180

4484

4177

4159

4353

4306

4545

4326

4834

4476

4595

4483

4620

4874Device

Days

Infections

CCHMC Central Venous Catheter (CVC) Associated

Laboratory Confirmed Bloodstream Infections (LCBIs)

0.2

1.8

0.9

0.8

0.90.9

1.9

2.7

1.11.41.4

0.8

2.2

2.12.4

2.5

1.5

1.8

1.2

1.6

1.31.5

2.6

1.81.6

1.1

2.5

2.0

1.4

2.4

1.9

3.7

4.0

2.5

3.0

2.9

1.2 0.9

0

1

2

3

4

5

6

7

8

Q3 '04

Q4 '04

Q1 '05

Q2 '05

Q3 '05

Q4 '05

Q1 '06

Q2 '06

Q3 '06

Q4 '06

Q1 '07

Q2 '07

Ju

l '0

7

Au

g '07

Sep

'07

Oct

'07

No

v '07

Dec '07

Jan

'08

Feb

'08

Mar

'08

Ap

r '0

8

May '08

Ju

n '08

Ju

l '0

8

Au

g '08

Sep

'08

Oct

'08

No

v '08

Dec '08

Jan

'09

Feb

'09

Mar

'09

Ap

r '0

9

May '09

Ju

n '09

Ju

l '0

9

Au

g '09

Sep

'09

Oct

'09

No

v '09

Dec '09

Jan

'10

Feb

'10

Mar

'10

Ap

r '1

0

May '10

Ju

n '10

FY2005 FY2006 FY2007 FY2008 FY2009 FY2010

Infe

cti

on

s p

er

10

00

De

vic

e D

ay

s

CVC-LCBIs Control Limits Goals [0.8 (Jul06-Jun07) / 1.0 (Jul07-Jun08) ] Baselines

Q3/04 - Revised Care Practices and

Q3/04 - CHG Scrub for Line Care

Q3/04 - CHG Scrub for CVC Insertions

Q4/04 - Maximal sterile barriers and CHG

Q4/04 - Interventional Radiology

New CCHMC

Collaborative

Began, Q1/06

CA-BSI NACHRI

derived CVC bundle

rolled out to hospital

3/15/07

Q2/05 - MaxPlus Cap in PICU B4 & A6

Q2/05 - MaxPlus Cap on A5N2

Q3/05 - MaxPlus Cap cancelled on A5N2

Q3/05 - MaxPlus Caps cancelled.

Updated Thru 31Aug09 by Art Wheeler, Legal/HPCE Depts. Source: Infection Control Dept.

Desired

Direction of

Change

Chart Type: u-chart

3/17/09 - Microclave

Cap Use Housewide

Jan09 - Microclave

Cap Use ICUs

Feb09 - Microclave Cap Use

HemOnc/BMT

From Derek Wheeler, CCHMC 24

API, 2018

Pareto Chart

of Isolated

pathogen Stratified by

Common Cause

and Special

Cause Periods

From Derek Wheeler, CCHMC

Compare Special Cause Period with Common Cause

25

API, 2018

CCHMC Central Venous Catheter (CVC) Associated

Laboratory Confirmed Bloodstream Infections (LCBIs)

0.2

1.8

1.10.9 1.0

0.91.2

2.9

3.0

2.5

4.0

3.7

1.9

2.4

1.4

2.0

2.5

1.1

1.6 1.8

2.6

1.51.3

1.6

1.2

1.8

1.5

2.5

2.4 2.1

2.2

0.8

1.4 1.4 1.1

2.7

1.9

0.9 0.9

0.8

0.9

0

1

2

3

4

5

6

7

8Q

3 '04

Q4 '04

Q1 '05

Q2 '05

Q3 '05

Q4 '05

Q1 '06

Q2 '06

Q3 '06

Q4 '06

Q1 '07

Q2 '07

Ju

l '0

7

Au

g '07

Sep

'07

Oct

'07

No

v '07

Dec '07

Jan

'08

Feb

'08

Mar

'08

Ap

r '0

8

May '08

Ju

n '08

Ju

l '0

8

Au

g '08

Sep

'08

Oct

'08

No

v '08

Dec '08

Jan

'09

Feb

'09

Mar

'09

Ap

r '0

9

May '09

Ju

n '09

Ju

l '0

9

Au

g '09

Sep

'09

Oct

'09

No

v '09

Dec '09

Jan

'10

Feb

'10

Mar

'10

Ap

r '1

0

May '10

FY2005 FY2006 FY2007 FY2008 FY2009 FY2010

Infe

cti

on

s p

er

10

00

De

vic

e D

ay

s

CVC-LCBIs Control Limits Goals [0.8 (Jul06-Jun07) / 1.0 (Jul07-Jun08) ] Baselines

Q3/04 - Revised Care Practices and CHG

Scrub for Line Care

Q3/04 - CHG Scrub for CVC Insertions

Q4/04 - Maximal sterile barriers and CHG

Q4/04 - Interventional Radiology

New CCHMC

Collaborative

Began, Q1/06

CA-BSI NACHRI

derived CVC bundle

rolled out to hospital

3/15/07

Q2/05 - MaxPlus Cap in PICU B4 & A6

Q2/05 - MaxPlus Cap on A5N2

Q3/05 - MaxPlus Cap cancelled on

A5N2

Q3/05 - MaxPlus Caps cancelled.

Desired

Direction of

Change

Chart Type: u-chart

3/17/09 - Microclave

Cap Use Housewide

Jan09 - Microclave

Cap Use ICUs

Feb09 - Microclave Cap

Use HemOnc/BMT

Spiros device

evaluated 8/1/09-

9/2/09

From Derek Wheeler, CCHMC

API, 2018

Rules or determining a special cause


API, 2018

#

of

Ne

ed

les

tic

ks

Employee Needlesticksc c ha r t

UCL = 12.60

Mean = 5.54

New Needles Test

1-05 3-05 5-05 7-05 9-05 11-05 1-06 3-06 5-06 7-06 9-06 11-06 1-07 2-07

0

5

10

15

20

28

API, 2018 29

Identify the signals of special causes on these 4 Shewhart Charts

API, 2018

Chart 1

30

58.3

75.4

LCL 41.3

30.0

40.0

50.0

60.0

70.0

80.0

90.0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Tim

e -

min

ute

s

Week

Time - Chart 1 Measure

API, 2018 31

61.4

UCL 75.4

LCL 47.3

30.0

40.0

50.0

60.0

70.0

80.0

90.0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Tim

e -

min

ute

s

Week


Chart 2

API, 2018 32

61.2

UCL 71.9

LCL 50.5

40.0

45.0

50.0

55.0

60.0

65.0

70.0

75.0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Tim

e-

Min

ute

s

Week


Chart 3

API, 2018

Chart 4

33

60.5

UCL 70.1

LCL 51.0

40.0

45.0

50.0

55.0

60.0

65.0

70.0

75.0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Tim

e (

min

ute

s)

Week


All common cause variation

API, 2018 HC Data Guide p. 151 34

API, 2018

Continuous (aka variables) Data

• Examples of Continuous Data: – Waiting time – LOS – Cost per case for a DRG – Daily patient weight – Time to complete procedure – Number of patients seen per day – Monthly accounts receivable – Temperature – Volume of prescriptions filled

35

API, 2018

Attribute Data Count or Classification Data

Examples of Count or Classification Data:

– # of complications per # of surgeries this month

– # of medication errors per 1000 doses

– Percent of patients who were readmitted

– # of patients who fell per 100 admissions

– Percent of diabetic patients who smoke

36

API, 2018

Control Chart for Continuous Data using Individual Measurements(I Chart)

The I chart for individual data is useful when: – There is no rational way to organize

the data into subgroups.

– Measures of performance of the process can only be obtained infrequently.

– The variation at any one time (within a subgroup) is insignificant relative to the between subgroup variation.

Examples: patient-specific clinical measures, monthly accounting data, laboratory test data, forecasts, and budget variances


API, 2018 38

Individual Patient Number order they presented to ED

Harries, Filiatrault, and Abu-Laban, “Application of quality improvement analytic methodology in

emergency medicine research: A comparative evaluation”, CJEM 2018:1–8.

Example of an I chart A study of Ottawa ankle rules (OAR) by

emergency department (ED) triage nurses.

API, 2018

X-bar and S Shewhart charts

• The collection of data for the construction of X-bar and S Shewhart charts requires that the data be organized in subgroups.

• A subgroup for continuous data is a set of measurements of some characteristic in a process, which were obtained under similar conditions or during the same time period • The subgroup size may vary for the X-bar and S chart. • The X-bar chart contains the averages of each subgroup and the S chart the spread (standard deviation) between the measurements within each subgroup.


API, 2018

Xbar and S Chart- Radiology Test Turn-around

HC Data Guide, p. 160-161

June 7 June

8

June 9 June

10

June

11

June

12

June

13

June

14

June

15

June

16

June

17

June

18

June

19

June

20 Test 1 105 50 58 67 73 57 78 76 18 39 86 32 70 39 Test 2

54 105 26 52 59 64 96 49 60 45 27 47 56 34 Test 3

79 85 38 92 62 24 107 40 39 35 91 57 40 146 Test 4

49 46 72 49 34 Test 5

31 27 23 Test 6

43 65

40

API, 2018 41 “Improving Adherence to Intraoperative Lung-Protective Ventilation Strategies at a University

Medical Center” Josephs, Lemmink, Strong, Barry, Hurford, www.anesthesia-analgesia.org

Intraoperative lung-protective

ventilation (ILPV) is defined

as tidal volumes <8 mL/kg

ideal bodyweight and is

increasingly a standard of

care for major abdominal

surgical procedures

performed under general

anesthesia.

Subgroups

of patients

based on

time periods

Example

Xbar/S Chart

API, 2018

Xbar S – Subgroups by Provider and Quarter D

ays

Average LOS by Provider for DRG XXX

3.75 4.00 3.62 4.35 3.79 4.12 3.71 3.93 3.62 3.81 3.68 3.50 5.18 4.65 4.27 3.95 3.89 3.75 4.13 4.42Ave LOS

Xbar chart

Sigma chart

UCL = 4.69

Mean = 4.00

LCL = 3.32

UCL = 1.93

Mean = 1.45

LCL = 0.96

21/Q1 Q2 73/Q1 Q2 35/Q1 Q2 18/Q1 Q2 10/Q1 Q2 31/Q1 Q2 66/Q1 Q2 45/Q1 Q2 89/Q1 Q2 17/Q1 Q2

2.0

2.5

3.0

3.5

4.0

4.5

5.0

5.5

6.0

0.5

1.0

1.5

2.0

2.5


API, 2018

Shewhart Charts for Attribute Data

Classifications of units: each unit is classified as either conforming or

nonconforming, pass/fail, blue/not blue, go/no-go, etc.

Count of incidence: a count of the number of nonconformities, defects

(complications, infections, errors), accidents, trips, telephone calls, etc.

Chart Type of Subgroup

Name Attribute Data Statistic Charted Size

P Chart classification % nonconforming units (P) constant or may vary

C Chart count number of incidence (C) constant

U Chart count incidents per unit (U) may vary

Table 5.4: Three Types of Attribute Shewhart Charts


API, 2018

P- Charts Example

45 Zafar MA, et al. BMJ Qual Saf 2017;26:908–918. doi:10.1136/bmjqs-2017-006529

API, 2018

P chart as Funnel Plot

P chart for MRSA Subgroups are Hospitals

46

API, 2018

C- Chart #

of N

ee

dle

stic

ks

Employee Needlesticksc chart

UCL = 12.60

Mean = 5.54

New Needles Test

1-05 3-05 5-05 7-05 9-05 11-05 1-06 3-06 5-06 7-06 9-06 11-06 1-07 2-07

0

5

10

15

20

HC Data Guide, p. 177-178 47

API, 2018

Figure 1

Example U Chart

API, 2018

“Cases Between” Occurrences of Rare Events

Instead of plotting the

number of incidences

each month, plot the time

(or number of cases,

patients, visits, etc.)

between incidences.

Plot a point each time an

incidence occurs

HC Data Guide: p. 227

API, 2018

g-chart for Doses Dispenses between ADE’s

HC Data Guide: p. 228, 230

gbar = average of g’s (units between events)

CL = 0.693 * gbar (estimate of median for geometric distribution)

UL = gbar + 3 * square root [gbar * (gbar +1)] 51

API, 2018

T chart Showing Improvement

HC Data Guide: p. 233 52

API, 2018

Establishing and Revising Limits for a Shewhart Charts

HC Data Guide p. 122

Update Trial Limits

Start with a Run Chart

Calculate Trial Limits

53

API, 2018


API, 2018

Updating Limits

56

API, 2018

Deming, W. Edwards: Foreword in

Statistical Method from the Viewpoint of

Quality Control. Dover Publications,

1986, p. ii. ONLY 18 MORE YEARS!

57

API, 2018

Helpful links Framework for Improving quality

www.qualityimprovement.ie Improvement Knowledge and Skills Guide

http://www.hse.ie/eng/about/Who/QID/aboutQID/

http://www.qualityimprovement.ie/

API, 2018

Thank you from all the team @QITalktime

[email protected]

[email protected]

Follow us on Twitter @QITalktime

Watch recorded webinars on HSEQID

QITalktime page

Please give us some feedback on survey monkey:

https://www.surveymonkey.com/r/8WNQVH9

We will be back with some exciting events post

summer.



qi talk time - hse.ie...– special cause of variation – assignable cause – unstable process –...

Documents