interpreting run charts and shewhart charts

50
Interpreting Run Charts and Shewhart Charts

Upload: kaemon

Post on 10-Feb-2016

183 views

Category:

Documents


3 download

DESCRIPTION

Interpreting Run Charts and Shewhart Charts. Agenda. Features of Run Charts Interpreting Run Charts A quick mention of variation Features of Shewhart Charts Interpreting Shewhart Charts. Displaying Key Measures over Time – Run Chart. Data displayed in time order Time is along X axis - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Interpreting Run Charts and Shewhart Charts

Interpreting Run Charts and Shewhart Charts

Page 2: Interpreting Run Charts and Shewhart Charts

Agenda

• Features of Run Charts

• Interpreting Run Charts

• A quick mention of variation

• Features of Shewhart Charts

• Interpreting Shewhart Charts

Page 3: Interpreting Run Charts and Shewhart Charts

Displaying Key Measures over Time – Run Chart

• Data displayed in time order

• Time is along X axis

• Result along Y axis

• Centre line = median

• One “dot” = one sample of data

0

20

40

60

80

100

Perc

ent

Process: Cardiac Surgical Patients with Controlled Post-operative Serum Glucose

Page 4: Interpreting Run Charts and Shewhart Charts

Process Improvement: Isolated Femur Fractures

0

200

400

600

800

1000

1200

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64Sequential Patients

Minu

tes E

D to

OR pe

r Pa

tient

Median 429

1. Determine if change is an improvement

Three Uses of Run Charts in Quality Work

The Data Guide, p 3-18

Page 5: Interpreting Run Charts and Shewhart Charts

Median 429

Three Uses of Run Charts in Quality Work

The Data Guide, p 3-18

2. Determine if improvement is sustained

Holding the Gain: Isolated Femur Fractures

0

200

400

600

800

1000

1200

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64Sequential Patients

Minu

tes

ED to

OR

per

Patie

nt

Page 6: Interpreting Run Charts and Shewhart Charts

Median 429

3. Make process performance visible

Three Uses of Run Charts in Quality Work

The Data Guide, p 3-18

Current Process Performance: Isolated Femur Fractures

0

200

400

600

800

1000

1200

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64Sequential Patients

Minu

tes E

D to

OR

per

Patie

nt

Page 7: Interpreting Run Charts and Shewhart Charts

How Do We Analyze a Run Chart?

• Visual analysis first• If pattern is not clear, then apply probability based rules

The Data Guide, p 3-10

% Timely Reperfusion

010

2030

405060

7080

90100

J-05

F M A M J J A S O N D J-06

F M A M J J A S O N D J-07

F M

Per

cent

Median 35.5

Figure 3.11: Run Chart with ED Team Uncertain About Improvement

Protocol V.3-Test Protocol V.1-Test Protocol V.2-Test

Page 8: Interpreting Run Charts and Shewhart Charts

Non-Random Signals on Run Charts

A Shift: 6 or more

An astronomical data point

Too many or too few runs

A Trend5 or more

The Data Guide, p 3-11

Evidence of a non-random signal if one or more of the circumstances depicted by these four rules are on the run chart. The first three rules are violations of random patterns and are based on a probability of less than 5% chance of occurring just by chance with no change.

Page 9: Interpreting Run Charts and Shewhart Charts

Source: Swed, Frieda S. and Eisenhart, C. (1943) “Tables for

Testing Randomness of Grouping in a Sequence of Alternatives.” Annals of Mathematical Statistics. Vol. XIV,

pp. 66-87, Tables II and III.

The Data Guide, p 3-14

Page 10: Interpreting Run Charts and Shewhart Charts

Trend? Note: 2 same values – only count one

Page 11: Interpreting Run Charts and Shewhart Charts

Shift? Note: values on median don’t make or break a shift

Page 12: Interpreting Run Charts and Shewhart Charts

Shift?

Page 13: Interpreting Run Charts and Shewhart Charts

Interpretation?

• There is a signal of a non-random pattern

• There is less than 5 % chance that we would see this pattern if something wasn’t going on, i.e. if there wasn’t a real change

Page 14: Interpreting Run Charts and Shewhart Charts

Plain Language Interpretation?

There is evidence of improvement – the chance we would see a “shift” like this in data if there wasn’t a real change in what we were doing is less than 5%.

Page 15: Interpreting Run Charts and Shewhart Charts

Two few or too many runs?1. bring out the table2. how many points do we have (not on median?)3. how many runs do we have (cross median +1)4. what is the upper and lower limit?

Page 16: Interpreting Run Charts and Shewhart Charts

Two few or too many runs?1. bring out the table2. how many points do we have 203. how many runs do we have (cross median +1) 34. what is the upper and lower limit? 6 - 16

Page 17: Interpreting Run Charts and Shewhart Charts

Two few runs? Plain language interpretation

There is evidence of improvement – our data only crosses the median line twice – three runs. If it was just random variation, we would expect to see more up and down.

Page 18: Interpreting Run Charts and Shewhart Charts

There is evidence of a non-random pattern. There is a pattern to the way the data rises and falls above and below the median. Something systematically different. Should investigate and maybe plot on separate run charts.

Two many runs? Plain language interpretation

Page 19: Interpreting Run Charts and Shewhart Charts

Astronomical Data Point?

Page 20: Interpreting Run Charts and Shewhart Charts

Understanding Variation

Walter Shewhart

(1891 – 1967)W. Edwards Deming

(1900 - 1993)

The Pioneers of Understanding Variation

Page 21: Interpreting Run Charts and Shewhart Charts

Intended and Unintended Variation

• Intended variation is an important part of effective, patient-centered health care.

• Unintended variation is due to changes introduced into healthcare process that are not purposeful, planned or guided.

• Walter Shewhart focused his work on this unintended variation. He found that reducing unintended variation in a process usually resulted in improved outcomes and lower costs. (Berwick 1991)

Health Care Data Guide, p. 107

Page 22: Interpreting Run Charts and Shewhart Charts

Shewhart’s Theory of Variation

Common Causes—those causes inherent in the system over time, affect everyone working in the system, and affect all outcomes of the system

– Common cause of variation– Chance cause– Stable process– Process in statistical control

Special Causes—those causes not part of the system all the time or do not affect everyone, but arise because of specific circumstances

– Special cause of variation– Assignable cause– Unstable process– Process not in statistical control

Health Care Data Guide, p. 108

Page 23: Interpreting Run Charts and Shewhart Charts

Shewhart Charts

The Shewhart chart is a statistical tool used to distinguish between variation in a measure due to common causes and variation due to special causes

(Most common name is a control chart, more descriptive would be learning charts or system performance charts)

Health Care Data Guide, p. 113

Page 24: Interpreting Run Charts and Shewhart Charts

Control Charts – what features differ from a run chart?

Page 25: Interpreting Run Charts and Shewhart Charts

Control Charts/Shewhart Charts

upper and lower control limits

•to detect special cause variation

Extend limits to predict future performance

Not necessarily ordered by time•advanced application of SPC – is there something different between systems

Revised Limits After Improvement

40

50

60

70

80

90

100

M-04

M J S N J -05

M M J S N J -06

M M J S N J -07

M M

Aver

age

Days

CL = 88.16

UL = 97.86

LL = 78.47

CL = 62.1

UL = 71.8

LL = 88.2

Page 26: Interpreting Run Charts and Shewhart Charts

Example of Shewhart Chart for Unequal Subgroup Size

Health Care Data Guide, p. 114

Page 27: Interpreting Run Charts and Shewhart Charts

Adapted from Health Care Data Guide, p. 151 & QI Charts Software

Page 28: Interpreting Run Charts and Shewhart Charts

%Percent Trauma Patients D/C to Home

M626.7

231

A658.0

241

M597.0

220

J600.0

227

J570.0

260

A651.0

233

S588.0

238

O628.0

250

N626.0

270

D645.0

240

J594.0

227

F600.0

228

M723.0

264

A658.0

278

M598.0

261

J543.0

208

J627.0

268

A658.0

293

S582.0

264

MonthTrauma Volume

# D/C to Homep chart

M A M J J A S O N D J F M A M J J A S25

30

35

40

45

50

55

UCL = 45.83

Mean = 39.93

LCL = 34.03

Page 29: Interpreting Run Charts and Shewhart Charts

Note: A point exactly on the centerline does not cancel or count towards a shift

Health Care Data Guide, p. 116

Page 30: Interpreting Run Charts and Shewhart Charts

Rat

e pe

r 100

ED

Pat

ient

sUnplanned Returns to Ed w/in 72 Hours

M41.78

17

A43.89

26

M39.86

13

J40.03

16

J38.01

24

A43.43

27

S39.21

19

O41.90

14

N41.78

33

D43.00

20

J39.66

17

F40.03

22

M48.21

29

A43.89

17

M39.86

36

J36.21

19

J41.78

22

A43.89

24

S31.45

22

MonthED/100

Returnsu chart

0.0

0.2

0.4

0.6

0.8

1.0

1.2

UCL = 0.88

Mean = 0.54

LCL = 0.19

Page 31: Interpreting Run Charts and Shewhart Charts

Rat

e pe

r 100

ED

Pat

ient

sUnplanned Returns to Ed w/in 72 Hours

M41.78

17

A43.89

26

M39.86

13

J40.03

16

J38.01

24

A43.43

27

S39.21

19

O41.90

14

N41.78

33

D43.00

20

J39.66

17

F40.03

22

M48.21

29

A43.89

17

M39.86

36

J36.21

19

J41.78

22

A43.89

24

S31.45

22

MonthED/100

Returnsu chart

0.0

0.2

0.4

0.6

0.8

1.0

1.2

UCL = 0.88

Mean = 0.54

LCL = 0.19

Special cause: point outside the limits

Page 32: Interpreting Run Charts and Shewhart Charts

%Percent Trauma Patients D/C to Home

M626.7

231

A658.0

241

M597.0

220

J600.0

227

J570.0

260

A651.0

233

S588.0

238

O628.0

250

N626.0

270

D645.0

240

J594.0

227

F600.0

228

M723.0

264

A658.0

278

M598.0

261

J543.0

208

J627.0

268

A658.0

293

S582.0

264

MonthTrauma Volume

# D/C to Homep chart

M A M J J A S O N D J F M A M J J A S25

30

35

40

45

50

55

UCL = 45.83

Mean = 39.93

LCL = 34.03

Page 33: Interpreting Run Charts and Shewhart Charts

%Percent Trauma Patients D/C to Home

M626.7

231

A658.0

241

M597.0

220

J600.0

227

J570.0

260

A651.0

233

S588.0

238

O628.0

250

N626.0

270

D645.0

240

J594.0

227

F600.0

228

M723.0

264

A658.0

278

M598.0

261

J543.0

208

J627.0

268

A658.0

293

S582.0

264

MonthTrauma Volume

# D/C to Homep chart

M A M J J A S O N D J F M A M J J A S25

30

35

40

45

50

55

UCL = 45.83

Mean = 39.93

LCL = 34.03

Special cause2 out of 3 consecutive points in outer third of limits or beyond

Page 34: Interpreting Run Charts and Shewhart Charts

# o

f Need

lest

icks

Employee Needlesticksc c ha r t

UCL = 12.60

Mean = 5.54

New Needles Test

1-05 3-05 5-05 7-05 9-05 11-05 1-06 3-06 5-06 7-06 9-06 11-06 1-07 2-070

5

10

15

20

Page 35: Interpreting Run Charts and Shewhart Charts

# o

f Need

lest

icks

Employee Needlesticksc c ha r t

UCL = 12.60

Mean = 5.54

New Needles Test

1-05 3-05 5-05 7-05 9-05 11-05 1-06 3-06 5-06 7-06 9-06 11-06 1-07 2-070

5

10

15

20

Page 36: Interpreting Run Charts and Shewhart Charts

Con

tam

inat

ions

/100

0Blood Culture Contaminations Org 1: last 2 years

u chart

20

25

30

35

40

45UCL

Mean

LCL

Page 37: Interpreting Run Charts and Shewhart Charts

Con

tam

inat

ions

/100

0Blood Culture Contaminations Org 1: last 2 years

u chart

20

25

30

35

40

45UCL

Mean

LCL

Common Cause

Page 38: Interpreting Run Charts and Shewhart Charts

Note: A point exactly on the centerline does not cancel or count towards a shift

Health Care Data Guide, p. 116

Page 39: Interpreting Run Charts and Shewhart Charts

Case Study #1a

Page 40: Interpreting Run Charts and Shewhart Charts

Case Study #1b

Percent of cases with urinary tract infection

Page 41: Interpreting Run Charts and Shewhart Charts

Case Study #1c

Percent of cases with urinary tract infection Percent of cases with urinary tract infection

Page 42: Interpreting Run Charts and Shewhart Charts

Case Study #1d

Percent of cases with urinary tract infection

Page 43: Interpreting Run Charts and Shewhart Charts

Case Study #1e

Percent of cases with urinary tract infection

Page 44: Interpreting Run Charts and Shewhart Charts

Case Study #1f

Percent of cases with urinary tract infection

Page 45: Interpreting Run Charts and Shewhart Charts

Note: A point exactly on the centerline does not cancel or count towards a shift

Health Care Data Guide, p. 116

Page 46: Interpreting Run Charts and Shewhart Charts

Case Study #2a

Percent of patients with Death or Serious Morbidity who are >= 65 years of age

Page 47: Interpreting Run Charts and Shewhart Charts

Case Study #2b

Percent of patients with Death or Serious Morbidity who are >= 65 years of age

Page 48: Interpreting Run Charts and Shewhart Charts

Case Study #2c

Percent of patients with Death or Serious Morbidity who are >= 65 years of age

Page 49: Interpreting Run Charts and Shewhart Charts

Case Study #2d

Percent of patients with Death or Serious Morbidity who are >= 65 years of age

Page 50: Interpreting Run Charts and Shewhart Charts

References

BCPSQC Measurement Report http://www.bcpsqc.ca/pdf/MeasurementStrategies.pdf

Langley GJ, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP (2009) The Improvement Guide (2nd ed).

Provost L, Murray S (2011) The Health Care Data Guide.

Berwick, Donald M, Controlling Variation in Health Care: A Consultation with Walter Shewhart, Medical Care, December, 1991, Vol. 29, No 12, page 1212-1225.

Perla R, Provost L, Murray S (2010) The run chart: a simple analytical tool for learning from variation in healthcare processes, BMJ Qual Saf 2011 20: 46-51.

Associates in Process Improvement website www.apiweb.org