making better decisions making better decisions – whaip info for action course 4 november 2010...
TRANSCRIPT
Making Better Decisions
Making Better Decisions – WHAIP Info for Action course
4 November 2010
Presenter: Mike Davidge
Making Better Decisions
Why do we analyse and present data?
• To understand the variation that lives within our data
• To help us make good management decisions in the way we react to that variation
Source: Robert Lloyd IHI 2006
Making Better Decisions
But often we have presented data in a different way ...
Making Better Decisions
The change in Merthyr is remarkable
Making Better Decisions
Eco-friendly
% Delayed transfers of Care by Type - source SITREPS 7/1/02-31/08/03
-20.00
0.00
20.00
40.00
60.00
80.00
100.00
07/0
1/0
2
07/0
2/0
2
07/0
3/0
2
07/0
4/0
2
07/0
5/0
2
07/0
6/0
2
07/0
7/0
2
07/0
8/0
2
07/0
9/0
2
07/1
0/0
2
07/1
1/0
2
07/1
2/0
2
07/0
1/0
3
07/0
2/0
3
07/0
3/0
3
07/0
4/0
3
07/0
5/0
3
07/0
6/0
3
07/0
7/0
3
07/0
8/0
3
%Aw aitAss<7 days
%Aw aitAss >7days
%Aw ait Public Funding
%Aw ait Further NHS care
%Aw ait Residential
%Aw ait Domiciliary package
%Patient Family choice
%Other Reasons
Making Better Decisions
Nomination for the Turner Prize% Delayed transfers of Care by Type - source SITREPS 7/1/02-31/08/03
-20.00
0.00
20.00
40.00
60.00
80.00
100.00
07
/01
/02
07
/02
/02
07
/03
/02
07
/04
/02
07
/05
/02
07
/06
/02
07
/07
/02
07
/08
/02
07
/09
/02
07
/10
/02
07
/11
/02
07
/12
/02
07
/01
/03
07
/02
/03
07
/03
/03
07
/04
/03
07
/05
/03
07
/06
/03
07
/07
/03
07
/08
/03
%Other Reasons
%Patient Family choice
%Await Domiciliary package
%Await Residential
%Await Further NHS care
%Await Public Funding
%AwaitAss >7days
%AwaitAss<7 days
Making Better Decisions
Every picture tells a story … Does it?
Reasons for Delayed Transfer August 2003
%AwaitAss<7 days9%
%AwaitAss >7days9%
%Await Public Funding5%
%Await Further NHS care12%
%Await Residential25%
%Await Domiciliary package
12%
%Patient Family choice18%
%Other Reasons10%
Looks pretty – but cannot show
change over time
Making Better Decisions
Why do our processes vary?
Mrs Jones runs a B&B establishment in a seaside town. She makes breakfast every morning for her guests before going off to do a cleaning job in the local factory.Recently she has been late and her supervisor has given her a verbal warning. What can she do to avoid being late in future?Her son has been measuring her finish times as part of a GCSE maths project. Let’s look at the results
Making Better Decisions
Breakfast finishing times
07:12
08:24
09:36
10:48
12:00
13:12
Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 Day 8 Day 9 Day10
Day11
Day12
Day13
Day14
Day15
Day16
Day17
Day18
Day19
Day20
Day21
Day22
Day23
Day24
Why does her finish
time vary?
Making Better Decisions
Walter A. Shewhart(early 1920’s, Bell Laboratories)
• While every process displays variation:
• some processes display controlled variation (common cause)stable,consistent pattern of variationconstant causes/ “chance”
• while others display uncontrolled variationpattern changes over timespecial cause variation/“assignable” cause
Making Better Decisions
Shewhart’s purpose
• Data contains both signal and noise. To be able to extract information, one must separate the signal from the noise within the data.
Making Better Decisions
01020304050607080
F M A M J J A S O N D J F M A M J J A S O N D
Upper process
limit
Mean
Lower process
limit
A typical process control chart
Making Better Decisions
The variation in Mrs Smith’s process• Common Cause = Consistent over time & part of the
process
• Special Cause = Irregular in time & indicates a changed process
• Group the causes into common & special
• Are there any that cannot be grouped?
Making Better Decisions
So why run or control charts?
• One picture, one message
• Shows change over time
• Allows you to see variation
MRSA performanceCity Hospital
0
2
4
6
8
10
12
14
16
18
20
17/0
1/05
17/0
1/05
21/0
1/05
21/0
1/05
01/0
2/05
04/0
2/05
10/0
2/05
20/0
2/05
24/0
2/05
03/0
3/05
07/0
3/05
07/0
3/05
07/0
3/05
08/0
3/05
16/0
3/05
03/0
4/05
04/0
4/05
09/0
4/05
12/0
4/05
13/0
4/05
17/0
4/05
22/0
4/05
28/0
4/05
03/0
5/05
06/0
5/05
Date of event
Gap
bet
wee
n s
ucc
essi
ve e
ven
ts
Gap betw een specimens Average LCL UCL
Hospital X
Making Better Decisions
Example #1: 2 data pointsMRSA at Hospital X
Are we getting better?
Measuring rare events
Making Better Decisions
Have things improved during 2005?
MRSA specimens at Hospital X
0
2
4
6
8
1012
14
16
18
20
Q1 Q2 Q3 Q4
2005
Sp
ecim
en
s p
er
mo
nth
Hospital collecting the date of specimen (event) for each MRSA specimen from January 2005 to February 2006
Improvement this
quarter?
Making Better Decisions
We have 2 quarterly data points - is this an improvement?
Executive Time Series
0
20
40
60
80
100
J F M A M J J A S O N D
Months
So
me
thin
g I
mp
ort
an
t
Higher is better
Making Better Decisions
Are we assuming something like this?
Executive Time Series - linear trend
0
20
40
60
80
100
J F M A M J J A S O N D
Months
So
meth
ing
Im
po
rtan
t
Making Better Decisions
But it could be like this ...
Executive Time Series - no trend
0
20
40
60
80
100
J F M A M J J A S O N D
Months
So
meth
ing
Im
po
rtan
t
Making Better Decisions
Or this ...
Executive Time Series - seasonal dip
0
20
40
60
80
100
J F M A M J J A S O N D
Months
So
me
thin
g I
mp
ort
an
t
Making Better Decisions
Or this!
Executive Time Series - one month blip
0
20
40
60
80
100
J F M A M J J A S O N D
Months
So
me
thin
g I
mp
ort
an
t
Making Better Decisions
How have I analysed the data?
• I have calculated the gap between successive specimen events in daysFor example if the first event was on 1 Jan 2005 and the
second was on 17 Jan 2005 then the gap is 16 days
• I have plotted the gaps on a run chart to see whether there is evidence of a change over time
• If MRSA is becoming less prevalent then the gaps between occurrences will get larger
• If MRSA is becoming more prevalent then the gaps between occurrences will get smaller
Making Better Decisions
MRSA performanceCity Hospital
0
2
4
6
8
10
12
14
16
18
20
17/0
1/0
5
17/0
1/0
5
21/0
1/0
5
21/0
1/0
5
01/0
2/0
5
04/0
2/0
5
10/0
2/0
5
20/0
2/0
5
24/0
2/0
5
03/0
3/0
5
07/0
3/0
5
07/0
3/0
5
07/0
3/0
5
08/0
3/0
5
16/0
3/0
5
03/0
4/0
5
04/0
4/0
5
09/0
4/0
5
12/0
4/0
5
13/0
4/0
5
17/0
4/0
5
22/0
4/0
5
28/0
4/0
5
03/0
5/0
5
06/0
5/0
5
Date of event
Gap
betw
een
su
ccessiv
e e
ven
ts
Gap betw een specimens Average LCL UCL
Hospital X
Setting the baselineUsing the first 25 gaps
Average is 5 days
But could have gaps of upto 18
days
Data plotted shows that
performance is predictable
within the limits
Making Better Decisions
Performance over the year
MRSA performanceCity Hospital
0
5
10
15
20
25
30
35
4017/0
1/0
5
04/0
2/0
5
07/0
3/0
5
03/0
4/0
5
17/0
4/0
5
08/0
5/0
5
18/0
6/0
5
27/0
7/0
5
12/0
9/0
5
23/1
1/0
5
26/0
1/0
6
05/0
2/0
6
Date of event
Ga
p b
etw
ee
n s
uc
ce
ss
ive
ev
en
ts
Gap betw een specimens Average LCL UCL
Hospital X
Baseline period
Making Better Decisions
MRSA performanceCity Hospital
0
5
10
15
20
25
30
35
4017/0
1/0
5
04/0
2/0
5
07/0
3/0
5
03/0
4/0
5
17/0
4/0
5
08/0
5/0
5
18/0
6/0
5
27/0
7/0
5
12/0
9/0
5
23/1
1/0
5
26/0
1/0
6
05/0
2/0
6
Date of event
Ga
p b
etw
ee
n s
uc
ce
ss
ive
ev
en
ts
Gap betw een specimens Average LCL UCL
Hospital X
Performance over the year
A more unstable
period with a large gap typically
followed by a short one
Making Better Decisions
MRSA performanceCity Hospital
0
5
10
15
20
25
30
35
4017/0
1/0
5
04/0
2/0
5
07/0
3/0
5
03/0
4/0
5
17/0
4/0
5
08/0
5/0
5
18/0
6/0
5
27/0
7/0
5
12/0
9/0
5
23/1
1/0
5
26/0
1/0
6
05/0
2/0
6
Date of event
Ga
p b
etw
ee
n s
uc
ce
ss
ive
ev
en
ts
Gap betw een specimens Average LCL UCL
Hospital X
Performance over the year
In January 2006, a series of events one
after the other suggesting a worsening in performance
Making Better Decisions
Questions
• What caused the unstable period in the second half of 2005?Was it simply that having found 1 event people were more
active in spotting another?Did one event cause the next?Was it a data collection issue?
• Did things really get worse in 2006?We don’t have quite enough points on the chart to be certain
but the question is worth asking as the pattern of events does seem to have changed
Making Better Decisions
The MBFC Index
You will need a coin
And two free hands!
Just how competitive can you get?
Making Better Decisions
Sustainable processes
• You need about 6 data points to confirm a change
• You need to continue measuring to assure yourself that the change is sticking
• “In God we trust, all others bring data” W Edwards Deming
Percent using SBAR
Making Better Decisions
XX
X
X
X
X
X
X
X
LCL
UCL
MEAN
X
X
X
X
XX
X
X
X
X
LCL
UCL
MEAN
X
Point above UCL
Point below LCL
SPECIAL CAUSES - RULE 1
Making Better Decisions
MEAN MEAN
Seven points above centre line
SPECIAL CAUSES - RULE 2
LCL
UCL
LCL
UCL
XX
X
X
X X
X
XX
XX X
X
XX
X X
X
XX
X
Seven points below centre line
Making Better Decisions
MEAN MEAN
Seven points in a downward direction
SPECIAL CAUSES - RULE 2
LCL
UCL
LCL
UCL
XX
XX
X
XX
X
X X
X
XX X
XX
XX
X
X
X
Seven points in an upward direction
Making Better Decisions
SPECIAL CAUSES - RULE 3
X
X
X
X
X
X
XX X
X
X
X
X
X
X
X
X
X
X
X
Cyclic pattern
X
X X
XX
XX
X
X
X
X
X
X
X
X X
X
X
XLCL
UCL
LCL
UCL
Trend pattern
Making Better Decisions
SPECIAL CAUSES - RULE 4Considerably less than 2/3 of all the points fall in this zone
X
XX X X
X
X
X
X
X
X X
XX
X
XX
LCL
UCL
X
X
X
X
X
X
XX
X
X
X
XX
X
XX
X
X
XX
X X
X
X
XX
LCL
UCL
Considerably more than 2/3 of all the points fall in this zone
Making Better Decisions
Rules for special causes
• Rule 1
• Rule 2
• Rule 3
• Rule 4
Any point outside one of the control limits
A run of seven points all above or all below the centre line, or all increasing or all decreasing
Any unusual pattern or trends within the control limits
The number of points within the middle third of the region between the control limits differs markedly from two-thirds of the total number of points
Making Better Decisions
PROCESS 1
X
X
X
X X
XX
X X
X X
X X
X
X
X
X
X
X
X
X
PREDICTABLE? YES / NO / NOT SURE RULE 1 2 3 4
Making Better Decisions
PROCESS 2
X X X
X
XX
X
X
X
X
X X X
X
X X X
X
X
X
X
PREDICTABLE? YES / NO / NOT SURE RULE 1 2 3 4
Making Better Decisions
PROCESS 3
X X
X
X
X
XX
X
X
X
X X
X
X
X X
X X X X
X
PREDICTABLE? YES / NO / NOT SURE RULE 1 2 3 4
Making Better Decisions
X X X
X X
XX X X X X
X
X X X
X
X X X
X X
PREDICTABLE? YES / NO / NOT SURE RULE 1 2 3 4
PROCESS 4
Making Better Decisions
PROCESS 5
X
X
X
X
XX
X
X
X
X
X
X
X
X X
X
X
X
X X
X
PREDICTABLE? YES / NO / NOT SURE RULE 1 2 3 4
Making Better Decisions
PROCESS 6
X
X X
X
X
XX
X
X X X
X
X
X
X X X
X
X X
X
PREDICTABLE? YES / NO / NOT SURE RULE 1 2 3 4
Making Better Decisions
SPC tools
• BP chartType BP Chart.xls into Google
• BaselineSee handout
• Winchart from Prism Europehttp://www.winchart.net/
• Chartrunner from PQ Systemshttp://www.pqsystems.com/products/SPC/CHARTrunner/
CHARTrunner.php
Making Better Decisions
Useful references
• Donald Wheeler. Understanding Variation. Knoxville: SPC Press Inc, 1995
• Donald Wheeler. Making sense of data. SPC for the service sector. Knoxville: SPC Press Inc, 2003
• WE Deming. Out of the crisis. Massachusetts: MIT 1986
• American Society for Quality www.asq.org/about/history/shewhart.html
• Donald M Berwick. Controlling variation in health care: a consultation from Walter Shewhart. Med Care 1991; 29: 1212-25.
• Walter A Shewhart. Economic control of quality of manufactured product. New York: D Van Nostrand 1931.
Making Better Decisions
Example #2: How often? Barnsley stroke pathway
time in hospital before getting onto stroke ward
Making Better Decisions
Monthly data shows improvement
Average length of pre-ward stay on Barnsley Stroke Ward
from 01/2007 to 07/2007
0
0.5
1
1.5
2
2.5
3
3.5
1 2 3 4 5 6 7
Months
Making Better Decisions
Weekly data tells a different story
Average length of pre-ward stay on Barnsley Stroke Ward
from 01/2007 to 07/2007
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
Weeks
Making Better Decisions
Example #3: Link to change SPI project
Medicines management
Making Better Decisions
% medicines reconciledUHW MAU
010
2030
405060
7080
90100
Apr
-07
May
-07
Jun-
07
Jul-0
7
Aug
-07
Sep
-07
Oct
-07
Nov
-07
Dec
-07
Jan-
08
Feb
-08
Mar
-08
Apr
-08
May
-08
% r
eco
nci
led
Form piloted
Form printed
Letter from CDs
Pharmacy included