l16: beyond statistical process control: advanced charts ... · l16: beyond statistical process...
Post on 03-Jul-2020
12 Views
Preview:
TRANSCRIPT
11/18/2009
1
Note: we are assuming excel skills…so buddy up withsomeone who has strong skills if yours are not as strong …thanks!
L16: Beyond Statistical Process Control: Advanced Charts for Healthcare
l d i i
API and CT Concepts, 2009
Lloyd P. Provost, Associates in Process Improvement
Sandra Murray, CT Concepts
L16: Beyond Statistical Process Control: Advanced Charts for Healthcare
This session will explore some of the more advanced uses for statistical process control (SPC) charts for health care data. Some issues to be discussed include how to tell if anything has improved when “yucky” events are already rare; why some control charts have such narrow limits with all of the data outside the limits; and how to factor in seasonalwith all of the data outside the limits; and how to factor in seasonal impacts on data. Our journey will include the use of T and G charts for rare events data, Prime charts for dealing with over dispersion in data, adjustments for autocorrelation, using changing center lines on control charts, and the use of the CUSUM control chart.
Session Objectives: • Select the appropriate SPC chart for rare events data
Id if h i i i CUSUM l h
API and CT Concepts, 2009
2
• Identify when it is most appropriate to use a CUSUM control chart
• Describe when to use a prime chart for over‐dispersion in data
• Identify autocorrelation and describe how to deal with it using a Shewhart control chart
11/18/2009
2
Introduction
Our Process– Will provide some context for Shewhart Charts
– Then address each 4 of objectives
– We will use case studies and Excel on your PC’s to give you practice building the charts
API and CT Concepts, 2009
3
SHEWHART CHART:What Is It?
• The Shewhart chart is a statistical tool used to distinguish between variation in a measure due to common causes and variation due to special causes. • Commonly called “control chart.” • A more descriptive name might be “learning charts” or “system performance charts”
• Format:•Data is usually displayed over time•Data usually displayed in time order
API and CT Concepts, 2009
•Shewhart chart will include:•Center line (usually mean)•Data points•Statistically calculated upper and lower limits
11/18/2009
3
What Does a Shewhart Chart Look Like? Coding Errors per 20 Records12 10 14 12 19 8 15 10 8 6 21 10 5 9 11 6 7 10 13 11 9 3 7 6 10Defects
c‐chart Set 1: UCL=19.60, Mean=10.08, LCL=0.56 (n=1)
25
30
Straight limits indicate equal subgroup size
# Co
ding
Errors
10
15
20 UCL
Mean
API and CT Concepts, 2009Sequential Subgroups of 20 Records
#
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
0
5
LCL
% of C-Sections Performed Due to Fetal DistressJan4015
Feb8015
Mar7020
Apr6520
May8525
Jun7015
Jul6520
Aug5510
Sep10011
Oct5515
Nov7010
Dec8012
Jan350
Months# C-SecFetal Ind
p chart
40
45
50
Varying limits indicate unequal subgroup size
20
25
30
35
UCL = 36.70
Mean = 21.61
API and CT Concepts, 2009Jan
Feb Mar AprMay Ju
n Jul
Aug Sep OctNov Dec Ja
n0
5
10
15
LCL = 6.52
11/18/2009
4
What do we use a Shewhart chart for?
• Learn how much variation exists in processA t bilit d d t i i t• Assess stability and determine improvement strategy (common or special cause strategy)
• Monitor performance and correct as needed • Find and evaluate causes of variation• Tell if our changes yielded improvementsS if i t “ ti ki ”
API and CT Concepts, 2009
• See if improvements are “sticking”
Shewhart’s Theory of Variation
• Common Cause: causes that are inherent in the process, over time affect everyone working in the process, and affect all outcomes of the process– Process stable predictableProcess stable, predictable– Action: if in need of improvement must redesign
• Special cause: causes that are not part of the process all the time, or do not affect everyone, but arise because of special circumstances
API and CT Concepts, 2009
– Process unstable, not predictable– May be evidence of improvement (change we tested working)
– Action: go investigate special cause and take appropriate action
11/18/2009
5
Standard Rules for Detecting Special Cause
API and CT Concepts, 2009
Unplanned Returns to Ed w/in 72 HoursM
41.7817
A43.89
26
M39.86
13
J40.03
16
J38.01
24
A43.43
27
S39.21
19
O41.90
14
N41.78
33
D43.00
20
J39.66
17
F40.03
22
M48.21
29
A43.89
17
M39.86
36
J36.21
19
J41.78
22
A43.89
24
S31.45
22
MonthED/100Returns
u chart
1.0
1.2
How Do We Detect Special Cause?
Rat
e pe
r 100
ED
Pat
ient
s
0.6
0.8
UCL = 0.88
Mean = 0.54
API and CT Concepts, 20091 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 190.0
0.2
0.4
LCL = 0.19
11/18/2009
6
ACEI for LVSD: All Data Within Limits But Still Special Causep char t
90
100
110
UCL = 101.98
Per
cent
Com
plia
nce
70
80
90Mean = 88.55
LCL = 75.12
API and CT Concepts, 2009
J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M
50
60
Type of Data
Count or Classification (Attribute Data) Qualitative data in terms of an integer (# errors #
Continuous (Variable Data) Quantitative data in the form of a measurement
Shewhart Chart Selection Guide
-
Qualitative data in terms of an integer (# errors, # nonconformities or # of items that passed or failed) Discrete: must be whole number when originally collected (can’t be fraction or scaled data when originally collected)This data is counted, not measured
Count (nonconformities)1,2,3,4, etc.
Classification (nonconforming)either/or, pass/fail, yes/no
Equal area of Unequal area Unequal or equal
Quantitative data in the form of a measurementTime, money, scaled data i.e. length, height, weight, temperature, mg. and throughput (volume ofworkload/ productivity)
Subgroup sizeUnequal or equal subgroup
Each subgroup is composed of a single data value
Each subgroup has more than one data value
API and CT Concepts, 2009
Equal area ofopportunity
Unequal area of opportunity
Unequal or equal subgroup size
Subgroup size of 1 (n=1)
equal subgroup size (n>1)
C Chart U Chart P ChartI Chart (also known as X Chart) X Bar and S
Number of nonconformities
Nonconformities per unit
Percent nonconforming
Individual measurement Average and standard deviation
11/18/2009
7
T and G Charts for Rare Events
• The problem and examples
• Intro to template
• Work Case Study – (10)
• Debrief ‐ Tips, issues, references
API and CT Concepts, 200913
The Problem: How do we tell if we are improving when undesirable events are rare anyway?
Month # Med Errors # Doses #Doses/1000 Notes
N 06 3 24222 24.222D 1 23616 23 616A D 1 23616 23.616J 07 4 23072 23.072F 2 19439 19.439M 2 24568 24.568A 3 21020 21.02 Chg 1 test and Imp, Ch 2 testM 2 28754 28.754 Chg 2 TestJ 2 23390 23.39 Chg 2 ImpJ 1 28475 28 475 Chg 3 Lg Test
AverageBefore=3
Average
API and CT Concepts, 2009
J 1 28475 28.475 Chg 3 Lg TestA 2 22079 22.079 Chg 3 ImpS 0 29206 29.206O 1 23390 23.39N 0 28475 28.475D 1 23390 23.39
Average After=1.3
11/18/2009
8
The Problem: when undesirable events are rare even a standard run or Shewhart control chart may not be very helpful
Harmful Medication Error Rate per 1000 Doses Dispensedu chart
nd Im
p, C
h 2
test
Chg
2 T
est
Chg
2 Im
p
Chg
3 L
g Te
st
Chg
3 Im
p0.4
rror R
ate/
1000
Dos
es
UCL = 0.23
Chg
1 te
st a
n0.2
0.3
API and CT Concepts, 2009
Med
E
Mean = 0.07
N 06 D J 07 F M A M J J A S O N D
0.1
0.0
How Do You Know You Need to View Data Differently?
• Have too many zeros (>25% of data points 0)
• Have no lower limitPercent C‐Sections
p‐chart (%) Mean=15.84100
Percen
t
40
50
60
70
80
90
UCL
API and CT Concepts, 2009Sequential Months
P
Mo
95/1
95/2
95/3
95/4
95/5
95/6
95/7
95/8
95/9
95/10
95/11
95/12
96/1
96/2
96/3
96/4
96/5
96/6
96/7
96/8
96/9
96/10
96/11
96/12
97/1
97/2
97/3
97/4
97/5
97/6
97/7
97/8
97/9
97/10
97/11
97/12
98/1
98/2
98/3
98/4
98/5
98/6
98/7
98/8
98/9
98/10
98/11
98/12
99/1
99/2
99/3
99/4
99/5
99/6
99/7
99/8
99/9
99/10
99/11
99/12
00/1
00/2
00/3
00/4
00/5
00/6
00/7
00/8
00/9
0
10
20
30
Mean
11/18/2009
9
When Will This Happen?Classification Data: Guidelines for Selecting Subgroup Size for an Effective P chart
Average PercentNonconforming
Units (pbar)
Minimum Subgroup Size (n) Required to Have
< 25% zero for p'sMinimum Subgroup Size
Guideline(n>300/pbar)
Minimum Subgroup Size Required to Have
LCL > 00.1 1400 3000 90000.5 280 600 18001.0 140 300 9001.5 93 200 6002 70 150 4503 47 100 3004 35 75 2205 28 60 1756 24 50 1308 17 38 104
10 14 30 8112 12 25 66
API and CT Concepts, 2009
12 12 25 6615 9 20 5120 7 15 3625 5 12 2830 4 10 2240 3 8 1450 2 6 10
Note: for p>50, use 100‐p to enter the table (e.g. for p=70% use table p of 30%, for p=99% use table p of 1%, etc.) Source: The Data Guide: L Provost and S. Murray, 2009
When Will This Happen?For Count Data
(U chart: errors or occurrences per unit: rate data)
• If center line less than 9 will be no lower limit• If center line less than 9 will be no lower limit
• If center line less than 1.4 will have too many 0’s
• What Can We Do?
API and CT Concepts, 2009
What Can We Do?– Increase subgroup size
11/18/2009
10
One Way to Transform Data: Limits: Appropriate Vs. Too Small Subgroup SizeAggregate Level: Total C‐Section Rate By Quarter
80
90
100
p‐chart (%)
% C‐Sections By Month % C‐Sections Quarter
80
90
100p‐chart (%)
60
70
%
30
40
50
UCL
C‐Section Ra
te%
30
40
50
60
70
UCL
%
API and CT Concepts, 2009 Sequential Quarters
95/1
95/2
95/3
95/4
96/1
96/2
96/3
96/4
97/1
97/2
97/3
97/4
98/1
98/2
98/3
98/4
99/1
99/2
99/3
99/4
00/1
00/2
0
10
20Mean
30
Sequential Months
Mo
95/1
95/2
95/3
95/4
95/5
95/6
95/7
95/8
95/9
95/10
95/11
95/12
96/1
96/2
96/3
96/4
96/5
96/6
96/7
96/8
96/9
96/10
96/11
96/12
97/1
97/2
97/3
97/4
97/5
97/6
97/7
97/8
97/9
97/10
97/11
97/12
98/1
98/2
98/3
98/4
98/5
98/6
98/7
98/8
98/9
98/10
98/11
98/12
99/1
99/2
99/3
99/4
99/5
99/6
99/7
99/8
99/9
99/10
99/11
99/12
00/1
00/2
00/3
00/4
00/5
00/6
00/7
00/8
00/9
0
10
20Mean
What Can We Do?
• Increase subgroup size
L k t ti t ( ti t i it• Look at time or count (cases, patients, visits, etc.) between an event
API and CT Concepts, 2009
11/18/2009
11
Alternative: Plot Time or Count Between Occurrences of Rare Events
Instead of plotting the number of incidences
Number of Incidences Per Month
1
2
mbe
r
each month, plot the time (or number of cases, patients, visits, etc) between incidences. Cases Between an Event
500
0
1
Jan-
01
Mar
-01
May
-01
Jul-0
1
Sep
-01
Nov
-01
Jan-
02
Mar
-02
May
-02
Jul-0
2
Sep
-02
Nov
-02
Jan-
03
Mar
-03
Num
API and CT Concepts, 2009
Plot a point each time an incidence occurs
0
100
200
300
400
Feb-01
Apr-01
Jun-01
Aug-01
Oct-01
Dec-01
Feb-02
Apr-02
Jun-02
Aug-02
Oct-02
Dec-02
Feb-03
# C
ases
Use Special Shewhart Charts for Rare EventsType of Data
Count or Classification (Attribute Data)‐Qualitative data such as # errors, # nonconformities or # of items that passed or failed) ‐ Discrete: must be whole number when originally collected (can’t be fraction or scaled data when originally collected)‐This data is counted, not measured
Continuous (Variable Data) ‐Quantitative data in the form of a measurement‐Requires a measurement scale ‐Time, Money, Scaled Data (i.e. length, height, weight, temperature, mg.) and Throughput (volume of workload/ productivity)
Count (Nonconformities)1,2,3,4, etc.
Classification (Nonconforming)Either/Or, Pass/Fail, Yes/No
Equal Area of Opportunity
Unequal Area of Opportunity
Unequal or Equal Subgroup Size
SubgroupSize of 1(n=1)
Unequal Or Equal SubgroupSize (n>1)
C Chart U Chart P ChartI Chart (also known as an X chart)
X‐Bar and S
API and CT Concepts, 2009
P Chart an X chart)
Number ofNonconformities
NonconformitiesPer Unit Percent
NonconformingIndividual Measures Average and Standard
Deviation
Other types of control charts for count/classification data:1. NP (for classification data)2. T-chart [time (or event, items, etc.) between rare events]3. Cumulative sum (CUSUM)4. Exponentially weighted moving average (EWMA)5. Geometric distribution chart (G chart for count data) 6. Standardized control chart
Other types of control charts for continuous data:7. X‐bar and Range8. Moving average9. Median and range10. Cumulative sum (CUSUM)11. Exponentially weighted moving average (EWMA)12. Standardized control chart
11/18/2009
12
g‐Chart for Rare Occurrences• Alternative to p‐chart, c‐chart, or u‐chart for count or classification data
when the measure is the number of cases between the incident or non‐conformity of interest.
• This chart allows the evaluation of each occurrence to be evaluated rather than having to wait to the end of a time period before the data is plotted. The g chart is g p p galso particularly useful for verifying improvements (such as reduced SSIs) and for processes with low rates.
• The number of units (surgeries, insertions, admissions, etc) between an incidence can often be modeled by the geometric distribution.
Calculation of control limits:g = number units between incidencesg‐bar= average of g’s
API and CT Concepts, 2009
g bar average of g sUL = ĝ + 3[ĝ * (ĝ +1)]1/2
LL = ĝ ‐+ 3[ĝ * (ĝ +1)]1/2 , (<0, so no LCL for this g chart)
CL = 0.693* g‐bar (median of g’s)
Notes: 1. The UL is approximately 4x the CL (for quick visual analysis)2. Use median for centerline to get data symmetric around CL for run rules
G chartU Chart - Infections per 100 ICU Admissions
10
15
20
25
30
35
Infe
ctio
n R
ate
UCL
CL = 7.2
Good
ComparisonOf U h t
Number of ICU Admissions Between Infections
30
35
40
45
50
ons
UCL = 39.7
0
5
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug
Good G Chart
Good
Of U chart and G chart for infections in the ICU
API and CT Concepts, 2009
0
5
10
15
20
25
30
Jan Feb Feb Mar Apr May Jun Aug Oct Nov Nov Jan Mar Jun
# of
adm
issi
o
CL = Median = 9.6
Mean = 13.8
11/18/2009
13
T‐Chart for Time Between Rare OccurrencesAlternative to p‐chart, c‐chart, or u‐chart for count or classification data when the
measure is the time (continuous variable) between the incident or non‐conformity of interest.
This chart allows the evaluation of each non‐conformity or non‐confirming unit to be evaluated rather than having to wait to the end of a time period before the data is plotted. If the rate of occurrence can be modeled by the Poisson distribution then the times between occurrences will becan be modeled by the Poisson distribution, then the times between occurrences will be exponentially distributed. The exponential distribution is highly skewed, so plotting the times would result in a control chart that is difficult to interpret. The exponential can be transformed to a symmetric Weibull distribution by raising the time measure to the 1/3.6 = 0.2777 power or
[y = t 0.2777]. Calculation of control limits:
t = time between incidences, y = transformed timeMR’ = average moving range of y’s
ŷ = average of y’s (center line)UL = ŷ + 2.66 * MR’
ŷ
API and CT Concepts, 2009
LL = ŷ ‐ 2.66 * MR’
Transform the limits back to time scale before plotting chart: t = y3.6
Notes: 1. Conduct the usual screening of MR’s to calculate the average moving range.2. Need to be careful about 0’s in the data (try to increase precision (hrs vs. days) when
near zero.3. LL < 0, there is no LL for this measure
Source: The Data Guide
T Chart for FallsChris McCarthy: Time Between FallsChris.Mccarthy@kp.org
12
Sam
ple
Coun
t
5
4
3
2
1
_C=1.385
UCL=4.915
C C ha r t o f # of F a l ls pe r M onth
Days Between
0
2
4
6
8
10
12
0 10 20 30 40 50 60 70 80 MoreBin
Freq
uenc
y = days ** 0.2777
6
Sa mple2421181512963
0
API and CT Concepts, 2009
0
1
2
3
4
5
6
1 1.4 1.8 2.2 2.6 3 More
Bin
Freq
uenc
11/18/2009
14
MRSA R
ate/
1000
Bed
Day
s
MRSA Rate per 1000 Occupied Bed Days.491
5.442
4.318
5.312
5.409
5.271
3.278
2.211
2.260
1.341
2.398
1.275
3.244
1#OBD/1000
# MRSAu c har t
UCL = 25.07
15
20
25
30
M Mean = 9.18
M/07 A M J J A S O N D J/08 F M A M
5
10
MRSA-Days Between4 1 8 7 10 10 3 12 7 1 10 6 9 7 6 3 7 9 3 4 11 5 2 8 6 13 15 16 17 23 20 15 21 19 18 17 10 19Days
t char t
UCL = 30 78
35
40
45
API and CT Concepts, 2009
Day
s
UCL 30.78
Mean = 5.57
LCL = 0.19
3/2/07 3/
63/7
3/15
3/22 4/
14/11
4/14
4/26 5/
35/3
5/13
5/19
5/28 6/
46/10
6/14
6/21
6/30 7/
37/7
7/18
7/23
7/25 8/
28/8
8/21 9/
59/21
10/8
10/31
11/20 12
/512
/26
1/14
/08 2/
12/18
2/28
3/19
0
5
10
15
20
25
30
Let’s Practice
• Lets look at Template and Excel Database
• Make a T or G chart– You will need to copy and paste the data into the appropriate template
• May make both charts if time allows
API and CT Concepts, 2009
11/18/2009
15
Cardiac Arrests per Monthc chart
4
5
What’s the problem with this chart??Good
# A
rrest
s
UCL = 3.12
2
3
API and CT Concepts, 2009
Mean = 0.67
J-02F M A M J J A S O N DJ-03F M A M J J A S O N DJ-04F M A M J J A S O N DJ-05F M A M J J A S O N DJ-06F M A M J J A S O N DJ-07F M A M J J A S OM DJ-08F M A M J J A0
1
ADEs per 1000 Doses Dispensedu chart
UCL = 0.22
0.25
0.30
No statistical signal of improvement
AD
E R
ate
0.10
0.15
0.20
API and CT Concepts, 2009
CTL = 0.06
N 06 D J 07 F M A M J J A S O N D J08 F M A M J J A S0.00
0.05
11/18/2009
16
Debrief• Both t chart and g chart can use all 5 rules for detecting special cause
– That is why g chart uses median as center line
• Resources:• The Data Guide. Provost and Murray, 2009. API‐Austin, Austin, TX, Chapter 7.
• Jackson, J. E. , “All Count Distributions are not Alike”, Journal of Quality Technology, Vol 4 (2), pp 86‐92, 1972.
• Yang, S., et al, “On the Performance of Geometric Charts with Estimated Control Li it ” J l f Q lit T h l V l 34 N 4 448 458 O t b 2002
API and CT Concepts, 2009
Limits” Journal of Quality Technology, Vol 34, No.4, pp 448‐458, October, 2002.
• Wall, R., et al, “Using real time process measurements to reduce catheter related bloodstream infections in the ICU, Qual. Saf. Health Care 2005;14;pp. 295‐302.
• Nelson, L., “A Control Chart for Parts‐Per‐Million Nonconforming Items”, Journal of Quality Technology, Vol 26, No. 3, pp. 239‐240, July, 1994.
Cumulative Sum Control Charts
• Problem (5)
• Examples, Intro to template (7)
• Work Case Study – (13)
• Debrief ‐ Tips, issues, references (5)
API and CT Concepts, 200932
11/18/2009
17
Some Advanced Control Charts
• Shewhart control charts plot information from only the last subgroup.
• The sensitivity of the chart can be improved by i ti i d t i h l tt d i t Thincorporating previous data in each plotted point. The moving average and moving range are two examples of this.
• Other effective alternatives to the Shewhart control charts are the cumulative sum (CUSUM) control chart and the exponentially weighted moving average(EWMA) control chart
API and CT Concepts, 2009
(EWMA) control chart. • Especially useful when it is important to detect small
shifts in the measure of interest.
Warning: With these Advanced Control Charts, Can Not Use Standard Rules 2‐5 for Determining a Special Cause
OK
API and CT Concepts, 2009
SPC-
11/18/2009
18
Data Used to Calculate Plotted Point (after 5 data points)
x1 x2 x3 x4 x5 x6 x7 x8 x9 x10
Run Chart or 0 0 0 0 1 0 0 0 0 0Run Chart or Control Chart
0 0 0 0 1 0 0 0 0 0
Cusum .2 .2 .2 .2 .2 0 0 0 0 0
Moving Average (3)
0 0 .33 .33 .33 0 0 0 0 0
Moving Average (5)
.2 .2 .2 .2 .2 0 0 0 0 0
API and CT Concepts, 2009
EWMA λ = 0.2 .09 .17 .24 .3 .2 0 0 0 0 0
EWMA λ = 0.1 .1 .2 .25 .35 .1 0 0 0 0 0
Data Used to Calculate Plotted Point (After 8 Data Points)
x1 x2 x3 x4 x5 x6 x7 X8 x9 x10
Run Chart or Control Chart
0 0 0 0 0 0 0 1 0 0
Cusum .125 .125 .125 .125 .125 .125 .125 .125 0 0
Moving Average (3)
0 0 0 0 0 .33 .33 .33 0 0
Moving Average (5)
0 0 .0 .2 .2 .2 .2 .2 0 0
API and CT Concepts, 2009
EWMA λ = 0.2 .05 .06 .09 .12 .14 .16 .18 .2 0 0
EWMA λ = 0.1 .06 .09 .11 .13 .15 .17 .19 .1 0 0
11/18/2009
19
CUSUM
The cumulative sum statistic (S) is the sum of the deviations of the individual measurements from a target value, for example:
Si = S i‐1 + (Xi ‐ T), Xi = the ith observation,T = Target (often from historical average)
API and CT Concepts, 2009
Si = the ith cumulative statistic.
CUSUM Calculation – Patient Satisfaction Data
Month %Sat(X) Target (Average) X-Target (Si) CUSUM CUSUM + Target
J-02 82 88.296 -6.296 -6.296 82
F 79 88.296 -9.296 -15.592 72.704
M 84 88.296 -4.296 -19.888 68.408
A 82 88.296 -6.296 -26.184 62.112
M 92 88.296 3.704 -22.48 65.816
J 80 88.296 -8.296 -30.776 57.52
J 94 88.296 5.704 -25.072 63.224
A 78 88.296 -10.296 -35.368 52.928
S 83 88.296 -5.296 -40.664 47.632
O 84 88.296 -4.296 -44.96 43.336
API and CT Concepts, 2009
O 84 88.296 4.296 44.96 43.336
N 92 88.296 3.704 -41.256 47.04
D-02 84 88.296 -4.296 -45.552 42.744
03, 04
M 04 95 88.296 6.204 0.008 88.304
11/18/2009
20
Run Chart of Patient
Satisfaction Data
% Patient Satisfaction Very Good/Ex.
80
85
90
95
100
Perc
en
CUSUM Chart
Satisfaction Data
70
75
80
J-02
F M A M J J A S O N D J-03
F M A M J J A S O N D J-04
F M
Process Changes
CUSUM Chart: Patient Satisfaction
708090
100
Cusum Chart of
API and CT Concepts, 2009
010203040506070
J-02
F M A M J J A S O N D J-03
F M A M J J A S O N D J-04
F M
CU
SU
Process Changes
Cusum Chart of Patient Satisfaction Data Target = Avg = 88.296
CUSUM Chart: Patient Satisfaction
2030405060708090
100
CU
SU
Target = average = 88.29CUSUM graph
sensitive t t t
CUSUM: Patient Satisfaction with 85% Target
140160180200
01020
J-02
F M A M J J A S O N D J-03
F M A M J J A S O N D J-04
F M
Process Changesto target selected
The Slope is Target = goal = 85%
API and CT Concepts, 2009
020406080
100120140
J-02
F M A M J J A S O N D J-03
F M A M J J A S O N D J-04
F M
CU
SUM
pthe Focus of the Interpretation
Target = 85%
11/18/2009
21
Control Limits for CUSUM Chart – V‐Mask
0
-20
Target=0
Vmask Chart of %Sat(X)Cu
mul
ativ
e Su
m -40
-60
-80
100
API and CT Concepts, 2009
Sample272421181512963
-100
-120
Moving the V‐Mask (h=5, k=0.5)
umul
ativ
e Su
m
0
-20
-40
-60
Target=0
Vmask Chart of %Sat(X)
mul
ativ
e Su
m
0
-20
-40
-60
-80
Target=0
Vmask Chart of %Sat(X)
Sample
Cu
272421181512963
-80
-100
Sample
Cum
272421181512963
-100
-120
-140
40
20
0 Target=0
Vmask Chart of %Sat(X)
100
50
Vmask Chart of %Sat(X)
API and CT Concepts, 2009
Sample
Cum
ulat
ive
Sum
272421181512963
-20
-40
-60
-80
-100
Sample
Cum
ulat
ive
Sum
272421181512963
0
-50
-100
Target=0
11/18/2009
22
Alternate Form of CUSUM Control Chart Called Tabular Form
• Do not need to use V‐mask• Need to plot two statistics for each measure• Procedure:
– Let xi be the ith observation on the process– Estimate σ using screened MR’s of series– Accumulate derivations from the target μ0 above the target with one statistic, C+
– Accumulate derivations from the target μ0 below the target
API and CT Concepts, 2009
g μ0 gwith another statistic, C—
– C+ and C‐‐ are one‐sided upper and lower cusums, respectively.
The Tabular CUSUM ‐ Alternate Form
The statistics are computed as follows:The Tabular Cusum
[ ]+−
+ ++μ−= 1i0ii C)k(x,0maxC
starting values areK is the reference value (or allowance or slack value)If either statistic exceed a decision interval H, the process is considered to be out of control Often taken as a H = 5σ
[ ][ ]−
−−
−
+−−μ=
μ
1ii0i
1i0ii
Cx)k(,0maxC
)(,
0CC 00 == −+
API and CT Concepts, 2009
out of control. Often taken as a H = 5σ
11/18/2009
23
Cusum Control Chart for Patient Satisfaction Data
50
CUSUM Chart of %Sat(X)
Ci+
Cum
ulat
ive
Sum
25
0
-25
0
UCL=22.3
LCL=-22.3
API and CT Concepts, 2009
Sample272421181512963
-50
-75
Ci-
Let’s Practice
• Lets look at Template and Excel Database
• Make a Cusum chart– You will need to copy and paste the data into the appropriate template
API and CT Concepts, 2009
11/18/2009
24
Debrief ‐ Cusum
Resources:• The Data Guide. Provost and Murray, 2009. API‐Austin, Austin, TX, Chapter 7.
• Roberts, S. W., “A Comparison of Some Control Chart Procedures”, Technometrics, Vol 8, No. 3, p. 411‐430, August, 1966.
• Lucas, J., “The Design and Use of V‐mask Control Schemes,” Journal of Quality Technology, Vol 8, No. 1, pp 1‐12, January, 1976.
• Banard, G. A. “Control Charts and Stochastic Processes”, Journal of the Royal Statistical Society B21, pp 230‐271, 1959
• Evans, W. D. , “When and How to Use Cu‐Sum Charts”, Technometrics, Vol. 5, pp. 1‐22, 1963
• Sibanda, T. and Sibanda N., “The CUSUM chart method as a tool for continuous monitoring of li i l t i ti l ll t d d t ” BMC M di l R h M th d l 2007
API and CT Concepts, 2009
clinical outcomes using routinely collected data”, BMC Medical Research Methodology 2007, 7:46 doi:10.1186/1471‐2288‐7‐46
• Noyez, Luc, “A review of methods for monitoring performance in healthcare Control charts, Cusum techniques and funnel plots”, Interact Cardiovascular Thoracic Surg 2009; 9:494‐499;
• Biau, D., “Quality control of surgical and interventional procedures: a review of the CUSUM”, Qual Saf Health Care 2007;16:203–207. 2006
Over‐dispersion and Prime Charts
• The problem and examples
• Intro to template
• Work Case Study – (10)
• Debrief ‐ Tips, issues, references
API and CT Concepts, 200948
11/18/2009
25
Problem of Over dispersion with a P chart
P‐chart: Health Plan Management of Patients by Phone
50
55
60patients
ULCL
30
35
40
45
Jan‐07 Mar ‐07 May‐07 Jul ‐07 Sep‐07 Nov‐07 Jan‐08 Mar‐08 May‐08
% of all active
p
LL
CL
API and CT Concepts, 2009
“Are all these points outside the limits due to special causes, or is something else going on here?”
Month J‐07 F‐07 M‐07 A‐07 M‐07 J‐07 J‐07 A‐07 S‐07 O‐07 N‐07 D‐07 J‐08 F‐08 M‐08 A‐08Members 8755 9800
17000
16400
19500
19800
21200
22300
21600
20500
18700
18900
14300
14800
14500
14600
Mng by Phone 3852 4100 7083 7339 9406 9310 7250
10400 9250 9950 9846 9854 8034 8162 8122 8200
percent 44.0 41.8 41.7 44.8 48.2 47.0 34.2 46.6 42.8 48.5 52.7 52.1 56.2 55.1 56.0 56.2
Over dispersion: Prime Charts
• Sometimes Shewhart charts look weird!
• This can happen when subgroup sizes large– Limits on charts for attribute data impacted by subgroup size
– Larger subgroup size means tighter limits
– May be issue when subgroup > 5000
API and CT Concepts, 200950
May be issue when subgroup > 5000
• We have an alternative: Prime charts
»P’ or U’
11/18/2009
26
Alternative to the P chart: the P’‐ChartP ‐ c h a r t : H e a l t h P l a n M a n a g e m e n t o f P a t i e n t s b y P h o n e
4 5
5 0
5 5
6 0
all active patients
L L
U LC L
3 0
3 5
4 0
Ja n ‐0 7 M a r ‐0 7 M a y ‐0 7 Ju l ‐0 7 S e p ‐0 7 N o v ‐0 7 Ja n ‐0 8 M a r ‐0 8 M a y ‐0 8
% of a
API and CT Concepts, 2009
U’ Chart for Medications Errors
U Chart for Medications Errors
20
30
40
50
60
ros per 1000 en
tries
UL
CL
Data obtained from a screening of computer order entry of prescriptions in the hospitals for one month.
0
10
0
A F I N H S U B M D L J T O P R V Q W K C X G EHospital
Err
LL
CL
U' Chart for Medications Errors
40
50
60
0 en
tries
UL
Subgroup sizes (number of prescription entries for the month) ranges from 4,467 to 27,203.
Hospitals on the u chart from the smallest to largest denominator (e.g. funnel plot format).
Th i t f h lf f th
*
API and CT Concepts, 2009
0
10
20
30
A F I N H S U B M D L J T O P R V Q W K C X G EHospital
Erros per 1000
LL
CL
The points for one-half of the hospitals were outside the limits.
Based on conversation with the subject matter experts (and the very large subgroup sizes), the quality analyst created the u’.
*Hospitals rationally ordered from smallest to largest
*
11/18/2009
27
Guidance for Attribute Charts• First develop the appropriate chart (p or u).
• If the limits “appear too tight” and very large subgroupIf the limits appear too tight and very large subgroup sizes are involved:
1. Look for ways to further stratify the data• monthly into weekly or daily subgroups• organization data into department subgroups• Overall clinic data subgrouped by clinician
2. If you still end up with large subgroup sizes and a chart that is full of special causes, spend time with the subject
API and CT Concepts, 2009
that is full of special causes, spend time with the subject matter expert trying to identify and understand the special causes.
3. If you are not able to learn from the special causes, then constructed a modified attribute chart (P’ or U’ ).
Guidance for Attribute ChartsStep 1 – calculate the p chart (getting pi and σpi for each subgroup).
Step 2 – convert the individual p values to z-values usingStep 2 convert the individual p values to z values using zi = [ pi – p-bar] / σpi.
Then use the I chart calculation of moving ranges to determine the sigma for Z-values: σzi = screened MRbar divided by 1.128.(note: as with any I chart, it is very important to screen the moving ranges for special caused prior to calculating the average moving range).
Step 3: Transpose the z-chart calculations back to p values to get the
API and CT Concepts, 2009
Step 3: Transpose the z chart calculations back to p values to get the limits for the p' chart by multiplying the theoretical sampling sigma (σpi) by the between subgroup sigma (σzi ) as follows:
CL = pbar (same as original p chart)UCL = pbar + 3 σpi σziLCL = pbar - 3 σpi σzi
11/18/2009
28
Let’s PracticePercent ANC Tested
30260
23,528
27477
21,504
25529
19,760
26046
22,700
26734
23,229
24989
20,166
27112
23,864
23990
21,797
24288
20,955
25076
21,421
22528
18,213
21844
16,899
26487
23,531
23985
19,898
26391
21,153
23479
20,465
22564
21,195
21232
18,648
Total ANCTested
p chart
95
100
%
UCL = 84.90CTL = 84.20LCL = 83.51
80
85
90
95
API and CT Concepts, 2009
1/1/20082/1/2008
3/1/20084/1/2008
5/1/20086/1/2008
7/1/20088/1/2008
9/1/2008
10/1/2008
11/1/2008
12/1/20081/1/2009
2/1/20093/1/2009
4/1/20095/1/2009
6/1/20097/1/2009
8/1/20099/1/2009
10/1/2009
11/1/2009
12/1/200965
70
75
Debrief: Cautions in using the Prime modification to p and u charts
1. Don’t consider the adjustment unless the subgroup i lsizes are very large.
2. In cases where average subgroup sizes are very large (>5000), first try using different subgrouping strategies to the stratify the data into smaller rational subgroups.
3 Spend time with subject matter experts trying to
API and CT Concepts, 2009
3. Spend time with subject matter experts trying to understand and learn from the initial indications of special causes on the attribute chart before considering the modification to these charts.
11/18/2009
29
Improper use of P’ Chart
Very important to not just automatically switch to a these modified charts until the source of the o e di pe ion h been
P‐chart: Percent of Diabetic Patients with Self‐Management Goals
0102030405060708090100
I B H L K C N M E A J D F G
% of patients with goals
LL
CL
UL
Clinic
P'‐chart: Percent of Diabetic Patients with Self‐Management Goals200
*over-dispersion has been thoroughly investigated.
Only when subgroup sizes are above 2000 should the adjustment be even considered.
The purpose of
‐150
‐100
‐50
0
50
100
150
200
I B H L K C N M E A J D F G
% of patients with goals
UL
CL
LL
Clinic
note: the l imits calculated below 0% and above 100% are shown here to clarify the impact of the P'chart calculation
P' h t P t f Di b ti P ti t ith S lf M t G l*
API and CT Concepts, 2009
The purpose of Shewhart’s method is to optimize learning, not get rid of special causes.
See problem Example. Subgroup sizes range from 180 to 1845.
P'‐chart: Percent of Diabetic Patients with Self‐Management Goals
% of p
atients with goals
0
50
100
I B H L K C N M E A J D F G
CL
Clinic
*Clinics rationally ordered from smallest to largest
*
References for Over dispersion with Attribute charts1. The Data Guide. Provost and Murray, 2009. API‐Austin, Austin, TX, Chapter 8.
Debrief
2. Heimann, P.A., “Attributes Control Charts with Large Sample Sizes”, Journal of Quality Technology, ASQ, 1996, Vol 28, pp 451‐459.
3. Spiegelhalter D. “Handling overdispersion of performance indicators”. Journal of Quality and Safety in HealthCare, BMJ, 2005;14:347–51.
4. Laney DB. “Improved Control Charts for Attribute Data”. Quality Engineering, 2002; Vol. 14, p. 531–7.
API and CT Concepts, 2009
5. Mohammed, M. A. and D Laney,” Over dispersion in health care performance data: Laney’s approach”, Journal of Quality and Safety in Health Care, 2006; Vol. 15, pp.383–384.
11/18/2009
30
Autocorrelation
• Autocorrelated – use registry data
• Problem
• Examples, Intro to template
• Work Case Study – (10)
API and CT Concepts, 2009
59
• Debrief ‐ Tips, issues, references
Autocorrelation
A basic assumption in determining the limits for Shewhart charts is that the data for each subgroup arecharts is that the data for each subgroup are independent, that is the data from one subgroup does not help us predict another period.
The most common way this assumption is violated is when special causes are present.
Then subgroups associated with the special causes tend to be more alike than subgroups affected only by special
API and CT Concepts, 2009
be more alike than subgroups affected only by special causes.
The result is that these subgroups show up as “special”, exactly what the chart was designed to do.
11/18/2009
31
AutocorrelationBut sometimes process operations or data collection procedures result in data affected only by common causes that is not independent from subgroup tocauses that is not independent from subgroup to subgroup.
This phenomenon is called autocorrelation.
With a positive autocorrelation, successive data points will be similar. For time ordered data, subgroups l t th ill t d t b lik th
API and CT Concepts, 2009
close together will tend to be more alike then subgroups far apart in time.
With negative autocorrelation, successive points will tend to be dissimilar, resulting in a saw tooth pattern.
Autocorrelation
This relationship between the plotted points would make all the additional rules used with Shewhart charts invalid.additional rules used with Shewhart charts invalid.
For the I chart and Xbar and S chart, the limits might not be accurate expressions of the common cause variation and the autocorrelation will result in an increase in false signals of special causes.
API and CT Concepts, 2009
11/18/2009
32
Autocorrelation with Registry Data
The “autocorrelation problem” occurs because the data for most patients is not updated each month; only the patients whopatients is not updated each month; only the patients who come in for a visit.
If one‐third of the patients come in during the current month and have their data updated, the monthly summary will use the same data as the previous month for two‐thirds of the patients.
This creates a statistical relationship between the monthly
API and CT Concepts, 2009
This creates a statistical relationship between the monthly measures – it creates autocorrelation.
Autocorrelation in Registry Data
•I chart for the average glycated hemoglobin test (HbA1c value) from a registry of about 130 adult patients with diabetes. Patients are scheduled to visit the clinic every three months, so about one-third visit each month and their registry values are updated.
API and CT Concepts, 2009
•Special causes: 10 points outside limits, runs below the center line, and numerous points near the limits.
•Are these special causes or the impact of autocorrelation due to the use of the registry values? Because of the way the data are collected for this chart, autocorrelation was expected.
11/18/2009
33
Examine Autocorrelation using a Scatter Diagram – Point i vs. Point i‐1
Autocorrelation: Current Value vs. Next Value7.55
y = 0.7909x + 1.5449R2 = 0.6688
7.4
7.45
7.5
next
val
ue (i
+1)
API and CT Concepts, 2009
7.25
7.3
7.35
7.25 7.30 7.35 7.40 7.45 7.50 7.55Current Value (i)
Dealing with Autocorrelation• Identify the source of the autocorrelation and take appropriate
actions to learn from it and incorporate it into improvement strategies.
• If the autocorrelation is due to the sampling or measurementIf the autocorrelation is due to the sampling or measurement strategy, modify the data collection to reduce its impact.
• Continue to learn from and monitor the process as a run chart (using only visual analysis, not using run chart rules).
• Use time series analysis to model the data series and analyze the residuals from the time series using a Shewhart chart.
• Make adjustment to the control limits to compensate for the
API and CT Concepts, 2009
autocorrelation . The recommended adjustment is to increase the limits by multiplying by the factor:
___1 / √ 1‐r2 or sigma = Rbar/ [d2 * sqrt(1‐r2)]
11/18/2009
34
I Chart with Autocorrelation Adjustment
HbA1C - I chart with Limits Adjusted for Autocorrelation
7 57.55
7.67.65
c
_ UL =7.572
7.157.2
7.257.3
7.357.4
7.457.5
J05
F M A M J J A S O N D J06
F M A M J J A S O N D J -07
F M A M J J A S
Reg
istr
y A
vera
ge H
bA1
LL =7.238
CL =7.405
API and CT Concepts, 2009
Control limits adjusted to compensate for the autocorrelation. The limits are increased by multiplying by the factor:
____ 1 / √ 1-r2 or sigma = Rbar/ [d2 * sqrt(1-r2)]
Let’s Practice
• Lets look at Template and Excel Database
• Check for Autocorrelation and Adjust limits– You will need to copy and paste the data into the appropriate template
API and CT Concepts, 2009
11/18/2009
35
References for Autocorrelation on Control ChartsThe Data Guide. Provost and Murray, 2009. API-Austin, Austin, TX, Chapter 8.
Debrief
Montgomery, D. C. and Mastrangelo, C. M., “Some Statistical Process Control Methods For Autocorrelated Data”, Journal of Quality Technology 23, 1991, pp. 179–193.
Wheeler, D. 1995, “Advanced Topics in Statistical Process Control”, SPC Press, Knoxville, TN, Chapter 12. (adjustment factor)
Nelson, C. R., Applied Time Series Analysis for Managerial Forecasting, Holden-Day Inc San Francisco 1973
API and CT Concepts, 2009
Day, Inc. San Francisco, 1973
Note: Don’t overreact to autocorrelation. Most of the time, special causes cause the detection of autocorrelation. Spend time identifying the special causes.
Caution: Do Not Over‐react to Autocorrelation
API and CT Concepts, 2009
11/18/2009
36
Examining Autocorrelation for Visit Time I Chart
• Scatterplot prepared to look at autocorrelation. • The high value of r2 (autocorrelation =• The high value of r (autocorrelation = .905) could indicate autocorrelation that must be dealt with in order to use the limits on the chart. • Receptionist noted that “it was pretty clear which of the specialists were in the office each day”. • Aware of different average cycle times for each of the doctors
API and CT Concepts, 2009
for each of the doctors. • The QI team prepared a chart to show times for each of the three specialists..
I Chart Clarifying Special Causes
API and CT Concepts, 2009
11/18/2009
37
Some Other Advanced Control Charts
API and CT Concepts, 2009
73
Standardized Shewhart Charts• Shewhart charts with variable subgroup sizes (Xbar and S Chart, P chart, U chart) result in variable control limits.
•Sometimes this complexity in the appearance of the chart results in them not b i dbeing used.
• An alternative is called the standardized Shewhart Chart. To construct the chart, the data is transformed using:
Z = (X-u) / σ where z is the standardized value, X is the original data value; µ is the mean and σ the standard deviation of the original data.•
API and CT Concepts, 2009
•• Using this transformation, data for all the types of Shewhart charts can be transformed so that the resulting chart has limits that are always:•
CL = 0 UL = 3 LL = -3
11/18/2009
38
Use of Standardized Chart
ent
Percent Unplanned Readmissionsp chart
UCL = 4.41
4
5
6
7
Standarized P-chart for Unplanned Readmissions
3
4
s
UL
Per
ce
Mean = 2.57
LCL = 0.72
LOS Reduction Testing Started
LOS Red. Lg Test
J05 F M A M J J A S O N D J-06 F M A M J J A S O N D J-07 F0
1
2
3
API and CT Concepts, 2009-4
-3
-2
-1
0
1
2
3
J05 F M A M J J A S O N D J-06 F M A M J J A S O N D J-07 F
Stan
ndar
dize
d Pe
rcen
t ras
adm
its
Data from Jan 05 - Mar 06 use to calculate limits
CL = 2.22%
LL
Control Charts with Slanted Center‐line
US Median % Obesity30
y = 0.8879x + 10.346R2=09877
10
15
20
25
Med
ian
% O
besi
ty
API and CT Concepts, 2009
R = 0.9877
50 5 10 15 20 25
Year since 1989 (year1)
11/18/2009
39
Seasonal Effects on a Shewhart Chart
API and CT Concepts, 2009
Seasonal Effects on a Shewhart Chart
API and CT Concepts, 2009
11/18/2009
40
Selecting the Appropriate Shewhart Chart
API and CT Concepts, 2009
Selecting the Appropriate Shewhart ChartType of Data
Count or Classification (Attribute Data)‐Qualitative data such as # errors, # nonconformities or # of items that passed or failed) ‐ Discrete: must be whole number when originally collected (can’t be fraction or scaled data when originally collected)‐This data is counted, not measured
Continuous (Variable Data) ‐Quantitative data in the form of a measurement‐Requires a measurement scale ‐Time, Money, Scaled Data (i.e. length, height, weight, temperature, mg.) and Throughput (volume of workload/ productivity)
Count (Nonconformities)1,2,3,4, etc.
Classification (Nonconforming)Either/Or, Pass/Fail, Yes/No
Equal Area of Opportunity
Unequal Area of Opportunity
Unequal or Equal Subgroup Size
SubgroupSize of 1(n=1)
Unequal Or Equal SubgroupSize (n>1)
C Chart U Chart P ChartI Chart (also known as an X chart)
X‐Bar and S
API and CT Concepts, 2009
P Chart an X chart)
Number ofNonconformities
NonconformitiesPer Unit Percent
NonconformingIndividual Measures Average and Standard
Deviation
Other types of control charts for count/classification data:1. NP (for classification data)2. T-chart [time (or event, items, etc.) between rare events]3. Cumulative sum (CUSUM)4. Exponentially weighted moving average (EWMA)5. Geometric distribution chart (G chart for count data) 6. Standardized control chart
Other types of control charts for continuous data:7. X‐bar and Range8. Moving average9. Median and range10. Cumulative sum (CUSUM)11. Exponentially weighted moving average (EWMA)12. Standardized control chart
11/18/2009
41
Develop a Control Chart – Case AThe QI team wants to make a 50% reduction in the use of restraints in the hospital. The number of inpatient restraints and the total patient days each month are recorded.
Type of control chart (s)
Subgrouping Strategy
API and CT Concepts, 2009
Develop a Control Chart – Case BYour clinic team is interested in the average time patients spend in the waiting area, so every day at 11:00a and 5:00p you pick the next 5 patients and measure their actual waiting time in minutes.
Type of control chart (s)Type of control chart (s)
Subgrouping Strategy
API and CT Concepts, 2009
11/18/2009
42
Develop a Control Chart – Case CThe ICU nurses want to monitor the ventilator-associated pneumonia (VAP) rate. They have data on the for each pneumonia cases, including the date the case occurred and the total number of vent days in the ICU each day. One of the nurses expressed concern that this would be difficult to track because they often go more than a month without adifficult to track because they often go more than a month without a VAP.
Type of control chart (s)
Subgrouping Strategy
API and CT Concepts, 2009
Develop a Control Chart – Case DThe clinic QI team is trying to improve evidence based care for their patients with diabetes. Each week they record the number of diabetic patients seen in the clinic and the number who received eye exams.
Type of control chart (s)Type of control chart (s)
Subgrouping Strategy
API and CT Concepts, 2009
11/18/2009
43
Develop a Control Chart – Case EThe Chief Medical Officer’s office monitors mortality in the hospital. They feel it is very important to quickly detect any change in the system that results in an increase in mortality.
Type of control chart (s)Type of control chart (s)
Subgrouping Strategy
API and CT Concepts, 2009
Develop a Control Chart – Case FOne of your diabetic patients records blood glucose test results at three specific times each day: 1st thing in the morning, at 11:00a before lunch, and at 1:00 after lunch. She has daily data for the past two months.
Type of control chart(s)
Subgrouping strategy?
API and CT Concepts, 2009
11/18/2009
44
Making Shewhart Charts More Effective
• Guidance regarding limits on Shewhart Charts– When to make limits
– When to revise limits
• Tips for good graphical display
API and CT Concepts, 2009
87
Principles for Making Limits
• If we have less than 12 data points use run chart rather than Shewhart chart– Just the data plotted as below. No limits, centerline
Is Our Volume of Net New Patients Stable?
ents 80
100
120
API and CT Concepts, 2009Sequential Months
# of Patie
1 2 3 4 5 6 7 8 9 10 110
20
40
60
11/18/2009
45
Principles for Making Limits• When have 12 or more data points may calculate and use
the trial limitstrial limits
•• Extend trial limitsExtend trial limits
atients
Trial Limits60
80
100
120
UL 94.0
Is Our Volume of Net New Patients Stable?
API and CT Concepts, 2009Sequential Months
# of Pa Trial Limits
1 2 3 4 5 6 7 8 9 10 11 12 130
20
40
60
Mean 15.9
Principles for Revising Limits
1. Revise limits when moving from trial limits to i iti l li iti iti l li itinitial limits initial limits
• Use trial limits until get 20‐30 data points then revise. (If made trial limits with, for example, first 12 points don’t update trail limits with each additional point)
• When 20‐30 subgroups become available revise the trial limits to create initial limitsinitial limits
API and CT Concepts, 2009
– 20 to 30 subgroups is when Shewhart chart becomes strong
11/18/2009
46
Is Our Volume of Net New Patients Stable?
100
120
Is Our Volume of Net New Patients Stable?
Trial Limits
100
120
UL 94.0
Initial Limits
# of Patients
40
60
80
UL 73.3
Mean 40.0
# of Patients
40
60
80
Mean 15.9
API and CT Concepts, 2009 Sequential Months1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
0
20
LL 6.7
Sequential Months
1 2 3 4 5 6 7 8 9 10 11 12 130
20
Principles for Revising Limits
2. Revise when the initial Shewhart chart has special causes and there is a desire to use the calculated li it f l i f d t t b ll t d i thlimits for analysis of data to be collected in the future
• If you have 20‐30 data points and you find special cause when you calculate your initial limits then:– Investigate and address special cause – Recalculate limits without the special cause data in your i i i l li i
API and CT Concepts, 2009
initial limits• Always leave data point(s) visible on chart• If you choose to remove data from limits/mean, annotate your decision
11/18/2009
47
Is Our Volume of Net New Patients Stable?8/98 9 10 11 12 1/99 2 3 4 5 6 7 8 9 10 11 12 1/00 2 3 4 5 6 760 36 48 31 52 87 64 47 24 39 35 45 29 33 30 31 41 34 24 37 33 30 39 32
MonthData
100
120
New PA hired
# of Patients
40
60
80
UL 64.4
API and CT Concepts, 2009Sequential Months1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
0
20
Mean 36.8
LL 9.2
Is Our Volume of Net New Patients Stable?
100
120Is Our Volume of Net New Patients Stable?
100
120
New PA hired
# of Patients
40
60
80UL 73.3
Mean 40.0 # of Patients
40
60
80
UL 64.4
Mean 36.8
API and CT Concepts, 2009Sequential Months1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1718 19 20 21 22 23 24
0
20
LL 6.7
Sequential Months1 2 3 4 5 6 7 8 9 1011 12 13 14 15 16 1718 19 20 21 22 23 24
0
20
LL 9.2
11/18/2009
48
Principles for Revising Limits
• You have 20‐30 data points and your initial limits indicate that your process is stable
G d ti i t t d th li it d– Good practice is to extend the limits and centerline into the future and plot incoming data against these initial limits
API and CT Concepts, 2009
Number of Surgical Complications
20
25
UL 18.78
# Co
mplications
10
15
Mean 9.52
Mean
API and CT Concepts, 2009Sequential Subgroups of 50 Surgeries1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1718 19 20 21 22 23 24 25 26 2728 29 3031 32 33 34 35 36 3738 39 4041 42 43 44 45 46 4748 49
0
5
LL 0.26
11/18/2009
49
Principles for Revising Limits
3. When improvements have been made to the d h lt d i i lprocess and have resulted in special cause
• Recalculate limits that represent the new improved process.
API and CT Concepts, 2009
Center DD:Time ED to OR ‐ Isolated Femur Fractures
2500
3000
Protocol change started
Minutes
1000
1500
2000
UL 1800.6
API and CT Concepts, 2009
g
Sequential Cases1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
0
500Mean 540.7
11/18/2009
50
Center DD:Time ED to OR ‐ Isolated Femur Fractures
2500
3000
Minutes
1000
1500
2000
UL 274.8
API and CT Concepts, 2009Sequential Cases1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
0
500
Mean 173.8
UL 270
Mean
LL 57
Emergency Fast Track Average Length of Stay (ALOS) for Admitted PatientsEmergency Fast Track Area
April 2004 - April 2005Individuals
1200
1400
1600
AL
OS
(Min
utes
)
400
600
800
1000UCL = 1031.79
Mean = 480.29
API and CT Concepts, 2009Source: PromedWeek Ending
9/4/
2004
9/11
/2004
9/18
/2004
9/25
/2004
10/2/
2004
10/9/
2004
10/16
/2004
10/23
/2004
10/30
/2004
11/6/
2004
11/13
/2004
11/20
/2004
11/27
/2004
12/4/
2004
12/11
/2004
12/18
/2004
12/25
/2004
1/1/
2005
1/8/
2005
1/15
/2005
1/22
/2005
1/29
/2005
2/5/
2004
2/12
/2005
2/19
/2005
2/26
/2005
3/5/
2005
3/12
/2005
3/19
/2005
3/26
/2005
4/2/
2005
4/9/
2005
4/16
/2005
4/23
/2005
0
200
11/18/2009
51
%)
Percent of Total patients Admitted through Main ED- TrendstarMain ED & EFT
October 2000 - March 2005p chart
40
50
Perc
ent
of P
atie
nts
Ad
mit
ted
(%
20
30UCL = 27.04
Mean = 23.88
LCL = 20.71
API and CT Concepts, 2009Source: TrendstarWeek Ending
10/1/
2000
11/1/
2000
12/1/
2000
1/1/
2001
2/1/
2001
3/1/
2001
4/1/
2001
5/1/
2001
6/1/
2001
7/1/
2001
8/1/
2001
9/1/
2001
10/1/
2001
11/1/
2001
12/1/
2001
1/1/
2002
2/1/
2002
3/1/
2002
4/1/
2002
5/1/
2002
6/1/
2002
7/1/
2002
8/1/
2002
9/1/
2002
10/1/
2002
11/1/
2002
12/1/
2002
1/1/
2003
2/1/
2003
3/1/
2003
4/1/
2003
5/1/
2003
6/1/
2003
7/1/
2003
8/1/
2003
9/1/
2003
10/1/
2003
11/1/
2003
12/1/
2003
1/1/
2004
2/1/
2004
3/1/
2004
4/1/
2004
5/1/
2004
6/1/
2004
7/1/
2004
8/1/
2004
9/1/
2004
10/1/
2004
11/1/
2004
12/1/
2004
1/1/
2005
2/1/
2005
3/1/
2005
0
10
Designing Effective Shewhart Charts
• Tip 1: Subgroup Size– Too small
• Classification data: see guidelines for P chart
• Count data: U chart if center line less than 9 will be no lower limit. If center line less than 1.4 will have too many 0’s
– Too large
API and CT Concepts, 2009
– Too large• Over‐dispersion issue. Consider Prime charts
11/18/2009
52
Guidelines for Selecting Subgroup Size for an Effective P chart
Average PercentNonconforming
Units (pbar)
Minimum Subgroup Size (n) Required to Have
< 25% zero for p'sMinimum Subgroup Size
Guideline(n>300/pbar)
Minimum Subgroup Size Required to Have
LCL > 00.1 1400 3000 90000.5 280 600 18001.0 140 300 9001.5 93 200 6002 70 150 4503 47 100 3004 35 75 2205 28 60 1756 24 50 1308 17 38 104
10 14 30 81
API and CT Concepts, 2009
10 14 30 8112 12 25 6615 9 20 5120 7 15 3625 5 12 2830 4 10 2240 3 8 1450 2 6 10
Note: for p>50, use 100‐p to enter the table (e.g. for p=70% use table p of 30%, for p=99% use table p of 1%, etc.) Source: The Data Guide: L Provost and S. Murray, 2009
Designing Effective Shewhart Charts
• Tip 2: Rounding Data: have choicesWh i t t l th– When using computer system, always err on the side of maintaining too many decimal place
– Rounding used for compliance may not be useful for learning
• E.G LOS in days for compliance may be better in minutes for learning
API and CT Concepts, 2009
– Rounding center lines/ limits: keep one more decimal than the statistic plotted on the chart.
• E.g. data 11.2…center line 9.79
11/18/2009
53
Designing Effective Shewhart Charts
• Tip 3: Formatting Charts– Put related graphs on same page
API and CT Concepts, 2009
%
Know Who To Call W ith Questions
UCL = 52.5
Mean = 32.8
LCL = 13.1
Provider Gives Card Test 1 Test 3ImplementTest 2
F-07 M A M J J A S O N D J-08 F M A M J J A S O N D J-09 F M A M J J A S0
10
20
30
40
50
60
70
80
90
100
%
Received Enough Information From Provider
UCL = 62.960
70
80
90
100
Good
Good
%
Mean = 42.1
LCL = 21.4
Ask Question Test 1/2
Test 3/4
Test 5
Implement
F-07 M A M J J A S O N D J-08 F M A M J J A S O N D J-09 F M A M J J A S0
10
20
30
40
50
%
Treated With Respect By Provider
UCL = 68.8
Mean = 47.9
LCL = 26.9
Provider Walk Pt Out Test 1Test 2 Test 4
20
30
40
50
60
70
80
90
100
Good
API and CT Concepts, 2009
Provider Walk Pt. Out-Test 1Test 3 Implement
F-07 M A M J J A S O N D J-08 F M A M J J A S O N D J-09 F M A M J J A S0
10
%
Would Recommend The Clinic
UCL = 79.3
Mean = 58.6
LCL = 37.9
Provider Gives Card
Provider Walks Pt. Out
Provider Asks Question
F-07 M A M J J A S O N D J-08 F M A M J J A S O N D J-09 F M A M J J A S0
10
20
30
40
50
60
70
80
90
100
Good
11/18/2009
54
Designing Effective Shewhart Charts
• Tip 3: Formatting Charts– Put related graphs on same page
– Presentation:• Size matters
– ratio of horizontal to vertical of 5:2
API and CT Concepts, 2009
%
Received Enough Information From Provider
UCL = 62.9
Mean = 42.1
LCL = 21.4
T t 3/4 Implement20
30
40
50
60
70
80
90
100
Ask Question Test 1/2
Test 3/4
Test 5
Implement
F-07 M A M J J A S O N D J-08 F M A M J J A S O N D J-09 F M A M J J A S0
10
Received Enough Information From Provider
UCL = 62.960
70
80
90
100
API and CT Concepts, 2009
%
Mean = 42.1
LCL = 21.4
Ask Question Test 1/2
Test 3/4
Test 5
Implement
F-07 M A M J J A S O N D J-08 F M A M J J A S O N D J-09 F M A M J J A S0
10
20
30
40
50
11/18/2009
55
Designing Effective Shewhart Charts
• Tip 3: Formatting Charts– Put related graphs on same page
– Presentation:• Size matters
– ratio of horizontal to vertical of 5:2
• Vertical Scalei l d th li it i th iddl 50% Oth 50% f h
API and CT Concepts, 2009
– include the limits in the middle 50%.Other 50% of graph space as “white space” on either side of limits. Don’t force scale to include 0 unless important to learning.
– If the data can’t go below 0 or exceed 100% don’t scale beyond these
%
% Patients Leaving Against Medical Advice (AMA)Pilot Unit (p chart)
UCL = 3.16
Mean = 2.05
LCL = 0.94
Team Formed
Change 1Change 2
Change 3
Change 4Change 5
Implementation
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
Just Right
F 06 M A M A M J J A S O N D J 07 F M A M J J A S O N D J-080.0
%
Percent of Patients Leaving Against Medical Advice (AMA)Pilot Unit (p chart)
UCL = 3.16
Mean = 2.05
LCL = 0.94Change 1 Change 2
Change 3
Change 4
Change 5Implementation
F 06 M A M A M J J A S O N D J 07 F M A M J J A S O N D J-08
1.00
1.50
2.00
2.50
3.00
3.50
Percent of Patients Leaving Against Medical Advice (AMA)
Too Small a Scale
API and CT Concepts, 2009
%
Percent of Patients Leaving Against Medical Advice (AMA)Pilot Unit (p chart)
UCL = 3.16
Mean = 2.05LCL = 0.94
Team Formed
Change 1Change 2
Change 3
Change 4 Change 5
Implementation
F 06 M A M A M J J A S O N D J 07 F M A M J J A S O N D J-080
2
4
6
8
10
12
14
16
18
20
Too Large a Scale
11/18/2009
56
%
% Patients with LVF Given ACEI at Discharge (p chart)
UCL = 106.07
Mean = 58.33
20
40
60
80
100
120
140
InappropriateScale
LCL = 10.60
Nov 06 D J -07 F M A M J J A S O N D J-08 F M A M J J A S O N D-20
0
20
% Patients with LVF Given ACEI at Discharge (p chart)UCL = 106.07
80
100
AppropriateScale
API and CT Concepts, 2009
%
Mean = 58.33
LCL = 10.60
Nov 06 D J -07 F M A M J J A S O N D J-08 F M A M J J A S O N D0
20
40
60
InappropriateScale
%
AMI-9 Inpatient Mortality
30
40
50
60
70
80
90
100
AppropriateScale
UCL = 20.53
Mean = 8.35
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 260
10
20
30
AMI-9 Inpatient Mortality
25
30
35
API and CT Concepts, 2009
%
UCL = 20.53
Mean = 8.35
Nov 06 D J -07 F M A M J J A S O N D J-08 F M A M J J A S O N D0
5
10
15
20
11/18/2009
57
Designing Effective Shewhart Charts
• Tip 3: ContinuedP t ti– Presentation:
• Labels – include user friendly labels on axes, centerline, limits, and other key values (targets, baselines, requirements, etc) on chart
• Annotations: Integrate key annotations
• Gridlines‐ keep gridlines and other lines/colors/markings to a minimum
API and CT Concepts, 2009
• Data‐May be helpful to display –if legible!
• Points‐ connecting the points is optional – If in time order may
– If data not time order do not connect points
Received Enough Information From Provider
80
90
100
%
UCL = 62.9
Mean = 42.1
LCL = 21.4
Test 3/4 Implement20
30
40
50
60
70
80
API and CT Concepts, 2009
Ask Question Test 1/2
Test 3/4
Test 5
F-07 M A M J J A S O N D J-08 F M A M J J A S O N D J-09 F M A M J J A S0
10
11/18/2009
58
Designing Effective Shewhart Charts
• Tip 3: ContinuedP t ti– Presentation:
• Labels – include user friendly labels on axes, centerline, limits, and other key values (targets, baselines, requirements, etc) on chart
• Annotations: Integrate key annotations
• Gridlines‐ keep gridlines and other lines/colors/markings to a minimum
API and CT Concepts, 2009
• Data‐May be helpful to display –if legible!
• Points‐ connecting the points is optional – If in time order may
– If data not time order do not connect points
AMI-9 Inpatient Mortality Nov 06
49
1
D
66
3
J -07
60
2
F
44
1
M
52
3
A
54
5
M
47
4
J
36
6
J
49
4
A
46
4
S
55
2
O
55
5
N
51
7
D
62
15
J-08
31
2
F
36
3
M
35
2
A
55
2
M
58
9
J
36
2
J
39
2
A
35
2
S
33
6
O
30
1
N DDate# Pts
Deaths
%
2.04%
3
4.55% 3.33% 2.27%
3
5.77%
5
9.26% 8.51%
6
16.67% 8.16% 8.70% 3.64%
5
9.09% 13.73%
5
24.19% 6.45%
3
8.33% 5.71% 3.64%
9
15.52% 5.56% 5.13% 5.71%
6
18.18% 3.33%
Deathspercent
UCL = 20.53
10
15
20
25
30
API and CT Concepts, 2009
Mean = 8.35
Nov 06 D J -07 F M A M J J A S O N D J-08 F M A M J J A S O N D0
5
10
11/18/2009
59
API and CT Concepts, 2009
API and CT Concepts, 2009
11/18/2009
60
Selecting the Appropriate Shewhart ChartType of Data
Count or Classification (Attribute Data)‐Qualitative data such as # errors, # nonconformities or # of items that passed or failed) ‐ Discrete: must be whole number when originally collected (can’t be fraction or scaled data when originally collected)‐This data is counted, not measured
Continuous (Variable Data) ‐Quantitative data in the form of a measurement‐Requires a measurement scale ‐Time, Money, Scaled Data (i.e. length, height, weight, temperature, mg.) and Throughput (volume of workload/ productivity)
Count (Nonconformities)1,2,3,4, etc.
Classification (Nonconforming)Either/Or, Pass/Fail, Yes/No
Equal Area of Opportunity
Unequal Area of Opportunity
Unequal or Equal Subgroup Size
SubgroupSize of 1(n=1)
Unequal Or Equal SubgroupSize (n>1)
C Chart U Chart P ChartI Chart (also known as an X chart)
X‐Bar and S
API and CT Concepts, 2009
P Chart an X chart)
Number ofNonconformities
NonconformitiesPer Unit Percent
NonconformingIndividual Measures Average and Standard
Deviation
Other types of control charts for count/classification data:1. NP (for classification data)2. T-chart [time (or event, items, etc.) between rare events]3. Cumulative sum (CUSUM)4. Exponentially weighted moving average (EWMA)5. Geometric distribution chart (G chart for count data) 6. Standardized control chart
Other types of control charts for continuous data:7. X‐bar and Range8. Moving average9. Median and range10. Cumulative sum (CUSUM)11. Exponentially weighted moving average (EWMA)12. Standardized control chart
top related