six sigma for clinicians what does it really mean?
TRANSCRIPT
SIX SIGMA FOR CLINICIANSWhat does it really mean?
SIX SIGMA IN HEALTHCARENew Orleans, Louisiana
March 3-4, 2005
Larry V. Staker MD, FACP
CMO Deseret Mutual
Salt Lake City, Utah
“If to do were as easy as to knowwhat were good to do, then chapelshad been churches and poor men’scottages, princes palaces.”
Merchant of VeniceWilliam Shakespeare
© Larry V. Staker MD
Clinicians Six Sigmaor
Clinical Practice Improvement
PEDAGOGY
(are there better ways to teach)
SIXSIGMA
EBM
STANDARDS
MEASUREMENT
OUTCOMES
ACCOUNTABILITY
$ENSE
1
2
3
4
6
5
1
METHODS
Start with Evidence Based Medicinefor
“CLINICAL PRACTICE IMPROVEMENT”
Evidence Based Medicine
SUMSwith disease without disease
test pos a b a+btest neg c d c+d
a+c b+d a+b+c+d
NUMBERS2 x 2 TABLE
SUMSwith disease without disease
test pos a b a+btest neg c d c+d
a+c b+d a+b+c+d
NUMBERS2 x 2 TABLE
UNDERSTANDING2 x 2 TABLES
SUM RATIOa b a+bc d c+d
a+c b+d T=(a+b)+(c+d)a/(a+c) d/(b+d)c/(a+b) b/(c+d)LR (+) LR (-) Likelihood
sens/(1-spec) (1-sens)/spec Ratios
(a+c)/T
NUMBER PERCENT / RATEa/(a+b)d/(c+d)
IMPROVING DIAGNOSTIC SKILLS The Basic Tool is a 2 x 2 Table
AImproving Diagnostic Skills
A TOOL FOR DETERMINING THE USEFULNESS OF A DIAGNOSTIC TEST (enter numbers in white boxes) © Larry V. Staker MD
SUMS RATIOSwith disease without disease
test pos 475.0 50.0 525 90.5% PPV 90.5%test neg 25.0 450.0 475 94.7% NPV 5.3%
500 500 1000 50.0% PTPSnOUT SENS
True (+) rate 95.0% 10.0% False (+) rate
False (-) rate 5.0% 90.0% True (-) rateSPEC SpIN
LR (+) LR (-)9.50 0.06
sens/(1-spec) (1-sens)/spec
NUMBERS PERCENTS / RATES / PROPORTIONS
Likelihood Ratios
Post Test Probability given positive result
(1/NPV) = Post Test Probability given negative result
Pre Test Probability of disease estimated from Hx / PE
THE BASIC TOOL FOR IMPROVING DIAGNOSTIC SKILLS
POST TEST PROBABILITY (pos)POST TEST PROBABILITY (neg)
PRE TEST PROBABILITY
LIKELIHOODRATIOS
A SOURCE OF INFORMATIONTO IMPROVE SKILLS
OF DIAGNOSIS
ISBN = 9-943126-74-6
DIAGNOSTIC STRATEGIESFor Common Medical Problems
Second Edition
Edited By: Edgar R Black MDPublisher: American College of Physicians
TEST SEN SPECETT and ST Seg ?
0.5 - 0.99 86.0% 77.0%1.0 - 1.49 65.0% 89.0%
1.50 - 1.99 42.0% 98.0%2.0 - 2.49 33.0% 99.0%
>= 2.5 20.0% 99.5%ETT and
Thallium 88.0% 91.0%SPECT 90.0% 72.0%
Adenosine 89.0% 83.0%Dipyridamole-o 87.0% 75.0%Dipyridamole-iv 90.0% 78.0%
Dobutamine 91.0% 86.0%Stress ECHO 81.0% 89.0%Dobutamine ECHO 81.0% 83.0%
THE OUTPUT OF THE TOOLpost-test probabilities and likelihood ratios
PreTOR PostTLR © Larry V. Staker MDDisease Pos Test
PreTProb Pre:1 Post:1 PTP Disease - Pos Test
75.0% 7.5 2.5 3.00 28.50 96.6% 21.6% CI 95 ( 95.5% 97.7% )10.0% 1 9 0.11 1.06 51.4% 41.4% CI 95 ( 48.3% 54.4% )20.0% 2 8 0.25 2.38 70.4% 50.4% CI 95 ( 67.5% 73.2% )
30.0% 3 7 0.43 4.07 80.3% 50.3% CI 95 ( 77.8% 82.7% )40.0% 4 6 0.67 6.33 86.4% 46.4% CI 95 ( 84.2% 88.5% ) PostTLR50.0% 5 5 1.00 9.50 90.5% 40.5% CI 95 ( 88.7% 92.3% ) >2560.0% 6 4 1.50 14.25 93.4% 33.4% CI 95 ( 91.9% 95.0% )70.0% 7 3 2.33 22.17 95.7% 25.7% CI 95 ( 94.4% 96.9% )
80.0% 8 2 4.00 38.00 97.4% 17.4% CI 95 ( 96.5% 98.4% )90.0% 9 1 9.00 85.50 98.8% 8.8% CI 95 ( 98.2% 99.5% )
PreTOR PostTLR © Larry V. Staker MDDisease Neg Test
PreTProb Pre:1 Post:1 PTP Disease - Neg Test
75.0% 7.5 2.5 3.00 0.17 14.3% 60.7% CI 95 ( 12.1% 16.5% )10.0% 1 9 0.11 0.01 0.6% 9.4% CI 95 ( 0.1% 1.1% )20.0% 2 8 0.25 0.01 1.4% 18.6% CI 95 ( 0.6% 2.1% )
30.0% 3 7 0.43 0.02 2.3% 27.7% CI 95 ( 1.4% 3.3% )40.0% 4 6 0.67 0.04 3.6% 36.4% CI 95 ( 2.4% 4.7% ) PostTLR50.0% 5 5 1.00 0.06 5.3% 44.7% CI 95 ( 3.9% 6.6% ) <0.2560.0% 6 4 1.50 0.08 7.7% 52.3% CI 95 ( 6.0% 9.3% )70.0% 7 3 2.33 0.13 11.5% 58.5% CI 95 ( 9.5% 13.5% )
80.0% 8 2 4.00 0.22 18.2% 61.8% CI 95 ( 15.8% 20.6% )90.0% 9 1 9.00 0.50 33.3% 56.7% CI 95 ( 30.4% 36.3% )
Test
Treat
Treat
PreTOdds
PreTOdds
BENEFIT OF NEGATIVE TEST
Observe
BENEFIT OF POSITIVE TEST
Observe
Test
CONFIDENCE INTERVALSLR rule: +>25; -<0.25
IMPROVING TREATMENT SKILLS The Basic Tool is a 2 x 2 Table
BChoosing Best Treatment
Enter numbers in white boxes © Larry V. Staker MD
SUMS MULT DIFF RECP RELATIONSHIPSTTG NTTG PROPORTIONS
experiment 140 20 160 87.5% ODDS
control 35 125 160 21.9% ODDS RATIOS
SUM 320
PERCENT 400.0% 0.0%MULTIPLICATION 17500 a*d c*b 700
DIFFERENCE 65.6% |EER-CER| ABI1.5 1/ABI NNT
RATIO 25 (a*d)/(c*b) ROR
BASIC TOOL FOR EVALUATING EFFECTIVENESS OF TREATMENT
NUMBERS
a/(a+b) TTG or EER
PERCENTS / RATES / PROPORTIONS
c/(c+d) NTTG or CER
RECIPROCAL
RB = EER/CER |ABI/CER| RBI
RBI, ABI, and NNT
THE OUTPUT OF THE TOOLevaluation standards of peer review journals
FormulasSE Ln Ln SE
p1=a/(a+b) 0.035 EER 50.0% 43.1% 56.9% EER Experiment Event Ratep2=c/(c+d) 0.029 CER 15.6% 10.0% 21.3% CER Control Event RateI EER-CER I 0.036 ARR 34.4% 27.3% 41.5% ARR Absolute Risk Reduction1/ARR NNT 2.9 2.4 3.7 NNT Number Needed to TreatEER/CER RR 320.0% 217.6% 470.7% RR Relative Risk
0.197 1.1632 CI95 LnRR 0.777 1.549 CI95 LnRR Natural Log RBI EER-CER I /CER RRR 220.0% 117.6% 370.7% RRR Relative Risk Reduction(a*d)/(c*b) ROR 5.4 3.2 9.0 ROR Relative Odds Ratio
0.260 1.686 CI95 LnOR 1.178 2.195 CI95 LnOR Natural Log OR
CI-95Standard Error Calculation Enumerative Statistical Analysis
RBI, ABI, and NNT or
RRR, ARR, and NNT
CONFIDENCEINTERVALS
PDSA
HEARING, SEEING and MEASURING
The Voice Of The Process
VOP
PDSA
STUDY
PLAN
DO
ACT
PROCESSIMPROVEMENT
PDSA: Minimize Variation
– There will always be some variation in a process
– But we can work to minimize variation around a mean or target
40
45
50
55
60
65
70
1 11 21 31 41 51 61 71 81
GOAL= reduce variation
Mean
UCL
LCL
RAPID CYCLE TESTING
Discovery Learning
P
D
S
A
P
D
S
A
P
D
S
A
P
D
S
A
Thomas W. Nolan Ph.D.
THE GAME OF IMPROVEMENTThe Work or
Process Base
The Measurement of Population Base
The Benchmark orEvidence Base
The Result or Outcome Base
© Larry V. Staker MD
TTG
1
2 4
3
5
SIX SIGMA
HEARING, SEEING, and MEASURING
Voice Of The CustomerDefects Reduction Error Free Yield
VOCSigma Metric
SIX SIGMA
1a
1b
1c2
3
4
5
PROCESSIMPROVEMENT
DEFINECORE PROCESSES
DEFINEKEY
CUSTOMERS
DEFINECUSTOMER
REQUIREMENTS
MEASURECURRENT
PERFORMANCE
ANALYZE
IMPROVE
CONTROLINTEGRATE
EXPANDP
D
C
A
SIX SIGMA: Customer ServiceThe process shown here is stable.
But why does it need to be improved?
} CustomerNeed
Time
LSL
USL
UCL
LCL
SIX SIGMA and PDSA
STUDY
PLAN
DO
ACT
1a
1b
1c2
3
4
5
PROCESSIMPROVEMENT
IDENTIFYCORE PROCESSES
IDENTIFYKEY
CUSTOMERS
DEFINECUSTOMER
REQUIREMENTS
MEASURECURRENT
PERFORMANCE
ANALYZE
IMPROVE
CONTROLINTEGRATE
EXPAND
Although we may do a good job of teaching the best medical practice or treatment available today, we do a poor job of teaching ourselves how to decide when what we learnedin the past is no longer good enoughand needs to be changed.
2
STANDARDS
NATIONALLY RECOGNIZED
“Evidence-Based”
CLINICAL STANDARDS
ATP III - Standards
STANDARDS FORDIABETES CARE
1. HbA1c <= 7.0
2. LDLC <= 100
3. BP <= 135/85
4. Eye Exam Every year
5. Foot Exam Every visit
6. Microalbumin / Creat Once a year
USING STANDARDS
1. Find an acceptable evidence-based Standard
2. Hold yourself to that Standard
3. Measure performance by that Standard
4. Evaluate and grade against the Standard (TTG)
5. Make $ense using the Standard
6. Negotiate Pay for Performance (P4P) from TTG
3
MEASUREMENT
Start by teaching the use of simpleMeasurement Tools
WHAT AND HOW WE MEASURE
SATISFACTION
COST QUALITY
PHYSICIANPERFORMANCE
WORK
WHAT AND HOW WE MEASURE
PHYSICIANPERFORMANCE
COST QUALITY
SATISFACTION
WORK
WHAT AND HOW TO MEASURE
OUTCOMESCOST QUALITY
SATISFACTION
© Larry V. Staker MD
UNDERSTANDING VARIATION
SPECIAL CAUSE VARIATION SPECIAL CAUSE VARIATION
COMMON CAUSE VARIATION
© Larry V. Staker MD
HOW WE MEASURETime Sequence Data Display
TIME
KPVMEDIAN
SPECIAL CAUSE VARIATION
SPECIAL CAUSE VARIATION
COMMON CAUSE VARIATION
UCL
LCL
COMMON CAUSE VARIATION
Intraoccular Traumatic Test
Rapid Feedback of Information
Time Ordered Sequence
LINE AND SPEC CHART
LEARNING TO USE
LINE CHART
KPV PLOTTED IN TIME ORDERED SEQUENCE
60
80
100
120
140
160
180
200
60
80
100
120
140
160
180
200
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
81 98 110 99 110108120 96 90 91 85 108 99 92 107102 83 92 125 98 102102109 95 116102106 98 80 130113
FB
S
dayfbs
SPECIFICATION CHARTPATIENT
60
80
100
120
140
160
180
200
60
80
100
120
140
160
180
200
1 2 3 4 5 6 7 8 9 10111213141516171819202122232425262728293031
FB
S
TREATMENT GOAL (TG)
UPPER SPECIFICATION LIMIT (USL)
LOWER SPECIFICATION LIMIT (LSL)
day
TREATMENT TO GOAL
SUPERIMPOSE
SPECIFICATION CHARTS ON LINE RUN OR CONTROL CHARTS
and use
THE INTEROCULAR TRAUMATIC TEST
( ITT )
Joseph Berkson MD, PhD. Mayo Clinic
© Larry V. Staker MD
LINE AND SPEC CHART
KPV PLOTTED IN TIME ORDERED SEQUENCE
60
80
100
120
140
160
180
200
60
80
100
120
140
160
180
200
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
81 98 110 99 110108120 96 90 91 85 108 99 92 107102 83 92 125 98 102102109 95 116102106 98 80 130113
FB
S
TREATMENT GOAL
USL
LSL
dayfbs
SPECIFICATION CHART Diabetes Mellitus
PATTERN: INTERVENTION:
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
60
80
100
120
140
160
180
200
3
4
5
6
7
8
9
10
11
DAY OF MONTH
FG (mg/dl) FG (mmol/l)
USL
CL
LSL
SPECIFICATION CHART Diabetes Mellitus
PATTERN: INTERVENTION:
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
60
80
100
120
140
160
180
200
3
4
5
6
7
8
9
10
11
DAY OF MONTH
FG (mg/dl) FG (mmol/l)
USL
CL
LSL
SPECIFICATION CHART Diabetes Mellitus
PATTERN: INTERVENTION:
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
60
80
100
120
140
160
180
200
3
4
5
6
7
8
9
10
11
DAY OF MONTH
FG (mg/dl) FG (mmol/l)
USL
CL
LSL
SPECIFICATION CHART Diabetes Mellitus
PATTERN: INTERVENTION:
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
60
80
100
120
140
160
180
200
3
4
5
6
7
8
9
10
11
DAY OF MONTH
FG (mg/dl) FG (mmol/l)
USL
CL
LSL
SPECIFICATION CHART Diabetes Mellitus
PATTERN: INTERVENTION:
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
60
80
100
120
140
160
180
200
3
4
5
6
7
8
9
10
11
DAY OF MONTH
FG (mg/dl) FG (mmol/l)
SPECIFICATION CHART Diabetes Mellitus
PATTERN: INTERVENTION:
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
60
80
100
120
140
160
180
200
3
4
5
6
7
8
9
10
11
DAY OF MONTH
FG (mg/dl) FG (mmol/l)
USL
CL
LSL
Peak Flow in AsthmaRun Chart - Ashtm a
0
50
100
150
200
250
300
350
400
450
500
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
Day
Pe
ak
Flo
w
Peak Flow in AshtmaRun Chart - Ashtm a
0
50
100
150
200
250
300
350
400
450
500
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
Day
Pe
ak
Flo
w
Peak Flow in AsthmaASTHMA - Control Chart (X)
UCL=381.85
LCL=290.65
CEN=336.25
UCL=395.48
LCL=202.02
CEN=298.75
UCL=459.85
LCL=387.65
CEN=423.75
0
50
100
150
200
250
300
350
400
450
500
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
Day
4
OUTCOMES
Doing what you do better
The best sources of clear learningobjectives in clinical medicine are theproblems presented by our own patients.
160 Diabetic Patients
Three Rapid Improvement Cycles
1. Patient self monitoring
2. Improved process of care
3. Use of best medication
DATA COLLECTIONPN DATE FBS HbA1c10251 4/8/1992 132 20.510063 4/8/1992 339 19.410163 4/20/1992 251 10.710075 4/23/1992 368 12.310719 4/23/1992 219 11.310251 5/6/1992 381 15.310025 6/1/1992 92 7.110719 6/2/1992 180 9.910063 6/3/1992 91 15.910248 6/9/1992 378 16.410251 6/10/1992 369 15.510075 6/29/1992 303 13.110163 7/1/1992 256 10.710491 7/9/1992 147 9.610075 7/23/1992 220 11.710248 8/14/1992 149 15.110251 8/26/1992 348 15.410191 9/10/1992 276 15.410719 9/10/1992 184 9.1
POPULATION BASED DATA
1992 - 1994
50 150 250 3500
20
40
60
80
100
0
20
40
60
80
100
BLOOD SUGAR
NU
MB
ER
COUNT : 631 MEDIAN : 168MEAN : 189MODE : 126STDEV : 84
NORMAL RANGE
© Larry V. Staker MD
DIABETES SPEC CHARTPATIENT
60
80
100
120
140
160
180
200
60
80
100
120
140
160
180
200
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
FB
S
MONTH
© Larry V. Staker MD
DIABETES SPEC CHART
KW D M N I D
0
50
100
150
200
250
300
350
400
0
50
100
150
200
250
300
350
400
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
374 219 180 184 182 265 198 190 191 173 153 144 133 143 132 150 136 129 120 111 124 141 120 149 118 141 130 131 128 120 133
FB
S
SPC
© Larry V. Staker MD
DIABETES SPEC CHART
N N D M I D
0
50
100
150
200
250
300
350
400
0
50
100
150
200
250
300
350
400
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
264274225231228166126139148105141 83 136151117116121126101121145122144113 69 122145126159181139
FB
S
SPC
© Larry V. Staker MD
DIABETES SPEC CHART D B D M B I D S
0
50
100
150
200
250
300
350
400
0
50
100
150
200
250
300
350
400
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
219228275252276246133182140176147138110 98 118140110146120128138 92 90 118126109122132103112117
FB
S
SPC
© Larry V. Staker MD
DIABETES SPEC CHART JF D M BIDS
60
80
100
120
140
160
180
200
60
80
100
120
140
160
180
200
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
81 98 110 99 110108120 96 90 91 85 108 99 92 107102 83 92 125 98 102102109 95 116102106 98 80 130113
FB
S
OCTOBER
© Larry V. Staker MD
FBS IN PATIENTS WITH DIABETES
0
50
100
150
200
250
YR 92 93 94 95 96217 189
176 171 166
DISPLAY OF MEAN OF ALL FBS DONE EACH YEAR
0
50
100
150
200
250
© Larry V. Staker MD
HbA1c IN PATIENTS WITH DIABETES
REF RANGE: 4.8% - 7.8% ION EXCHANGE METHOD
NOTE: 1.0% CHANGE IN HbA1c = 30 MG/DL CHANGE IN FBS
DISPLAY OF MEAN OF ALL Hb A1c DONE EACH YEAR
0
2
4
6
8
10
12
YR 92 93 94 95 96
11.5 11.2
9.5 9.4 8.9
0
2
4
6
8
10
12
© Larry V. Staker MD
HYPOTHESIS TESTINGHo: μA = μB or Ho: σA = σB
CYCLE 2 CYCLE 3 CYCLE 4 OUTCOME
Z Z Z Z Z Z Z Z Z93-92 94-93 95-94 96-95 97-96 98-97 99-98 00-99 00-950.999759 1.000000 0.674829 0.518497 1.000000 1.000000 1.000000 0.594465 1.000000
T T T T T T T T T93-92 94-93 95-94 96-95 97-96 98-97 99-98 00-99 00-950.013251 0.000005 0.628359 0.841768 0.000010 0.000205 0.000086 0.863190 0.000116
F F F F F F F F F93-92 94-93 95-94 96-95 97-96 98-97 99-98 00-99 00-950.252622 0.083474 0.529625 0.664346 0.000938 0.003996 0.244735 0.253290 0.000291
Improvement 1995-2000Hold Gains
BASELINE CYCLE 1
DCCT Published NEJM - 1993 Use of DM Spec and Run Charts
Improve DM Care
Use Best Meds
1992 1993 1994 1995 1996 1997 1998 1999 2000
Percent TTG (Yeild) 13.3% 20.4% 38.6% 41.4% 37.7% 59.2% 73.4% 60.8% 63.8%Sigma 0.4 0.7 1.2 1.3 1.2 1.7 2.1 1.8 1.9
N 1000 1000 1000 1000 1000 1000 1000 1000 1000D or DPTO 866.667 795.580 613.757 585.586 623.188 407.692 266.254 391.534 361.963
eRATE or %d or %NTTG 86.7% 79.6% 61.4% 58.6% 62.3% 40.8% 26.6% 39.2% 36.2%Yield Probability 0.00
Number of Projects 9
Nth Root of Yield Probability 0.40Estimated Ave TTG (Yeild) 40.3%
DPMO MILLION DPKO THOUSAND %TTG %NTTG SIGMABASELINE PERFORMANCE
597483 1000000 597.483 1000 40.25% 59.75% 1.3TEN FOLD IMPROVEMENT
59748 1000000 59.748 1000 94.03% 5.97% 3.1
298742 1000000 298.742 1000 70.13% 29.87% 2.0TWO FOLD IMPROVEMENT
This tool calculates SIGMA for multiple clinical outcomes from input of percent treated to goal (TTG). It also allows forcasting or projection of overall DPKO, %TTG, eRATE (%NTTG) and SIGMA for ten fold and two fold improvement.
HbA1c <= 8.0
If you know the percent of patients treated to goal use this tool.
© Larry V. Staker MD
ESTIMATED NORMALIZED TTG and CALCULATION OF OVERALL SIGMAEnter Percent TTG (as a number like 75.5) in fields colored green
EVALUATION OF OVERALL SIX SIGMA PERFORMANCE
1992 1993 1994 1995 1996 1997 1998 1999 2000
Percent TTG (Yeild) 8.1% 6.6% 20.1% 26.1% 22.5% 36.5% 50.2% 36.5% 38.7%Sigma 0.1 0.0 0.7 0.9 0.7 1.2 1.5 1.2 1.2
N 1000 1000 1000 1000 1000 1000 1000 1000 1000D or DPKO 918.519 933.702 798.942 738.739 775.362 634.615 498.452 634.921 613.497
eRATE or %d or %NTTG 91.9% 93.4% 79.9% 73.9% 77.5% 63.5% 49.8% 63.5% 61.3%Yield Probability 0.00
Number of Projects 9
Nth Root of Yield Probability 0.23Estimated Ave TTG (Yeild) 22.8%
DPMO M DPKO K %TTG %NTTG SIGMABASELINE PERFORMANCE
772255 1000000 772.255 1000 22.77% 77.23% 0.8TEN FOLD IMPROVEMENT
77225 1000000 77.225 1000 92.28% 7.72% 2.9
386127 1000000 386.127 1000 61.39% 38.61% 1.8TWO FOLD IMPROVEMENT
This tool calculates SIGMA for multiple clinical outcomes from input of percent treated to goal (TTG). It also allows forcasting or projection of overall DPKO, %TTG, eRATE (%NTTG) and SIGMA for ten fold and two fold improvement.
HbA1c <= 7.0
If you know the percent of patients treated to goal use this tool.
© Larry V. Staker MD
ESTIMATED NORMALIZED TTG and CALCULATION OF OVERALL SIGMAEnter Percent TTG (as a number like 75.5) in fields colored green
EVALUATION OF OVERALL SIX SIGMA PERFORMANCE
5
ACCOUNTABILITY
A PERSONAL GRADING SYSTEM
“HOW AM I DOING?”
The Sigma Metric: for Motorola
Percent DPMO 30.23% 697,672 1
69.15% 308,537 2
93.32% 66,807 3
99.38% 6,210 4
99.977% 233 5
99.9997% 3.4 6
ProcessCapability
Defects perMillion Opportunities
(distribution shifted ±1.5 s )
Increase in Sigmarequires exponentialdefect reduction
Error Free Yield
MANUFACTURING
DPMO
1
10
100
1000
10000
100000
1000000
1 2 3 4 5 6
DPMO
DPMO6976723085376680762102333.4
Lo
g S
cale
Sigma
Exponential Defect Reduction
The Sigma Metric: for Doctors
Percent DPHO GRADE
30.23% 69.7672 1 F
69.15% 30.8537 2 C
93.32% 6.6807 3 A
99.38% 0.6210 4
99.977% 0.0233 5
99.9997% 0.00034 6
ProcessCapability
Defects perHundred Opportunities
(distribution shifted ±1.5 s )
Increase in Sigmarequires exponentialdefect reduction
Error Free Yield or
TTG
CALCULATION OF SIGMA
SIGMA = NORMSINV (1 - ( #defects / #observations)) + 1.5
The “Sigma Metric” allows reliable comparison of improvement
6
$ENSE
MAKING THE BUSINESS CASE
“Breakeven and ROI”
RETURN ON INVESTMENT
A Profitability Ratio
Net Profit (from Profit and Loss Statement)________________________________________
Net Worth (from Balance Sheet)
Return On Investment- the profitability ratio –
value = ?? 0.10 ??
$ (Savings from best care)
$ (Investment in Equipment + Care)_______________________________
Types of Economic Evaluations
• Cost Comparison Analysis
• Cost Benefit Analysis
• Cost Effectiveness Analysis
• Cost Utility Analysis
• Cost Outcomes Analysis
Cost Comparison Analysis
• Comparison of costs and of two or more alternative therapies that have identical outcomes
• Examples– Generic versus brand name
– Different routes of administration
COST COMPARISON ANALYSIS
$800414.0CAD47.8%0.9295.0%19.060.234.7683.0%81.0%Dobutamine ECHO
$400414.0CAD46.1%0.8596.7%29.450.217.3689.0%81.0%Stress ECHO
$1,200414.0CAD29.5%0.4296.3%26.000.106.5086.0%91.0%Dobutamine
$1,200414.0CAD33.9%0.5194.2%16.360.134.0978.0%90.0%Dipyridamole-iv
$1,200414.0CAD40.9%0.6993.3%13.920.173.4875.0%87.0%Dipyridamole-o
$1,200414.0CAD34.6%0.5395.4%20.940.135.2483.0%89.0%Adenosine
$1,200414.0CAD35.7%0.5692.8%12.860.143.2172.0%90.0%SPECT
$1,000414.0CAD34.5%0.5397.5%39.110.139.7891.0%88.0%Thallium
ETT and
414.0CAD76.3%3.2299.4%160.000.8040.0099.5%20.0%>= 2.5
414.0CAD73.0%2.7199.2%132.000.6833.0099.0%33.0%2.0 - 2.49
414.0CAD70.3%2.3798.8%84.000.5921.0098.0%42.0%1.50 - 1.99
414.0CAD61.1%1.5795.9%23.640.395.9189.0%65.0%1.0 - 1.49
414.0CAD42.1%0.7393.7%14.960.183.7477.0%86.0%0.5 - 0.99
$300 NEGNEGPOSPOS ETT and ST Seg ↓
CostTestICD9DxPTProbPTLRPTProbPTLRLR -LR +SPECSENTEST
Incremental Cost EffectivenessPresentation of Results
CA - CB
EOA - EOB
=
Cost for an additionalunit of effectiveness
Example:
CA - CB
EOA - EOB
=
$3194A - $2617B
45.6A - 42.9B
= $214 to gain anadditional unit ofeffectiveness with A
IMPROVING SKILLS OF MANAGING COST
CEvaluation of Cost of Care
TEST SEN SPEC LR + LR - PostTLR PostTProb PostTLR PostTProb Dx ICD9 Cost/TestETT and ST Seg ? POS POS NEG NEG
0.5 - 0.99 86.0% 77.0% 3.74 0.18 5.61 84.9% 0.27 21.4% CAD 414.0 $3001.0 - 1.49 65.0% 89.0% 5.91 0.39 8.86 89.9% 0.59 37.1% CAD 414.0 $300
1.50 - 1.99 42.0% 98.0% 21.00 0.59 31.50 96.9% 0.89 47.0% CAD 414.0 $3002.0 - 2.49 33.0% 99.0% 33.00 0.68 49.50 98.0% 1.02 50.4% CAD 414.0 $300
>= 2.5 20.0% 99.5% 40.00 0.80 60.00 98.4% 1.21 54.7% CAD 414.0 $300ETT and
Thallium 88.0% 91.0% 9.78 0.13 14.67 93.6% 0.20 16.5% CAD 414.0 $1,000SPECT 90.0% 72.0% 3.21 0.14 4.82 82.8% 0.21 17.2% CAD 414.0 $1,200
Adenosine 89.0% 83.0% 5.24 0.13 7.85 88.7% 0.20 16.6% CAD 414.0 $1,200Dipyridamole-o 87.0% 75.0% 3.48 0.17 5.22 83.9% 0.26 20.6% CAD 414.0 $1,200Dipyridamole-iv 90.0% 78.0% 4.09 0.13 6.14 86.0% 0.19 16.1% CAD 414.0 $1,200
Dobutamine 91.0% 86.0% 6.50 0.10 9.75 90.7% 0.16 13.6% CAD 414.0 $1,200Stress ECHO 81.0% 89.0% 7.36 0.21 11.05 91.7% 0.32 24.3% CAD 414.0 $500Dobutamine ECHO 81.0% 83.0% 4.76 0.23 7.15 87.7% 0.34 25.6% CAD 414.0 $800
COST MINIMIZATION and COST EFFECTIVENESS ANALYSES
LR rule: +>25; -<0.25
IMPROVING SKILLS OF MANAGING COST
TOOL FOR EVALUATION OF COST AND EFFECTIVENESS OF TREATMENT TO GOAL (TTG) [Gives Cost of Quality] © Larry V. Staker MDEnter Numbers in Red Boxes:
POPULATION WITH A DISEASE PROPORTION $300 cost visits/yrTTG NTTG n p $1,000 cost Rx/yr
experiment a 140 20 b 160 a+b 87.5% EER or %TTG $500 cost tests/yr
control c 25 135 d 160 c+d 15.6% CER or %NTTG $1,800 TOTALPanel Size Totals: 165 155 320 (a+b)+(c+d) 5.3% Prevalence
3000 a+c b+d0.5 2.7
FormulasSE Ln Ln SE
p1=a/(a+b) 0.026 EER 87.5% 82.4% 92.6% EER Experiment Event Ratep2=c/(c+d) 0.029 CER 15.6% 10.0% 21.3% CER Control Event RateI EER-CER I 0.027 ARR 71.9% 66.6% 77.2% ARR Absolute Risk Reduction1/ARR NNT 1.4 1.3 1.5 NNT Number Needed to TreatEER/CER RR 560.0% 388.8% 806.5% RR Relative Risk
0.186 1.7228 CI95 LnRR 1.358 2.088 CI95 LnRR Natural Log RBI EER-CER I /CER RRR 460.0% 288.8% 706.5% RRR Relative Risk Reduction(a*d)/(c*b) ROR 37.8 20.1 71.2 ROR Relative Odds Ratio
0.323 3.632 CI95 LnOR 2.999 4.266 CI95 LnOR Natural Log OR
COST ANALYSISOUTCOME
$2,504Δ SIGMA: $704
Standard Error Calculation Enumerative Statistical AnalysisCI-95
Cost of Poor Quality
THE SIX SIGMA METHOD
Healthcare Delivery Systems
Larry V. Staker MD, FACP
The Acronym is DMAIC
1
2
3
4
5
DEFINE
MEASURE
ANALYZE
IMPROVE
CONTROL
SIX SIGMA
1a
1b
1c2
3
4
5
PROCESSIMPROVEMENT
DEFINECORE PROCESSES
DEFINEKEY
CUSTOMERS
DEFINECUSTOMER
REQUIREMENTS
MEASURECURRENT
PERFORMANCE
ANALYZE
IMPROVE
CONTROLINTEGRATE
EXPANDP
D
C
A
SIX SIGMA
hearingseeing
and measuring
The Voice Of The CustomerVOC
SIPOC - Process Analysis
SUPPLIERS
CUSTOMERS
OutputsInputs PROCESS
SIX SIGMA: Minimize Defects
Percent DPMO 30.23% 697,672 1
69.15% 308,537 2
93.32% 66,807 3
99.38% 6,210 4
99.977% 233 5
99.9997% 3.4 6
ProcessCapability
Defects perMillion Opportunities
(distribution shifted ±1.5 s )
Increase in Sigmarequires exponentialdefect reduction
Error Free Yield
The Sigma Metric: for Doctors
Percent DPHO GRADE
30.23% 69.7672 1 F
69.15% 30.8537 2 C
93.32% 6.6807 3 A
99.38% 0.6210 4
99.977% 0.0233 5
99.9997% 0.00034 6
ProcessCapability
Defects perHundred Opportunities
(distribution shifted ±1.5 s )
Increase in Sigmarequires exponentialdefect reduction
Error Free Yield or
TTG
SIX SIGMA: Customer ServiceThe process shown here is stable.
But why does it need to be improved?
} CustomerNeed
Time
LSL
USL
UCL
LCL
SIX SIGMA
DATA DRIVEN
Voice Of The CustomerDefects Reduction Error Free Yield
VOCSigma Metric
REFERENCES
Staker LV. Practice Based Learning For Improvement: The pursuit of clinical excellence. Texas Medicine; V96, N10, Oct 2000, page 53-60.
Staker LV. Changing Clinical Practice by Improving Systems: The pursuit of clinical excellence through practice-based measurement for learning and improvement. QualityManagement in Health Care; V9, N1, Fall 2000, page 1-13.
REFERENCE
Carey, Raymond G. Improving Healthcare with Control Charts: Basic and Advanced SPC Methods and Case Studies. ASQ Quality Press, Milwaukee, WI. September, 2002, 194 pages. ISBN 0-87389-562-2
Chapter 10, pages 159-183 by Larry V. Staker MDThe Use of Run Charts and Control Charts in the Improvement of Clinical Practice.