identify and analyze possible causes (x’s) identify and
TRANSCRIPT
1
Identify and analyze possible causes (X’s) for the undesirable output
Identify and understand which of the possible causes (X’s) are the biggest contributors to the undesirable output
Identify which causes (X’s) are within the team’s control and those outside their control
Identify methods to verify the suspected big causes (X’s)
2
Identify what data should be collected to validate the suspected big causes (X’s)
Identify and perform appropriate statistical tests to confirm suspected big causes (X’s)
Determine team commitment to improvement targets for the big causes (X’s)
Review and amend Cost of Poor Quality (COPQ) estimates
Develop class project Analyze phase presentation
3
Ishikawa (Fishbone Diagrams)
5 Why’s
Failure Mode and Effects Analysis (FMEA)
Charts / Plots (Box & Whisker)
Correlation and Regression Analysis (SLR)
Hypothesis Testing
4
Process Flow Analysis
Brainstorming
Pareto Charts
Check Sheets
Capability Analysis
Control Charts
Design of Experiments
Gap Analysis
Waste Analysis
Cost of Poor Quality
5
Phase Objectives Key Activities Possible Tools and Techniques Key Deliverables Document the problem statement and establish the charter. Demonstrate alignment with the Business metrics and strategies. Determine Customer requirements and performance standards.
▪ Select Team with Champion
▪ Develop problem statement
▪ Develop Charter ▪ Create SIPOC ▪ Address gap between VOC
and process ▪ Estimate financial benefits
▪ Problem Statements ▪ Project Charter ▪ SIPOC map ▪ COPQ or CODND ▪ Communication Plan
Develop a reliable and valid measurement system of the business process to effectively evaluate the success of meeting customer requirements.
▪ Obtain Customer requirements
▪ Create overall project plan ▪ Develop measurement
plan & compile project metrics
▪ Determine defect tracking requirements
▪ Assess baseline performance-estimate process capability
▪ Measurement Systems Analysis
▪ Process Description ▪ Project Plan & Timeline ▪ Metrics and collection plan ▪ Baseline Performance results ▪ Process capability analysis ▪ Lean Tools Assessment ▪ Measurement Systems
Analysis ▪ Process model – ‘as is’
Utilization of data techniques to gain insight into process. Divide data into groups based on key characteristics and assess the root causes of errors and poor performance. Determine where to focus efforts for improvement.
▪ Statistical tests / tools ▪ FMEA ▪ Pareto chart ▪ Correlation/Regression ▪ Fishbone Diagram ▪ Box plot ▪ Hypothesis Testing
▪ Describe findings – identify potential root causes RCA
▪ Validate findings
▪ Data relationships ▪ Validated Key Input Variables
(KPIVs) & Key Output Variables (KPOVs)
▪ Prioritize sources of variation ▪ root causes
▪ Identify & communicate potential improvements
▪ Summarize benefits & annualized financial benefits
Identify key change opportunities and proactively test for optimization. Develop implementation and communication plan including a change management approach to assist the organization in adaptation of the improvements.
▪ Design of Experiments – describe purpose & build test/ analysis strategy
▪ Evaluate and Confirm results
▪ Analyze KPIVs ▪ Create action plan for
implementation including change management and communication needs
▪ Buy-in assessment
▪ Quantified relationship between key
input and key output variables ▪ Defined process improvements
including impacts and benefits ▪ Implementation Plan ▪ Process model – ‘Should be’ ▪ Impacted Employees are Trained
Definition of optimal process settings and conditions with specified metrics. Implementation of improvements with a control plan to assess & maintain gains.
▪ Implement improvements ▪ Evaluate results ▪ Integrate & manage
improvements in work processes
▪ Complete closure activities
▪ Document process change ▪ Control plan ▪ Determine new process capability ▪ Leverage opportunities for replication ▪ Communicate results ▪ Financial audit ▪ Hand-off to process owner
1.0 Define
Opportunity
2.0 Measure
performance
3.0 Analyze
Opportunity
4.0 Improve
Performance
5.0 Control
Performance
6
Six Sigma Process Improvement Road Map
1
Migration e-Pro Process ImprovementProject Charter
Project Description Error corrections and clarification of benefits are generatingrework throughout the migration and case installationprocesses, accounting for 20% of the total number of e-Prochange transactions. It is estimated that the volume of errorand rework will grow proportionally as the number ofaccounts migrating by 1/1/2004 increases, driving aproportionate increase in cost and potentially dissatisfyingcustomers.
Start Date April 1, 2003
Completion DateScheduled to be completed by September 5, 2003
Baseline Metrics For 1/1/03 migrated accounts:National Accounts- Average number of change transactions: 14.3, of which
2.9 are due to error and rework- Average hours of rework: 309 hoursRegional Accounts- Average number of change transactions: 8.0, of which
1.6 are due to error and rework- Average hours of rework: 137 hours
Primary Metrics 1. Total e-Pro change transactions2. Percentage of change transactions due to error and
benefits clarification3. Average rework hours per error and benefits clarification
Secondary Metrics none
Goal Reduce error and rework in the migration process by 50%starting with 1/1/04 migrating accounts
Customer Customer migration survey results
Financial CODND (Cost of Doing Nothing Differently)4
th Qtr 2003: $500K
Year 2004: $2.5M
Ben
efits
Internal Productivity Estimated cycle time reduction of 18,868 hours (assuming195 accounts migrating 1/1/04).
Define April 1 – April 21, 2003
Plan Projects & Metrics April 14 – April 18, 2003
Baseline Project April 21 – May 2
Consider Lean Tools May12 – May 16, 2003
MSA May 19 – June 2, 2003
Wisdom of the Org. June 2 – June 6, 2003
Passive Analysis June 9 – June 20, 2003
Proactive Testing June 23 – August 4, 2003
Ph
as
e M
ilesto
nes
Control August 4 – September 5, 2003
SUPPLIER INPUT PROCESS OUTPUT CUSTOMER
Sales
Client / Policy HolderHR BenefitsCoordinator
Client Consultant
Third Party BenefitsVendor
Member
GO Decision
Policy Renewal Date
Summary of Benefits
AdministrativeRequirements
AccountOrganizational
Structure
Detail Benefits
Account DataLoaded in System
Member andDependentEligibilityInformationLoaded in System
Member ID Card
Client / PolicyHolder
Third Party BenefitsVendor
Member andDependent
Providers
Claim
Call
1. Conduct migration analysis
2. Complete account profile
3. Load account structure in system
4. Set up and validate account benefits in system
5. Produce account eligibility record
6. Load account data in product claim engines
0Subgroup 10 20 30 40
0
10
20
30
Ind
ivid
ua
l V
alu
e
Mean=10.98
UCL=26.81
LCL=-4.854
0
10
20
Mo
vin
g R
an
ge
1
R=5.952
UCL=19.45
LCL=0
Total e-Pro Change Transactions by Account from Sep 2002 thru Mar 2003Conduct
Analysis
Create Implementation
Guide
Expert Team
Meeting
Draft
EPRO
Draft
e-
PRO
Release e-PRO
Record
e-
PRO
Impl.
Guid
e
Update
EPRO
OK For
Release
to Vendor
Track
Systems
Loads
IMPLEMENTATION
ERW
From
Eligibilit
y
GO
Decisio
n
SALE
S
Set Up Client ID in
End State
Structure
Request
Codes
STRUCTURE
Complete
Structure
Inspection/Verify
with e-PRO
Get
Underwriting
Approval
Yes
No OK? Yes
No
Go back to
Rates
Structur
e in
CDB
No
Yes
Release
ATC
To Vendor
OK?
Review
Draft e-
PRORequest/Receive
Codes
Enrollmen
t File
CLIENT /CUSTOMERClient
Input
To Sales
Run Legislative
Tool
Check vs. e-
PRO
e-PRO
Redo?
Yes
No
YesOK?
Review
Draft
e-PROCreate
Codes
Legislative Tool
Review
BPC &
Class
Codes
To
Structure
BENEFITS
TS
ID Claim
Scenarios
Load Data into
Downstream
Systems
OK?Yes
No
Data
Engines
Loaded
(e.g ATC,
DocGen,
etc)
Test Scenarios
Check vs. e-
PRO
OK?No
Yes
Fix Claim
Errors
No
No
VOB
Yes
EPRO
Rework
CDB
From
Vendor
Member
cancelle
d
in
Legacy
Reformat
Client
Eligibility DataReview
Draft
EPRO
ELIGIBILITY
Receive Enrollment
Data
Match &
Merge
Load data in
CEO
Are
errors
resolve
d?
Fix Errors
YesNo Cancel
Member in
Legacy
Create
ERW
ERW
Eligibility
In CED
VENDO
R(ID
CARDS)
ID
Card
s
From
Benefit
s
Get
Underwriting
Approval
ERW To
Implementatio
n
ERW To
Implementatio
n
Create Client IDClient
ID
To
Structure,
Benefits,
and
Eligibility
Migration
Structure
Mapping Job
Aid
GO
Decision
To
Structure,
Benefits,
and
Eligibility
e-PRO
Rework
EPRO
Rework?
SMT linking
legacy
structure to
end state
To
Eligibility
CAIP
Processes shaded in green are specific to
migration
Processes out of scope, but critical to
Employer Services
Employer Services functional areas
OUT OF SCOPE
PROCESS STEPS
Production
Migration
Support
Cancel
Legacy
Structure
Elig.
Rework
e-Pro
Rework
Rework Loops highlighted in Red
10
100
50
0
Contracting
T4
-T1
PMHS
Overpayment
Process Data / Materials
PeopleTechnology
Ÿ Auth / Referral Info missing/incomplete/incorrect
Ÿ OI Info missing/incomplete/incorrect
Ÿ Member Eligibility Info missing/incomplete/
incorrect
Ÿ Benefit Info missing/incomplete/incorrect
Ÿ Provider Fee Schedule Info missing/incomplete/
incorrect
Ÿ Provider/Vendor TIN/SSN Info missing/
incomplete/incorrect
Ÿ Additional Information Necessary to Process
Claim
Ÿ Transaction/Codeset data excluded at gateway
Ÿ Standard Operating Procedures (SOPs)
Ÿ Claim Audit Process >$5K
Ÿ Second/Third Party Internal Review
(Medical Management, Claim Benefit
Build)
Ÿ iTrack - drives usage of paper reports to
sort older claims
Ÿ Skill Level of Processor
Ÿ Accessiblity of Site Coach/Training Staff
Ÿ Aggressive Productivity goals conflict with low
quality requirements
Ÿ Rushed Training Schedule
Ÿ Lack of up-training / reinforcement training
Ÿ Best Practice / Skill Training not conducted
Ÿ OJT training on SOP usage
Ÿ System Error During Processing
Ÿ Data Fallout
Ÿ Aurhorization Mis-Match
Ÿ System Restrictions - LPI Manual Calc
Ÿ Data Set Up Issues (eligibility, provider, benefits)
Ÿ Timeliness of Batch Processing
Ÿ Bank Acct Set-Up Delays
Ÿ Customer Touchpoints Delays
Ÿ Inappropriate assignment or missing hold codes
Ÿ Provider Mis-Match
Ÿ Transaction Limitations on data collected at
gateway
Manual Adjudication &
PMHS Provider Selection
- Manual
End
Manually check
provider/ vendor
on claim system
Check provider
data and claim
data against iView
image
Mismatch?Manually try to find
correct data
Found data
Correct data &
verify COB
Service request to
appropriate area
YES
NO
YES
iTrack
Verify in claim
screen and follow
COB Checklist
Attempt to
adjudicate claim
NO
Claim processed
Hold codes that
require further
research
NO
Re-open the claim
YES
Process will
depend on Hold
Code & SOP/ Job
Aid
CIRF
Attempt to resolve
all Hold Codes at a
line level
Resolve service
requests
Adjudicate claim
(manual or
systematic)
End
ID Task Name
1 DEFINE PHASE
12 MEASURE PHASE
13 Plan Project and Metrics
22 Baseline the Project
23 Select KPOV metric to track process output
24 Estimate process capability/performance at the 30,000-foot-level
25 Categorical failures
26 Create pareto chart
27 Rescope project to a large Pareto category
28 Repeat Baseline the Project steps 23 through 27
29 Non categorical failures
30 Revise estimate for COPQ/CODND
31 Project status update w ith executive sponsor
32 Consider Lean Tools
39 Conduct Measurement Systems Analysis (MSA)
40 Ensure data integrity
41 Perform Gauge R&R
42 Improve gauge
43 Project status update w ith executive sponsor
44 Wisdom of the Organization
55 ANALYZE PHASE
56 Use visualization of data techniques to gain insight to processes
57 Conduct inferential statistical tests and confidence interval calculations on individual KPOVs
58 Conduct appropriate sample size calculations
59 Conduct hypothesis tests
60 Describe statistical f indings to others using visualization of data techniques
61 Implement agreed-to process improvement findings
62 Project status update w ith executive sponsor
63 IMPROVE PHASE
65 d13 d
18 d
18 d
18 d
20 d
20 d
20 d
23 d
23 d
23 d
27 d
76 d33 d
69 d38 d
38 d
40 d
43 d
69 d48 d
61 d48 d
53 d
60 d53 d
60 d53 d
60 d53 d
60 d53 d
53 d
58 d
62 d
63 d
02 09 16 23 30 06 13 20 27 04 11 18 25 01 08 15 22 29 06 13 20 27 03 10 17 24 31 07 14 21 28 05
March April May June July August September Octob
PMHS
# of Audits
7,321
04/05/2003
# $'s
Under 300 4% Under 1,030,680$
Over 676 9% Over 2,303,562$
No $ Error 914 12% No $ Error -$
No Error 5,431 74% No Error -$
7,321 3,334,243$
A)SEVERITY B)OCCURRENCE
Probability
C)DETECTION
Probability
RISK
PRIORITY
NUMBER ACTION TO IMPROVE
Rate 1-10 Rate 1-10 Rate 1-10 RPN
10=Most
Severe
10=Highest
Probability
10=Lowest
Probability AxBxC A B C
Provider Mis-Match 10 8 9 720
Provider Data Incorrect/Incomplete 9 8 9 648
Data Fallout 9 6 10 540
Data Set Up Issues 9 6 10 540
Provider Fee Schedule Unclear 9 6 9 486
OI Information Needed 9 6 6 324
System Restrictions 6 6 9 324
Hold Codes 9 3 10 270
FAILURE MODE
Process Name: PMHS Claim Processing
Date: 6/30/2003 Revision Level: 3
REVISED VALUES
0Subgroup 10 20 30 40
0
10
20
30
In
div
idu
al
Valu
e
Mean=10.98
UCL=26.81
LCL=-4.854
0
10
20
Mo
vin
g R
an
ge
1
R=5.952
UCL=19.45
LCL=0
Total e-Pro Change Transactions by Account from Sep 2002 thru Mar 2003
Gage R&R http://www.aiag.org/ Part Number http://www.qimacros.com/free-lean-six-sigma-tips/aiag-msa-gage-r&r.html
Average & Range Method 1 2 3 4 5 6 7 8 9 10 Sum
Appraiser 1 Trial 1 0.65 1 3.250
Enter your data here-> Trial2 0.6 1
Trial3
Trial4
Trial 5
Total 1.25 2
Average-Appraiser 10.625 1 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Range1 0.05 0 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Appraiser 2 Trial 1 0.55 1.05 3.100
Enter your data here-> Trial2 0.55 0.95
Trial3
Trial4
Trial 5
Total 1.1 2
Average-Appraiser 20.55 1 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Range2 0 0.1 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Appraiser Trial 1
Enter your data here-> Trial2
Trial3
Trial4
Trial 5
Total
Average-Appraiser 3#N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
Range3 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A
EV (Equipment Variation)0.0332 Equipment Variation (EV)
%EV 11.3% 39.9% # Parts #Trials #Ops % of Total Variation (TV)
AV: (Appraiser Variation)0.02066 2 2 2 Appraiser Variation(AV)
%AV 7.0% 24.8% % of Total Variation (TV)
R&R (Gage Capability) 0.0391 Repeatability and Reproducibility (R&R)
%R&R 13.3% 47.0% NDC 11 % of Total Variation (TV)
PV (Part Variation) 0.2917 Part Variation (PV)
%PV 99.1% 350% % of Total Variation (TV)
Skewness0.41Stdev1.670.20Max6.20
7
There are two different sets of tools for determining the Root Cause of issues in the process◦ Subjective
◦ Analytic
We will cover the most commonly used tools in this module
8
What is a ‘Root Cause Analysis’?
◦ Reactive assessment of basic or contributing causal factors associated with a specific event
In English – what was the real “cause” of the issue!
◦ Analysis focused primarily on system and process issues rather than assigning individual responsibility
9
A method used to help drill down into the process steps to determine the basic causal factors associated with each failure mode
Allows the group to obtain root cause information about an event
Uncovering root causes is critical to process improvement.
If left uncovered the team is simply “band-aiding” a problem temporarily
10
Event Occurs
Take Immediate
Action
Enter Event into
Information System
RCA Required?
Complete Root Cause Analysis
Implement Corrective
Action
Evaluate Corrective
ActionAgainst Goals
Subjective Tools
Analytic Tools
Root Cause Statements
11
Often used 1st on the process
Often used to verify subjective tests
Subjective (Soft) Tools◦ Flow Analysis (M)
◦ Ishikawa (Fishbone)
◦ 5 Whys
◦ FMEA
◦ Graphical Analysis
◦ Brainstorming Process (I)
Analytic Tools◦ Pareto (M)
◦ Checksheets (M)
◦ Capability Analysis (M)
◦ Control Charts (M)
◦ Regression Analysis
◦ Analytical Tests (Hypothesis Testing)
◦ Design of Experiments –DOE (I)
12(M) = covered in the Measure Phase; (I) = covered in the Improve Phase
• Also known as Ishikawa fishbone diagram• Visual representation of known causes to a particular
effect
• Allows the team to drill down in a systematic way to identify major contributing KPIV’s
13
An Ishikawa Diagram (a.k.a fishbone diagram)
can be used to map the process input
variables (PIVs) that affect each KPOV.
14
HEALTHCARE PATIENT TREATMENT FLOW
Step 1 Step 2 Step 3Patient
Outcome
Output of
Treatment
Step
Equipment
Materials Environment
People
Methods
Information
HEALTHCARE PATIENT PROCESS VARIATION
Var(Process) = Var(Step 1) + Var(Step 2) + Var (Step 3) + . . .
Var( Treatment Step) =
Var(Methods) + Var(Materials) + Var(Environment)
+ Var(People) + Var(Equipment) + Var(Information)
Inspection Time appears
To be Excessive
KPIVs
KPOV
15
Analyze 3 – ISHIKAWA diagram template
As a class, construct a Fishbone Diagram for the Class Scenario◦ You can use sticky notes or use a spreadsheet to
list the factors in the columns
◦ Start by determining the outcome
◦ Fill in the “bones”
Write down all the factors
Don’t debate their relative merits; the purpose is to understand the possible inputs
The class has 5-10 minutes for this exercise
16
For each step in the process ask:◦ What problems occurred during this step?
◦ Why did these problems occur?
If the answer from the first question does not provide the root cause, keep asking “why?” until the root cause is reached.
17
Patient leavesHospital bed to go to
bathroom
Patient slips
and falls
Patient found on floor with broken
hip
Why did the patient leave the bed? To go to the bathroom
Why did the patient go to the bathroom unattended? Patient was not deemed a fall risk, normal precautions taken.
Why was the patient not deemed a fall risk? Fall assessment score was well below threshold.
Why was the patient fall risk below threshold?Fall assessment tool does not address specific medications taken.
18
19
Patient leavesHospital bed to go to
bathroom
Patient slips
and falls
Patient found on floor with broken hip
Why did the patient slip and fall? Patient lost balance on the way to rest room.
Why did the patient lose his balance?Patient was not wearing non-slip socks
Why was the patient not wearing non slip socks?Patient recently admitted, socks not available on floor.
Why were there no non-slip socks on the floor?Par-level was too low, restocking was in progress.
Why did the restocking take so long?Weekend, night shift, low staff coverage
When I fill my gas tank it overflows. Can I determine the root cause?
At work we run out of material. Can I determine the root cause?
20
◦ Used to identify all possible failure modes and their effects on a system
◦ Used to identify critical parameters
◦ An excellent tool for supporting a company’s commitment to continually improve products and services wherever possible
◦ Can focus on a process or the design of a new product
21
Think of this as a priority list;
NOT of the things that can go wrong… BUT the things that have to go right.
2 Types of FMEA• DFMEA – Design FMEA
• PFMEA – Process FMEA
22
Improved product functionality & robustness
Reduced Warranty costs
Reduced day-to-day operations issues
Improved safety of products & implementation process
Reduced business process problems
23
Know it is a living document and needs to be reviewed periodically
Conduct early in an improvement to:◦ Design out failure modes by identifying/removing
root causes
◦ Reduce seriousness of failure if elimination is not possible
◦ Reduce the occurrence of failures
◦ Improve detection of failures
24
Note an input to a design or process ( e.g. process step, KPIV, Cause & effect matrix)
List 2 or 3 ways the input/function can go wrong (a failure)
List at least one effect for each potential failure mode
For each failure mode, list 1 or more causes of input going wrong
For each cause list at least 1 method of preventing or detecting the failure
Enter SOD values
You can use the template DOE Gage R&R FMEA >
Failure Mode Effects Analysis
◦ Tab PFMEA (A)
25
Process Operation:◦ Process step under investigation
Process Failure:◦ Way the process could fail to meet the customers
requirements. Every process parameter failure should be taken into account even it is controlled
26
Process
Operation
Process
FailureEffect SEV Cause OCC Controls DET RPN
Actions
Taken
Effect:◦ Effect of the process failure on the product, process
parameter, or customer Severity (SEV)◦ Rank on a scale of 1 to 10. The highest #10 associated with
a safety concern and the lowest #1 associated with a non-concern.
27
Process
Operation
Process
FailureEffect SEV Cause OCC Controls DET RPN
Actions
Taken
Rating Description Definition (Severity of Effect)
10 Dangerously high Failure could injure the customer or an employee.
9 Extremely high Failure would create noncompliance with federal regulations.
8 Very high Failure renders the unit inoperable or unfit for use.
7 High Failure causes a high degree of customer dissatisfaction.
6 Moderate Failure results in a subsystem or partial malfunction of the product.
5 Low Failure creates enough of a performance loss to cause the customer to complain.
4 Very Low Failure can be overcome with modifications to the customer’s process or product,
but there is minor performance loss.
3 Minor Failure would create a minor nuisance to the customer, but the customer can
overcome it without performance loss.
2 Very Minor Failure may not be readily apparent to the customer, but would have minor effects
on the customer’s process or product.
1 None Failure would not be noticeable to the customer and would not affect the customer’s
process or product.
28
Caution! Severity ranking should NOT be considered low just because its occurrence is low, or because its detection is very effective.
Note: A reduction in SEVERITY rank is normally achieved through a design change to the system/sub-system that uses the device.
29
Cause:◦ How could the failure occur? Is there something that could
be controlled?
Occurrence (OCC):◦ Rank on a scale of 1 to 10, on the basis of the likelihood
that the process failure will occur. Rank of 10 meaning the failure is sure to occur and 1 meaning the failure is unlikely to occur.
30
Process
Operation
Process
FailureEffect SEV Cause OCC Controls DET RPN
Actions
Taken
Rating Description Potential Failure Rate
10 Very High: Failure is
almost inevitable.
More than one occurrence per day or a probability of more than three occurrences
in 10 events (Cpk < 0.33).
9 High: Failures occur
almost as often as
not.
One occurrence every three to four days or a probability of three occurrences in 10
events (Cpk ≈ 0.33).
8 High: Repeated failures. One occurrence per week or a probability of 5 occurrences in 100 events (Cpk ≈
0.67).
7 High: Failures occur often. One occurrence every month or one occurrence in 100 events (Cpk ≈ 0.83).
6 Moderately High:
Frequent failures.
One occurrence every three months or three occurrences in 1,000 events (Cpk ≈
1.00).
5 Moderate: Occasional
failures.
One occurrence every six months to one year or five occurrences in 10,000 events
(Cpk ≈ 1.17).
4 Moderately Low:
Infrequent failures.
One occurrence per year or six occurrences in 100,000 events (Cpk ≈ 1.33).
3 Low: Relatively few
failures.
One occurrence every one to three years or six occurrences in ten million events
(Cpk ≈ 1.67).
2 Low: Failures are few and
far between.
One occurrence every three to five years or 2 occurrences in one billion events
(Cpk ≈ 2.00).
1 Remote: Failure is
unlikely.
One occurrence in greater than five years or less than two occurrences in one
billion events (Cpk > 2.00).
31
Controls:◦ What are the controls that are currently in existence to prevent
process failure from occurring OR to detect the effect of failures.
Detectability (DET):◦ Rank on a scale of 1 to 10, based on the probability that the
process controls will detect the process failure (prevention) or the effect of the process failure (detection).
◦ Rank of 10 indicates that there is absolute certainty of non-detection and 1 means the control is certain to detect the failure.
32
Process
Operation
Process
FailureEffect SEV Cause OCC Controls DET RPN
Actions
Taken
Rating Description Definition
10 Absolute Uncertain No inspection or the defect caused by failure is not detectable.
9 Very Remote Product is sampled, inspected, and released based on Acceptable Quality Level
(AQL) sampling plans.
8 Remote Product is accepted based on ‘no defectives’ in a sample.
7 Very Low Product is 100% manually inspected.
6 Low Product is 100% manually inspected using go/no-go or other mistake-proofing
gauges.
5 Moderate Some Statistical Process Control (SPC) is used in process and product is final
inspected off-line.
4 Moderately High SPC is used and there is immediate reaction to out-of-control conditions.
3 High An effective SPC program is in place with process capabilities (Cpk) greater than
1.33.
2 Very High All product is 100% automatically inspected.
1 Almost Certain The defect is obvious or there is 100% automatic inspection with regular
calibration and preventive maintenance of the inspection equipment.
33
Risk Priority Number (RPN):◦ Quantifies the risk associated with a given process failure
mode.
RPN = Severity (S) x Occurrence (O) x Detection (D)
RPN ranks between 1 and 1000
Caution! Even if the RPN is low, a severity rating of 10 needs to be addressed.
Action:◦ Activity that needs to be initiated due to high risks
identified by the RPN rating. Although there is no rule for a threshold, typically above 125 is considered an actionable level.
34
35
Investigate the process and data to determine the biggest contributors to the undesirable output (Y = f(X))
Find patterns and data to support observations
Separate the vital few from the trivial many KPIV’s
Prove the major contributors with data
36
37
Graphical Analysis reveals apparent signs of process differences leading to potential solutions◦ Example: The Box Plot will show the differences
in variation for multiple groups for data
Statistical Analysis proves statistical differences which can be exploited for finding solutions; graphical analysis is used as a prelude to statistical analysis
3838
You have already seen:◦ Pareto Diagram◦ Run Charts◦ Histograms◦ Control (SPC) Charts
We will introduce:◦ The Box and Whisker Plot
Other tools on QI Macros include:Dot PlotsScatter PlotsMulti-Vari ChartsValues Plot
39
40
-0.006771
0.0132292
0.0332292
0.0532292
0.0732292
0.0932292
0.1132292
3/21/2008 3/22/2008 3/23/2008 3/24/2008 3/25/2008 3/26/2008 3/27/2008 3/28/2008 3/29/2008
Min
ute
s t
o R
esp
ond
Days of Data Collection
Box & Whisker In-House Printing Dept Response Time
• The yellow and aqua areas of each Box and Whisker contain the 2nd and 3rd quartiles of data
• The 1st and 4th quartiles are in the “whiskers”
• The asterisks are outliers (calculated by a formula)
As part of your analysis you try different routes to see if they make a difference graphically
Use the file B&W to generate a Box and Whisker plot◦ Highlight A, B and C
◦ Run Box, Dot & Scatter > Box and Whisker
◦ Select “Columns” on the “Group by…” dialogue box
◦ Click OK on the titles
41
B&W
27.275
32.275
37.275
42.275
47.275
52.275
57.275
62.275
Route 1 Route 2 Route 3
Valu
es
Subgroups
Route 1 - Route 3
What observations can you make?Which appears to be the best route? Why?
42
43
Measures the strength of association between the input and the output
◦ Y = f(x)
The simplest tool for determining the effect on the output based on a change in the input
Based on your high school math◦ Y = mx + b (where “m” is the slope and “b” is
the y-intercept)
44
Follow along using the file SLR to analyze the drive time versus the number of stop lights hit while taking two different routes to work
Open file SLR
Highlight columns A and B (route 1)
Run Box, Dot & Scatter Plot > Scatter
Click OK through the title slides
Repeat on columns D and E (route 2)
45
SLR
y = 1.575x + 45.033
R² = 0.9923
47
49
51
53
55
57
59
1 2 3 4 5 6 7 8 9
Dri
ve T
ime
Stop Lights Hit - Route 1
Drive Time vs Stop Lights Hit - Route 1
Prediction Equation
R2 - The amount of influence the data points have on the shape of the line
46
y = 2.2703x + 41.568
R² = 0.7063
45
47
49
51
53
55
57
59
61
63
65
2 3 4 5 6 7 8 9
Tim
e
Stop Lights Hit - Route 2
Time vs Stop Lights Hit - Route 2
Compare the graphs. What observations can you make?
47
If I hit no stop lights, which has the shortest drive time?
If I assume that I will hit an average of 5 stop lights each day, how long will the trip take for each route?
If I hit 8 stop lights, which route has the shortest drive time?
48
49
Use to statistically determine if there are differences between a sample and a target or between two or more sample groups
Used to determine whether making a change to the input variable will result in changes to an output
Without hypothesis testing teams may make adjustments that are not actually required◦ These knee jerk responses can amplify variation
and cause additional problems
50
In manufacturing, you might want to compare two or more raw materials and determine if they produce the same quality
Hypothesis testing helps identify ways to reduce cost and improve quality
51
Define a null (Ho) and an alternative (Ha) hypothesis◦ Ho = the sample is the same as the target or the samples
are the same
◦ Ha = at least one of the samples are different from the target or other sample(s)
There are hypothesis tests for means, medians, proportions, variance and dependence
The goal is to prove that they are not statistically the same at some level of confidence (usually 95%, 99%)
52
Three types of Hypothesis testing1. Classical Method – comparing a test statistic to
a critical value (very statistically oriented)
2. p value Method - the probability of a test statistic being contrary to the null hypothesis
• If the p value is equal to or greater than the a
value (or level of significance), the null hypothesis is confirmed (remember sample sizing)
3. Confidence Interval Method – is the test statistic between or outside of the confidence interval (used for target values)
53
The p-value is the probability that your conclusion of the null hypothesis is incorrect (e.g. your results are highly unlikely to occur in a real world)◦ Keep in mind: the data is not good or bad, it just
does not fit your hypothesis
◦ You may not have enough data in your sample to prove your original hypothesis
54
Run Normality first on ALL variable data◦ T tests and ANOVA are used for normal data
Note: samples are run separately and each sample set must be normal to run these tests
◦ Non-Parametric tests are used for non-normal data
F tests – run equal variance tests when comparing two or more samples◦ Bartlett's – test for equal variance for normal data
◦ Levene's - test for equal variance for non-normal data
55
Normality (Variable)◦ Testing for normality of the data
F Test (variable)◦ Comparing variances (normal and non-normal)
t-Tests (variable)◦ Used for comparing means
ANOVA◦ Used for comparing more than two means
Non-Parametric (variable)◦ Used to analyze non-normal data
Proportion (attribute)◦ Used to analyze proportions
Chi Squared (attribute)◦ A test of the dependency between input and output◦ Excellent for transactional processes
56
Normality – normality of data◦ Statistical Tools > Descriptive Statistics - Normality Test
Tests for Equal Variance of Multiple Samples◦ Statistical Tools > F Test: Two sample for variance (normal
data)◦ Statistical tools > Levene’s test for variance (non normal
data)
Mean◦ 1 sample t test - Statistical Tools: > t test one sample
Comparing one sample to a target
◦ 2 sample t test - Statistical Tools > T test; two sample assuming equal variances Comparing two samples
◦ ANOVA Single Factor – Statistical Tools > Anova single factor Comparing three or more samples
57
Analyze 4 - Hypothesis Roadmap
Non Parametric (for non normal data)◦ Statistical Tools > Stat Templates > 1 sample sign
Comparing sample data versus target
◦ Statistical Tools > Stat Templates > Mann Whitney
Comparing two samples of data
◦ Statistical Tools > Stat Templates > Kruskal Wallis
Comparing three samples of data
58
Proportion Tests◦ Attribute data
◦ Statistical Tools > 1-2 Proportions Test
Chi Square Test◦ Dependence / Independence of the interaction
between inputs and outputs
◦ Statistical Tools > Chi Squared
59
You want to check and see if two different routes have different drive times.◦ Use columns B and C from the file B&W to
determine if they are statistically different
Go to the Statistical Tools > Descriptive Statistics -Normality Test for means
60
B&W
Route 2 Anderson-Darling Data is Normal
38.0 A-Squared 0.359
41.5 p 0.375
42.0 95% Critical Value 0.787
30.9 99% Critical Value 1.092
Route 3 Anderson-Darling Data is Normal
31.9 A-Squared 0.561
41.1 p 0.109
39.7 95% Critical Value 0.787
41.4 99% Critical Value 1.092
Route 2 Route 3
The p-value is preset to be 5% or 0.05 for the normality test; both p-values are greater than 0.05 so we accept our null hypothesis that the
data is normal
61
◦ Run the proper test for equal variances
Go to Statistical Tools > F Test; Two-sample for variance
If one or both were not normal, you would have run Levene’s test for variance instead
Keep the significance at 0.05 and click OK on the titles
62
F-Test Two-Sample for Variances a 0.05
Route 2 Route 3
Mean 37.38 37.12
Variance 16.64178 72.21067
Observations 10 10
df 9 9
F 0.23
P(F<=f) one-tail 0.020 0.040 Two-tail
F Critical one-tail 3.18 4.03 Two-tail
One-tail Reject Null Hypothesis because p < 0.05 (Variances are Different)
Two-tail Reject Null Hypothesis because p < 0.05 (Variances are Different)
Conclusion: The variances are different
Go to Statistical Tools > t Test: Two-sample assuming unequal variances
63
t-Test: Two-Sample Assuming Unequal Variances a 0.05
Equal Sample Sizes
Route 2 Route 3
Mean 37.38 37.12
Variance 16.64178 72.21067
Observations 10 10
Hypothesized Mean Difference 0
df 13
t Stat 0.087
P(T<=t) one-tail 0.466Cannot Reject Null Hypothesis because p > 0.05 (Means are the same)
T Critical one-tail 1.771
P(T<=t) two-tail 0.932Cannot Reject Null Hypothesis because p > 0.05 (Means are the same)
T Critical Two-tail 2.160
Chi Square testing is used to see if the results are independent or dependent on an input◦ The null hypothesis is that they are independent (p-
value > a)
◦ Chi Square testing is excellent for analyzing survey data
Open file Chi Square
Highlight columns A, B, C and D
Statistical Tools > Chi Squared
64
Chi Square
65
21-40 41-60 60+ Total Chi-Sq 18.31879241
Like 27 12 24 63 p 0.001069037
Don’t Care 35 67 31 133 a 0.05
Hate 13 21 17 51 Variables are Related
Total 75 100 72 247
21-40 41-60 60+ Contribution
Like 3.238126084 7.15178716 1.729451835
Don’t Care 0.717948718 3.213296703 1.556929182
Hate 0.399032574 0.006008573 0.306211576
The highest contribution comes from the interaction that is least expected• This gives you clues about the
population
A p-value less than 0.05 tells you that the
variable and the output ARE dependent!
Your historic defect rate has been 5.4%. You have made some improvements and want to see if that is been reduced. You collect 154 samples and there are 5 defects. You declare success based on your new 3.2% defect rate. Based on this sample, have you really made a difference? ◦ Open a blank Excel Spreadsheet◦ Statistical Tools > 1-2 Proportions Test◦ Choose the tab labeled “One Proportion”
Enter 0.946 (success rate) in the yellow box of column A
Enter 154 in the yellow box of column B (trials)
Enter 149 in the yellow box of column C (successes)
Keep the confidence level at 0.95 (a = 0.05)
66
67
Look for the Red colored boxes in the p (normal) area. The sample is NOT different from the historical average!
It shows the p-values for all three of the Possible scenarios:• Sample (H1) = Historic Proportion (H0)• Sample > Historic Proportion• Sample < Historic Proportion
Note the p values are all greater signifying that the Null Hypothesis is true. If any of the p values were less than 0.05 it would signify a relationship that was not true (e.g. H1<H0)
Try gathering More Data… More data gives a more accurate number
0.95 Confidence Level
Proportion Trials Successes Sample p 95% Confidence Intervals p (Direct) p (Normal)
0.946 154 149 0.967532 0.925860 0.989375 0.154 H1<>H0 0.237 H1<>H0
0.93954 0.995525 0.077 H1>H0 0.119 H1>H0
0.923 H1<H0 0.881 H1<H0
Champion Analyze Phase Checklist
68
Analyze 2 – Tollgate and Approval
Analyze Phase CommentsWhat are your deliverables for this phase? Summarize the findings.
Has your Problem Statement or Objective Statement changed? If yes, why?
Have you completed you Fishbone (Ishikawa) analysis to identify variable in our
process?
How many significant (vital few) variables influence the process and what are they?
What sources of variation have been identified?
Have you started your FMEA?
Have you performed any Root Cause Analysis on your process?
Have you done any Graphical Analysis to identify key input varibles in your
process?
Have you completed any regression analysis?
Have you confirmed any findings using Hypothesis Testing? What are the
conclusions?
What is the potential contribution of each of the vital few variables?
What interim actions have you taken to contain defects until a final solution can be
developed and implemented? Has the FMEA been completed?
What tools have you used in this phase and how were they helpful?
What are your improvement plans (containment actions and long term solutions) and
next steps to get there (including timing, responsibility and expected results)?
Has your COPQ changed?
What are you conclusions from this phase?
Are you on track to meet the scheduled completion date?
Are you satisfied with the level of cooperation and support you are getting?
Have you obtained the signatures from leadership to move on to the next phase?
Project Team Analyze Phase Checklist
69
Analyze Tollgate Approval
Champion Approval Signature/Date:
Tollgate review approved unconditionally:
Tollgate review approved with the following contingencies:
Tollgate review dis-approved, list issues for resolution:
70