Download - 10 statistical quality control
(c) Gopalaswamy RameshNo part of this presentation can be duplicated without express writtenconsent from the author
Statistical Quality Control
SQC – Session Objectives During this session you would get to:
Characterize what is “statistical” QC, taking from QA vs QC
Briefly revise the basic concepts of statistics Appreciate the attributes of an effective
measurement system / metrics (essential part of SQC)
Know some of the common pitfalls to watch out for while establishing a Metrics Program
Look at some of the common metrics for different project types
Look at the SQC Tools
2
References MGSP – Chapter 5 Statistics for Management – Levin and
Rubin; PHI EEE – Chapter 10
3
QA, QC and SQC
4
Quality Assurance Vs Quality Control
Quality Assurance Process Oriented Prevention Oriented Proactive Everybody’s Responsibility
Quality Control Pertains to Product Detection Oriented Reactive “Tester/Reviewer’s” Responsibility
5
Statistical Quality Control Basic Premises:
Variability is the opposite of quality Perfect non-variability is not possible causing
“randomness”) Variability (“randomness) has to be “under
control” Variation is of two types
Random variation caused by “general causes” Systemic variation caused by “special causes”
Variation characterized by a random variable having two attributes Mean Standard Deviation
6
Metrics
“ When you can measure what you are speakingabout, you know something about it; but when youcannot measure it, when you cannot express it innumbers, your knowledge is of a meager andunsatisfactory kind; it may be the beginning ofknowledge, but you have scarcely in your thoughtadvanced to the stage of science”
-- Lord Kelvin (of Kelvin Temperature fame!)
7
Importance of Metrics
“If you don’t know where you are going, any road would do” - A Chinese Proverb
“If you don’t know where you are, a map won’t help” Watts Humphrey
8
Let us take a real life example!
Metrics – Topics Importance of Metrics Metrics Roadmap A typical Metrics strategy People and Organizational Issues in
Metrics Common Pitfalls to watch out for
9
Metrics Roadmap
10
“Where do we want to go?”
“What do we have to do?”
“How do we measure progress?”
“Where are we today?”
Metrics Strategy
Decide what you want to measure Set Targets and track them Understand and try to minimize
variability Act on data! Consider the human angle
11
What to Measure?
12
Measure things that make an impact on the company goals
Decide What You Want To Measure Is the Metric tied to company vision? Is it natural? Is it controllable? Is it measurable? Do you know the cost of measuring? Do you know the benefits of measuring? Can you afford not to measure it?
13
End Goal Metrics and In-process Metrics14
Customer encounters a problem
Customer gets aresolution
ProblemReporting
ProblemTo
Developers
ProblemResolution
Fix Distribution
tbte
t1 t2 t3 t4 t5
End goal Metric: te - tb
In-process Metrics: t2 – t1 (Reporting Time), t3 – t2 (Initial Analysis Time), t4 – t3 (Resolution Time), t5 – t4 (Distribution Time),
Dictated by Market
Controlled internally
Set Targets and Track Them .. SMART Metrics
S pecific M easurable A ggressive yet Achievable R esults-oriented T ime-bound
15
Understand and Minimize Variability: Use of Control Charts or x- charts
16
UCL
LCL
Mean
Characteristics of a Process Mean
Average; measure of expected value Upper Control Limit
Maximum value below which a specified % of observations will lie
Lower Control Limit Minimum value above which a specified % of
observations will lie UCL and LCL are fixed statistically to
start with (typically µ+3σ and µ-3σ) and refined later
17
Case 1: All observations hovering within UCL-LCL band
0
20
40
60
80
100
1 2 3 4 5 6 7 8 9 10
Engineer
Lin
es o
f co
de
/ d
ay
Value
UCL
LCL
Mean
18
Case 2: Some observations outside UCL-LCL band
020406080
100120140
1 2 3 4 5 6 7 8 9 10
Engineer
Lin
es o
f co
de
/ d
ay
Value
UCL
LCL
Mean
19
Case 3: Most observations outside UCL-LCL band
020406080
100120140
1 2 3 4 5 6 7 8 9 10
Engineer
Lin
es o
f co
de
/ d
ay
Value
UCL
LCL
Mean
20
Case 4: Observations within UCL-LCL band but with a clear trend
0
20
40
60
80
100
1 2 3 4 5 6 7 8 9 10
Engineer
Lin
es o
f co
de
/ d
ay
Value
UCL
LCL
Mean
21
Case 5: "Hugging the Limits"
0
20
40
60
80
100
1 2 3 4 5 6 7 8 9 10
Engineer
Lin
es o
f co
de
/ d
ay
Value
UCL
LCL
Mean
22
Act on Data!
Analyze the Data and ask “So What”/”What If”/”Why Not”
Convert Data to Information! Analyze Root Causes Process Change? Product Change? Any Metrics to be added or dropped? Complete the feedback loop to the
people who gave the raw data
23
REMEMBER
• WHAT YOU CANNOT MEASURE, YOU CANNOT MANAGE
• STORY OF THE THREE BLIND MEN AND THE ELEPHANT
24
Management Issues in Metrics Management Commitment Top Down Vs Bottom Up Not shooting the messenger Aggregate Results Vs Individual
Performance Ensuring operational issues are
addressed
25
Common Pitfalls in Metrics “Cool Tool Syndrome” Metrics for appraisals Not being flexible Not clarifying operational responsibilities Not having a common understanding on the
significance of the metrics collected Viewing metrics as a “policing” activity Perception of metrics as a bureaucratic
activity Not closing the feedback loop fast enough
26
Common Metrics by Project Types:Development Projects
Requirement Stability Index Defect Density by module Defect Distribution by cause / severity Defect Containment Efficiency Rework …
27
Common Metrics by Project Types:Testing Projects Defect Find Rate Defect Yield of Tests Code Coverage (and its variants) Defects by phase of injection / phase of
detection
28
Common Metrics by Project Types:Maintenance Projects Defect Root Cause Classification
By Severity By Function / Module / product
Defect Frequency By Severity By Function / Module / product
Defect Age Mean Time To Repair
Defect Density By Module or by LOC
29
Common Infrastructure Metrics HR Metrics
Mean Time to fill a vacancy Conversion rate of offers joins Retention metrics Interviewing effort
Training Metrics Training program planned vs. conducted Attendees planned vs. actual Satisfaction levels and trends
System Administration Metrics Server uptime Link uptime MTTR Service calls by priority / cause
30
Common “Process” Metrics Across Project Types Effort Variance Effort Distribution Schedule Variance Productivity Metrics Review Effectiveness
31
Other SQC Tools Pareto Chart
80-20 rule Identifies major factors to be addressed to
get the maximum “bang for the buck” Fish Bone Diagram
Identifies root causes for each anomaly
32
Example of Pareto Chart(Maintenance Book and also Stats Book pg 541)
33
% of Defects By Category
01020304050
Me
mo
ry e
rro
rs
Po
inte
r e
rro
rs
Bo
un
da
ryco
nd
itio
ns
Mis
cella
ne
ou
s
De
sig
n e
rro
rs
% o
f D
efe
cts
% of Total
34
Delay in Responding toCustomer bugs
Delay inGetting Right Info from customer
Delay inRouting theProblem correctly
Delay inResolving The bugs
Delay due toExternal factors
Inadequately TrainedSupport Analysts
Sub-optimal WorkRouting
Unclear NormsFor Routing bugsTo correct person
Too many productsAnd shuttling acrossSupport Analysts
Inadequately Traineddevelopers
Poor Documentation /Training
High Turnover
Code difficult to maintain
Not well architected,High cohesion amongModules
Poor Documentation
UnresponsiveCustomers
Right Info NotAsked From Customers
Lack ofAutomation
Lack of a Reproducible test case
Multiple Vendors Involved
Interfaces notWell defined No agreed upon SLAs
New Hardware / SoftwareConfigurations not supported
Metrics – Topics Re-cap Importance of Metrics Metrics Roadmap A typical Metrics strategy People and Organizational Issues in
Metrics Common Pitfalls to watch out for
35
SQC – Session Objectives Recap During this session you would get to:
Characterize what is “statistical” QC, taking from QA vs QC
Briefly revise the basic concepts of statistics Appreciate the attributes of an effective
measurement system / metrics (essential part of SQC)
Know some of the common pitfalls to watch out for while establishing a Metrics Program
Look at some of the common metrics for different project types
Look at the SQC Tools
36
