10 statistical quality control

of 36/36
(c) Gopalaswamy Ramesh No part of this presentation can be duplicated without express written consent from the author Statistical Quality Control G Ramesh [email protected]

Post on 18-Jun-2015

557 views

Category:

Software

4 download

Embed Size (px)

TRANSCRIPT

  • 1. (c) Gopalaswamy Ramesh No part of this presentation can be duplicated without express written consent from the author Statistical Quality Control G Ramesh [email protected]

2. SQC Session Objectives During this session you would get to: Characterize what is statistical QC, taking from QA vs QC Briefly revise the basic concepts of statistics Appreciate the attributes of an effective measurement system / metrics (essential part of SQC) Know some of the common pitfalls to watch out for while establishing a Metrics Program Look at some of the common metrics for different project types Look at the SQC Tools 2 3. References MGSP Chapter 5 Statistics for Management Levin and Rubin; PHI EEE Chapter 10 3 4. QA, QC and SQC 4 5. Quality Assurance Vs Quality Control Quality Assurance Process Oriented Prevention Oriented Proactive Everybodys Responsibility Quality Control Pertains to Product Detection Oriented Reactive Tester/Reviewers Responsibility 5 6. Statistical Quality Control Basic Premises: Variability is the opposite of quality Perfect non-variability is not possible causing randomness) Variability (randomness) has to be under control Variation is of two types Random variation caused by general causes Systemic variation caused by special causes Variation characterized by a random variable having two attributes Mean Standard Deviation 6 7. Metrics When you can measure what you are speaking about, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thought advanced to the stage of science -- Lord Kelvin (of Kelvin Temperature fame!) 7 8. Importance of Metrics If you dont know where you are going, any road would do - A Chinese Proverb If you dont know where you are, a map wont help Watts Humphrey 8 Let us take a real life example! 9. Metrics Topics Importance of Metrics Metrics Roadmap A typical Metrics strategy People and Organizational Issues in Metrics Common Pitfalls to watch out for 9 10. Metrics Roadmap 10 Where do we want to go? What do we have to do? How do we measure progress? Where are we today? 11. Metrics Strategy Decide what you want to measure Set Targets and track them Understand and try to minimize variability Act on data! Consider the human angle 11 12. What to Measure? 12 Measure things that make an impact on the company goals 13. Decide What You Want To Measure Is the Metric tied to company vision? Is it natural? Is it controllable? Is it measurable? Do you know the cost of measuring? Do you know the benefits of measuring? Can you afford not to measure it? 13 14. End Goal Metrics and In-process Metrics 14 Customer encounters a problem Customer gets a resolution Problem Reporting Problem To Developers Problem Resolution Fix Distribution tb te t1 t2 t3 t4 t5 End goal Metric: te - tb In-process Metrics: t2 t1 (Reporting Time), t3 t2 (Initial Analysis Time), t4 t3 (Resolution Time), t5 t4 (Distribution Time), Dictated by Market Controlled internally 15. Set Targets and Track Them .. SMART Metrics S pecific M easurable A ggressive yet Achievable R esults-oriented T ime-bound 15 16. Understand and Minimize Variability: Use of Control Charts or x- charts 16 UCL LCL Mean 17. Characteristics of a Process Mean Average; measure of expected value Upper Control Limit Maximum value below which a specified % of observations will lie Lower Control Limit Minimum value above which a specified % of observations will lie UCL and LCL are fixed statistically to start with (typically +3 and -3) and refined later 17 18. Case 1: All observations hovering within UCL- LCL band 0 20 40 60 80 100 1 2 3 4 5 6 7 8 9 10 Engineer Linesofcode/day Value UCL LCL Mean 18 19. Case 2: Some observations outside UCL-LCL band 0 20 40 60 80 100 120 140 1 2 3 4 5 6 7 8 9 10 Engineer Linesofcode/day Value UCL LCL Mean 19 20. Case 3: Most observations outside UCL-LCL band 0 20 40 60 80 100 120 140 1 2 3 4 5 6 7 8 9 10 Engineer Linesofcode/day Value UCL LCL Mean 20 21. Case 4: Observations within UCL-LCL band but with a clear trend 0 20 40 60 80 100 1 2 3 4 5 6 7 8 9 10 Engineer Linesofcode/day Value UCL LCL Mean 21 22. Case 5: "Hugging the Limits" 0 20 40 60 80 100 1 2 3 4 5 6 7 8 9 10 Engineer Linesofcode/day Value UCL LCL Mean 22 23. Act on Data! Analyze the Data and ask So What/What If/Why Not Convert Data to Information! Analyze Root Causes Process Change? Product Change? Any Metrics to be added or dropped? Complete the feedback loop to the people who gave the raw data 23 24. REMEMBER WHAT YOU CANNOT MEASURE, YOU CANNOT MANAGE STORY OF THE THREE BLIND MEN AND THE ELEPHANT 24 25. Management Issues in Metrics Management Commitment Top Down Vs Bottom Up Not shooting the messenger Aggregate Results Vs Individual Performance Ensuring operational issues are addressed 25 26. Common Pitfalls in Metrics Cool Tool Syndrome Metrics for appraisals Not being flexible Not clarifying operational responsibilities Not having a common understanding on the significance of the metrics collected Viewing metrics as a policing activity Perception of metrics as a bureaucratic activity Not closing the feedback loop fast enough 26 27. Common Metrics by Project Types: Development Projects Requirement Stability Index Defect Density by module Defect Distribution by cause / severity Defect Containment Efficiency Rework 27 28. Common Metrics by Project Types: Testing Projects Defect Find Rate Defect Yield of Tests Code Coverage (and its variants) Defects by phase of injection / phase of detection 28 29. Common Metrics by Project Types: Maintenance Projects Defect Root Cause Classification By Severity By Function / Module / product Defect Frequency By Severity By Function / Module / product Defect Age Mean Time To Repair Defect Density By Module or by LOC 29 30. Common Infrastructure Metrics HR Metrics Mean Time to fill a vacancy Conversion rate of offers joins Retention metrics Interviewing effort Training Metrics Training program planned vs. conducted Attendees planned vs. actual Satisfaction levels and trends System Administration Metrics Server uptime Link uptime MTTR Service calls by priority / cause 30 31. Common Process Metrics Across Project Types Effort Variance Effort Distribution Schedule Variance Productivity Metrics Review Effectiveness 31 32. Other SQC Tools Pareto Chart 80-20 rule Identifies major factors to be addressed to get the maximum bang for the buck Fish Bone Diagram Identifies root causes for each anomaly 32 33. Example of Pareto Chart (Maintenance Book and also Stats Book pg 541) 33 %of Defects By Category 0 10 20 30 40 50 Memoryerrors Pointererrors Boundary conditions Miscellaneous Designerrors %ofDefects % of Total 34. 34 Delay in Responding to Customer bugs Delay in Getting Right Info from customer Delay in Routing the Problem correctly Delay in Resolving The bugs Delay due to External factors Inadequately Trained Support Analysts Sub-optimal Work Routing Unclear Norms For Routing bugs To correct person Too many products And shuttling across Support Analysts Inadequately Trained developers Poor Documentation / Training High Turnover Code difficult to maintain Not well architected, High cohesion among Modules Poor Documentation Unresponsive Customers Right Info Not Asked From Customers Lack of Automation Lack of a Reproducible test case Multiple Vendors Involved Interfaces not Well defined No agreed upon SLAs New Hardware / Software Configurations not supported 35. Metrics Topics Re-cap Importance of Metrics Metrics Roadmap A typical Metrics strategy People and Organizational Issues in Metrics Common Pitfalls to watch out for 35 36. SQC Session Objectives Recap During this session you would get to: Characterize what is statistical QC, taking from QA vs QC Briefly revise the basic concepts of statistics Appreciate the attributes of an effective measurement system / metrics (essential part of SQC) Know some of the common pitfalls to watch out for while establishing a Metrics Program Look at some of the common metrics for different project types Look at the SQC Tools 36