defect removal effectiveness
DESCRIPTION
Anna University, Final CSE, Software Quality ManagementTRANSCRIPT
Defect Removal EffectivenessDefect Removal EffectivenessSoftware Quality ManagementSoftware Quality Management
Unit 3
G R A A ldG. Roy Antony ArnoldAsst. Professor / CSE
GRAA
f l i f h i• Defect removal is one of the top expenses inany software project and it greatly affectsschedules.
• Effective defect removal can lead toreductions in the development cycle time andgood product quality.g p q y
• It is important for all developmentorganizations to measure the effectiveness oforganizations to measure the effectiveness oftheir defect removal processes.
GRAA
• Fagan (1976) defined error detection efficiency as:Errors found by an Inspection
Total errors in the product before inspectionX 100%
• Jones's definition (1986), stated here, is very similar to Fagan's:– Removal Efficiency = Defects found by removal operationy p
Defects present at removal operationX 100%
Defects found
Defects found + Defects not found (found later)X 100%
• IBM Houston received the first NASA Excellence Award for Quality and Productivity in 1987
GRAA
• One of the four metrics IBM used to manage quality isOne of the four metrics IBM used to manage quality is the early detection percentage, which is actually inspection defect removal effectiveness
Number of major inspection errors
Early Detection Percentage =
Number of major inspection errors
Total number of errorsX 100%
• where total number of errors is the sum of major inspection errors and valid discrepancy reports (di t i th h i f t ki t t(discrepancy report is the mechanism for tracking test defects).
GRAA
GRAA
• The effectiveness measure by Dunn (1987) differs little fromThe effectiveness measure by Dunn (1987) differs little from Fagan's and from Jones's second definition.
• Dunn‘s definition is:
Number of defects found by activity
Effectiveness of activity (development phase ) =
Number of defects found by activity
Number of defects found by subsequent activitiesX 100%
• This metric can be tuned by selecting only defects present at the time of the activity and susceptible to detection by the y p yactivity.
GRAA
• Daskalantonakis (1992) describes the metrics used at Motorola for software developmentsoftware development.
Number of pre‐release defects
Total Defect Containment Effectiveness (TDCE) =
Number of pre‐release defects
Number of pre‐release defects + Number of post‐release defects
Number of Phase i errors
Phase Containment Effectiveness (PCEi) =
• Where phase i errors are problems found during that development
Number of Phase i errors + Number of phase i defects
Where phase i errors are problems found during that development phase in which they were introduced, and
• Phase i defects are problems found later than the development phase in which they were introduced.phase in which they were introduced.
GRAA
( )Defects removed (at the step)
Defects existing on step entry + Defects injected during development of the stepX 100%
GRAA
GRAA
B d i l t d i i d b th D t t• Based on a special study commissioned by the Departmentof Defence, Jones estimates the defect removaleffectiveness for organizations at different levels of the
( )development process capability maturity model (CMM):– Level 1: 85%– Level 2: 89%Level 2: 89%– Level 3: 91%– Level 4: 93%
l– Level 5: 95%• These values can be used as comparison baselines for
organizations to evaluate their relative capability withorganizations to evaluate their relative capability withregard to this important parameter.
GRAA
• Based on historical and recent data from threesoftware engineering organizations at GeneralDynamics Decision Systems, Diaz and King (2002)report that the phase containment effectiveness byCMM level as follows:– Level 2: 25.5%– Level 3: 41.5%– Level 4: 62.3%– Level 5: 87.3%
GRAA
Phase Inserted Cumulative % of DefectsPhase Inserted Cumulative % of Defects removed through Acceptance
TestRequirements 94%Top‐Level Design 95%Top Level Design 95%Detailed Design 96%Code and Unit Test 94%Code and Unit Test 94%Integration Test 75%System Test 70%System Test 70%Acceptance Test 70%
GRAA