COMPARING THE DEFECT REDUCTION BENEFITS OF CODE INSPECTION AND TEST-DRIVEN
DEVELOPMENT
By Veronika Movagharianpourand Adam Brakel
IntroductionSoftware Developers face challenges:
Producing high-quality software with low-defect levels while doing it on-time and on-budget
4 Cases to ExploreCase 1: No Code Inspections or Test Driven Development
Case 2: Code Inspection Only
Case 3: Test Driven Development
Case 4: Code Inspection and Test Driven Development
No Code Inspections and NoTest Driven Development
We know from our reading and other student’s reports that programs will have bugs.
That the everyday debugging that occurs does not catch all of them.
So NOT testing in addition to the minimal amount carried out is just a poor design choice!
Code Inspection
We all know what the process for Code Inspections is…
Error Detection Efficiency of Software Inspection
Defect yield rates for SI ranges between 66 and 82 percent where the total number of defects prior to inspection is:
The Defect Detection Yield was found to be:
Test Driven Development
Writing automated unit tests before code Subsequent coding completed when all new
and previous tests succeed Process completed iteratively until software
is complete Successful unit tests execution verified after
write code and refactor steps
The Process of TDD Identifies and removes defects at the
introduced point in the development process
Early defect elimination is the TDD most significant benefit
Leads to huge cost saving due to early defect removal
An Example of Test Driven Development
Example Explanation –
Run Demo -
Purpose & Research Questions: 1. Which software defect reduction method
is most effective at reducing software defects?
H1:Code inspection is more effective than TDD at reducing software defects. 2. Are there interaction effects associated
with the combined use of these methods?H2: The combined use of code inspection and TDD is more effective than either method alone. 3. What are the relative costs of these
software defect reduction methods?H3:Code Inspection and TDD differ in implementation cost
Research Method
Programming assignment: creating part of a spam filter using Java
554 non-commentary source statements on average
261 NCSS provided in a starter project
Participants Excluded by Group:
Participants: undergraduate computer science students With good Java programming and object-
oriented design knowledge Highly motivated to perform well
Experimental Procedures: All Participants were trained in the
Inspection Process and TDD using JUnit. They were then given a detailed
description of the spam filter they were to code and were given two weeks to do so.
Then the resulting code from each of the four groups was analyzed.
Measurements:
Measurement explanation: - Using “yield” to measure selected method effectiveness- Yield measure is not reliably for TDD
number of defects found by using the method Yield = --------------------------------------------------------------------Total Number of defects prior to inspection
Cost of Testing Methods: The total cost is the sum of three areas of
activity; 1. the original development hours by the code
author. 2. the hours spent by software inspectors and
moderator (preparation time + meeting time). 3. hours spent by the author correcting
defects by inspection. TDD seems to save the implementation
cost Inspection had the highest
implementation cost
Implementation Cost:
Threats to Validity: Internal:
Section bias: possibility of unequal participants division
Mortality bias: unequal number of members in each group
Maturation bias: unequal participants learning rates during experiment
Order bias: vulnerability to order bias caused by performed inspections order and whether they were performed on the first or second inspection day
Implementation bias: variability in the treatment conditions implementing or applying means
Threats to Validity (continue)
External: Participants were undergraduate junior or senior
students, not professional programmers Code inspectors were also college students
without code inspection experience Code inspection process did not follow the
normal regulations. Authors did not participate the inspections The iterative cycle of rework, follow-up, and reinspection
did not performed Inspectors performed multiple inspections of
code performing the same function in a short time resulting in finding more defects in the later inspections
Results:
Implementation Cost Summaries:
Summary of Results
Summary and Conclusions: Compare of defect rates and implementation
costs associated with code inspection and test-driven development software defect reduction methods: Inspection:
More effective at reducing defects More expensive to implement
TDD May result in implementation cost savings
More research is require to validate this findings
¿¿¿ Questions ???