“How to Measure the Impact of Specific Development Practices

on Fielded Defect Density”by Ann Marie Neufelder

Presented by: Feride Padgett

Outline

• Introduction

• Results

• Conclusion

Introduction

• The author evaluated 17 organizations based on a questionnaire of 177 parameters

• The 17 organizations were:– Defense systems(5)– Process control manufacturing equipment(11)– Mathematical software(1)

The Questionnaire

• It had 5 sections:– Organization commitment– Life cycle practices– Product characteristics– Change control practices– Informational questions

Scoring

• Mathematical correlation of observed practices vs. observed defect density in terms of KSLOC of assembler

• A correlation of -1 means the practice perfectly correlates to LOWER defect density

High Scorers

• The common features of organizations with the lowest defect densities:– Software engineering is a part of the process,

not an art form– A well-rounded sound set of development

practices vs. a “single bullet”– Formal and informal reviews of requirements

prior to design and code– Testers are involved in requirements phase

Low Scorers

• The common features of organizations with the highest defect densities:– Lack of software management– Misconception of programmer knows what

customer needs better than the customer– Complete void of requirements definition

phase– Insufficient testing methods

Ten Parameters with the Highest Correlation

• Consistent and documented formal and informal reviews of the software and system requirements prior to design and code -.87

• The language and operating system is well supported by industry-.86

• Existence and use of test beds-.85

Ten Parameters with the Highest Correlation

• Incremental testing (as opposed to “big bang” testing.) -.84

• Scheduled regression testing consistently performed by independent testers -.83

• Defect and failure tracking systems that are used by testers for test plan development-.81

Ten Parameters with the Highest Correlation

• There is a defined life cycle model that best suits the application, market, ad organization-.81

• Testers involved during requirements and design. Test plans are started during the requirements phase -.76

Ten Parameters with the Highest Correlation

• Automated unit testing tools-.76

• Explicit test cases for user documentation

-.76

Reviews

• Requirements reviews-.87

• Design reviews-.65

• Coding reviews-.12

Results

• Software managers and engineers often assume engineering practices that decrease defects will also increase development time

• Her study shows that this assumption is not true

• Organizations with the best practices had the highest probability of making their schedules

Results

• Some companies were striving for more releases while their customer surveys showed that their customer wanted fewer releases with higher quality

• More corrective action releases mean more downtime for customers

Results

• This correlation can and is used to:

• Predict defect density

• Improve software development practices for the best return on investment

Conclusion

• This is an ongoing work. New data is added on a yearly basis

• The study shows that making software engineering principles a part of the process decreases the defect density and increases the probability of on time delivery