abstract a study on developing a software …...i would also like to express my appreciation to...
TRANSCRIPT
i
ABSTRACT
A STUDY ON DEVELOPING A SOFTWARE INSPECTION METHODOLOGY
Demirhan, Taylan
M.S., Computer Engineering Department
Thesis Supervisor : Asst. Prof. Dr. Alok MISHRA
June 2006, 92 pages
In this study, specifications, advantages, disadvantages and significance of the
software inspection methods that are being used in the software development projects
are examined. The elements used for the application of these methods, like checklists
and software are introduced with examples. A research is made on the widely used
methodologies, and its application is studied for projects of different size for
software companies and improvements and modifications are introduced to fasten the
method. In addition, a web based tool is prepared to support the method.
Keywords: Software Inspection Methods, Software Quality Assurance, Computer
Support in Software Inspection
ii
ÖZ
YAZILIM DENETİM METODOLOJİSİ GELİŞTİRİLMESİ
ÜZERİNE BİR ÇALIŞMA
Demirhan, Taylan
Yüsek Lisans, Bilgisayar Mühendisliği Bölümü
Tez Danışmanı : Yrd. Doç. Dr. Alok MISHRA
Haziran 2006, 92 sayfa
Bu çalışmada, yazılımda denetimi sağlamak için geliştirilen ve yazılım sektöründe
kabul görmüş denetim metotlarının özellikleri, avantajları, dezavantajları, ve önemi
incelenmiş, bu metotların uygulanması sırasında kullanılan kontrol listeleri,
programlar gibi argümanlardan, örnekler verilerek incelenmiştir. Buna ek olarak en
çok kullanılan metotlar üzerinde inceleme yapılıp, uygulamada yazılım firmaları için
farklı büyüklükteki projelerde izlenebilecek yöntemler araştırılmış ve yazılım
geliştirme sürecini daha hızlı ve verimli hale getirecek eklemeler ve değişiklikler
önerilerek yeni bir metot geliştirilmiştir. Ayrıca bu metodu desteklemek üzere web
tabanlı bir yazılım geliştirilmiştir.
Anahtar kelimeler: Yazılım Denetim Metotları, Yazılım Kalite Güvencesi, Yazılım
Denetiminde Bilgisayar Desteği
iii
ACKNOWLEDGMENTS
I would like to thank to my supervisor Asst. Prof. Dr. Alok Mishra and Asst. Prof.
Dr. Deepti Mishra for their guidance, and insights through the study.
I would also like to express my appreciation to examination committee members
Prof. Dr. Ali YAZICI, Asst. Prof. Dr. Çiğdem Turhan, Asst. Prof. Dr. Mehmet
Cantürk, and Assoc. Prof. Dr. Mohammed Rehan for their constructive comments.
I would like to express my thanks to my dear lifemate Gözde Aytan for her
continuous morale support and encouragement during this study.
iv
TABLE OF CONTENTS
Page
ABSTRACT........................................................................................................................................ i
ÖZ ..................................................................................................................................................... ii
ACKNOWLEDGMENTS ............................................................................................................... iii
TABLE OF CONTENTS ................................................................................................................ iv
LIST OF TABLES ........................................................................................................................... vi
LIST OF FIGURES ........................................................................................................................ vii
1 INTRODUCTION .....................................................................................................................1
1.1 IMPORTANCE OF REVIEWS 1
1.2 ORGANIZATION OF THE SURVEY 3
2 LITERATURE SURVEY .........................................................................................................4
2.1 REVIEW TECHNIQUES 4
2.1.1 Inspection..........................................................................................................5
2.1.2 Team Review (Structured Walkthrough)..........................................................6
2.1.3 Walkthrough .....................................................................................................8
2.1.4 Pair Programming ...........................................................................................10
2.1.5 Peer Deskcheck...............................................................................................11
2.1.6 Passaround ......................................................................................................12
2.1.7 Ad Hoc Review...............................................................................................13
2.2 CHOOSING A REVIEW APPROACH 13
2.3 SOFTWARE INSPECTION 16
2.3.1 SOFTWARE INSPECTION: BACKGROUND ............................................16
2.3.2 IMPACTS AND BENEFITS..........................................................................17
2.3.3 INSPECTION PROCESS...............................................................................24
2.3.3.1 Preparation Phase............................................................................................24 2.3.3.2 Individual Checking........................................................................................25 2.3.3.3 Logging Meeting ............................................................................................25 2.3.3.4 Process Brainstorming Meeting......................................................................26 2.3.3.5 Final phases ....................................................................................................26 2.3.3.6 Exit criteria .....................................................................................................26
2.3.4 SOFTWARE INSPECTION STANDARDS..................................................27
2.3.5 SOFTWARE INSPECTION TECHNIQUES.................................................28
2.3.5.1 FAGAN'S SOFTWARE INSPECTION.........................................................29 2.3.5.2 Inspections in Software Development Life Cycle ..........................................31 2.3.5.3 HUMPHREY'S INSPECTION MODEL........................................................32 2.3.5.4 FTARM ..........................................................................................................33 2.3.5.5 GILB INPECTION.........................................................................................35 2.3.5.6 TWO-PERSON INPECTION.........................................................................37 2.3.5.7 PHASED INSPECTION.................................................................................37
v
2.3.5.8 N-FOLD INSPECTION .................................................................................38 2.3.5.9 ACTIVE DESIGN REVIEW..........................................................................38 2.3.5.10 MEETINGLESS INSPECTION.....................................................................39
2.4 COMPUTER SUPPORT IN SOFTWARE INSPECTION 44
2.4.1 PROBLEMS IN CONVENTIONAL METHODS .........................................44
2.4.2 COMPUTER SUPPORT ................................................................................45
2.4.2.1 Document handling.........................................................................................45 2.4.2.2 Individual preparation.....................................................................................46 2.4.2.3 Meeting support ..............................................................................................46 2.4.2.4 Data collection ................................................................................................47
2.4.3 COMPUTER SUPPORT TOOLS ..................................................................47
2.4.4 ASYNCHRONOUS SOFTWARE INSPECTION.........................................49
2.4.4.1 Collaborative Software Review System (CSRS) ............................................50 2.4.4.2 Notes Inspector ...............................................................................................51 2.4.4.3 Asynchronous Inspector of Software Artifacts (AISA) ..................................51 2.4.4.4 InspectA..........................................................................................................52 2.4.4.5 Asynchronous/Synchronous Software Inspection Support Tool (ASSIST)....52 2.4.4.6 HyperCode System (WWW-based Software Inspection) ...............................53 2.4.4.7 Web Inspection Prototype (WiP) ....................................................................53 2.4.4.8 FEATURE-BASED CLASSIFICATION OF SOFTWARE
INSPECTIONS ..............................................................................................................55 3 DEVELOPING A NEW INSPECTION METHODOLOGY ..............................................57
3.1 ROLES AND RESPONSIBILITIES 57
3.2 HYBRID INSPECTION PROCESS OVERVIEW 58
3.3 FORMALIZING THE INSPECTION PROCESS 60
3.4 PROCESS SEQUENCE OF THE HYBRID INSPECTION MODEL 62
3.5 ADVANTAGES/DISADVANTAGES OF THE MODEL 63
3.6 A SIMPLE TOOL FOR THE INSPECTION METHOD 64
3.6.1 Overview.........................................................................................................64
3.6.2 Use Case Diagrams.........................................................................................64
3.6.3 Classes ............................................................................................................65
3.6.4 Class Diagram.................................................................................................67
3.6.5 Usage of the Inspection TOOL .......................................................................70
4 CONCLUSION ........................................................................................................................77
5 REFERENCES ........................................................................................................................80
6 APPENDICES..........................................................................................................................86
6.1 SAMPLE CHECKLIST FOR DESIGN INSPECTION 86
6.2 SAMPLE CHECKLIST FOR CODE INSPECTION 88
6.3 SAMPLE CHECKLIST FOR REQUIREMENT BASED TESTING 91
vi
LIST OF TABLES
Page
Table 1 Comparison of Some Characteristics of Review Types ...................................... 10
Table 2 Activities Included in Different Types of Software Peer Reviews ..................... 13
Table 3 Suggested review methods for meeting various objectives ................................. 15
Table 4 Cost of Inspections with respect to human effort ................................................ 21
Table 5 Results of Inspections with respect to quality...................................................... 22
Table 6 Fagan’s Inspection ................................................................................................. 32
Table 7 Comparison of Inspection Techniques ................................................................. 41
Table 8 List of selected features in computer supported software inspection ................ 56
Table 9 Comparison between the proposed tool “T” and the recent tools ..................... 79
vii
LIST OF FIGURES
Page
Figure 1 Cost of Correcting Requirements Defect [Boehm, 81] ........................................ 2
Figure 2 Peer review formality spectrum ............................................................................ 5
Figure 3 The effect of Inspection in Software Development life-Cycle ........................... 18
Figure 4 Learning Effect for authors when participating at inspections........................ 19
Figure 5 Rework vs. Production in development phases [Boehm87] .............................. 20
Figure 6 Cost of rework in software development [Boehm87]......................................... 20
Figure 7 Optimal Inspection Values................................................................................... 23
Figure 8 Inspection Process................................................................................................. 24
Figure 9 IEEE software inspection process [Wheeler, 96] ............................................... 28
Figure 10 Phases in Fagan’s inspection process ................................................................ 30
Figure 11 Software inspections in development process .................................................. 31
Figure 12 Phases in FTARM’s inspection process ............................................................ 34
Figure 13 Phases in Gilb’s inspection process ................................................................... 36
Figure 14 Summary of activities within different software inspection techniques ........ 42
Figure 15 Summary of Inspection tasks............................................................................. 43
Figure 16 Phases in Developed Inspection model.............................................................. 62
Figure 17 Login Page ........................................................................................................... 70
Figure 18 Project and Product Selection Page .................................................................. 71
Figure 19 Comment Entrance Page ................................................................................... 72
Figure 20 Check List............................................................................................................ 73
Figure 21 List Of Comments Page (reviewer’s) ................................................................ 74
Figure 22 Review Comments Page (author’s) ................................................................... 75
Figure 23 Add Author’s Comment Page............................................................................ 75
Figure 24 Action Item Form ............................................................................................... 76
1
1 INTRODUCTION
In this study, software inspection methodologies are researched, and a new software
inspection methodology called ‘Hybrid Inspection Methodology’ is proposed.
Techniques and significance of reviews in software quality assurance are discussed.
Out of these review techniques, the most detailed in process and the formal one,
which is inspection, is discussed in detail. A general inspection process is given and
the requirements of the standards are described.
Because of the importance of software inspection in software development, the
experienced engineers introduce many inspection techniques. Out of these, the most
popular and applicable ones are described and a comparison between them is made.
Also, tool supported techniques are introduced. And finally a new methodology
based on Gilb, Fagan, Humphrey, FTARM Inspection is introduced and formalized.
A simple tool is developed and explained that can be used for the introduced
methodology to visualize the inspection process.
1.1 IMPORTANCE OF REVIEWS
During the Sixties, the crises of software have considerably affected the development
of new methodologies and reviewing procedures became a great part of software
development, particularly in large software projects. These procedures vary from
informal to formal reviewing processes; each of them has its own strengths and
weaknesses. Daly (1977) indicated that management and technical reviews are
important activities of the software development process, which lead to a successful
software project and an end product. The reviews of management help to order
budgets and schedules. Technical reviews are concerned with the exactitude and the
quality of the product and the process, more other technical issues. While software
development projects grow in size, the process becomes more detailed and complex.
In posterior phases of the software development life cycle, management completely
depends on the technical review results and summaries whose reports provide
principle elements for managerial decision-making. [Daly, 77]
2
One of the most important questions, which had to be addressed, was the pressure to
reduce the number of defects, which reached customers. The processes, which are
required for early defect detection, are applied by technical reviews. The evaluations
of defect rework in the total development effort as a percentage, extends from 30 to
80 percent. The two most obvious manners to satisfy these needs were to reduce the
number of defects found during the development, and to find and fix those, which
were found as close to their point of origin in the process as possible. Figure 1
exposes the correcting cost of a requirement defect when it is detected in any 5
phases of the software development.
1,5 1
10
60
100
0
10
20
30
40
50
60
70
80
90
100
Co
st
Un
its
During Design Before Code Before Test During Test In Production
Cost of Correcting Requirements Defect [Boehm, 81]
Figure 1 Cost of Correcting Requirements Defect [Boehm, 81]
3
1.2 ORGANIZATION OF THE SURVEY
This survey is organized as follows. Section 2 presents the literature survey which
includes a study on software review techniques, a discussion on choosing a review
approach, background, impacts and benefits of software inspection, the phases of
inspection process, in addition with the standards and techniques of software
inspection. In this section, there is also a study on computer support in software
inspection that discusses the problems in conventional methods; computer support
tools that are being used in industry and their differences. Section 3 introduces the
proposed software inspection methodology called “Hybrid Inspection Methodology”,
the process sequence of this methodology, advantages/disadvantages of this
methodology, and the tool developed to visualize the inspection process of this
methodology. Section 4 concludes along with future work direction.
4
2 LITERATURE SURVEY
2.1 REVIEW TECHNIQUES
There is often confusion on the roles of reviews, walkthroughs, and inspections. The
fact is that both reviews and walkthroughs (taking these two activities as being
essentially of the same type) and inspections are important in the development of
work products. Reviews/walkthroughs are conducted during development to
demonstrate and improve approaches, compare viewpoints, develop improvements,
and identify problems and create solutions to them. Inspections, on the other hand,
are conducted when a work product is considered to be complete. The objective of
inspection is to find defects only, not to carry out the activities of
reviews/walkthroughs. A defect is defined as a case in which the work product fails
to satisfy the exit criteria of the operation, including operational defects that could be
reported by customers.
It was pointed out by Gilb that walkthroughs should be used for training, design
reviews for consensus, but inspections should be used to improve the quality of the
document and its process [Gilb93]. Walkthrough is a relatively informal review
technique in which the author (see 3.1 Roles and Responsibilities section) leads the
reviewers through a document and the reviewers identifies possible problems and
improvements. The main purpose is to find faults; their correction is not considered.
Walkthrough does not involve use of checklists. The document under inspection is
not checked line by line, it is checked in a process-oriented way. The role of the
author is the most important one in a walkthrough. The author can control the
assessment up to a certain level by directing the focus of his presentation to less
critical parts of the document. The inspection process is more formal than the
walkthrough. The planning phase requires more resources than the planning phase in
a walkthrough. The inspection process is longer than the walkthrough; however, the
extra time and cost is justified by their efficiency in defect detection in the early
phases of the development process when they are easiest and least costly to correct
[Ackerman83]. Design review is a formal, documented and systematic study of a
design proposal done by experts. The experts responsible for design reviews are not
necessarily engaged in the design [Gilb93]. In contrast to inspections and
5
walkthroughs, the focus of reviews is not on detecting technical flaws, but it is on
ensuring that the design meets the project requirements and provides customer
satisfaction. Reviews are effective early on during requirements verification and
conceptual model validation [Hollocker87]. Peer review is the least informal
technique carried out by colleagues. When we consider review techniques from most
formal to least formal, we can put in order as in Figure 2. These review techniques
will be explained in more detail in the next sections.
Figure 2 Peer review formality spectrum
2.1.1 Inspection
The most systematic and accurate type of peer review is inspection. Since it follows a
well-defined, multi stage process with specific roles assigned to individual
participants. Most general inspection process includes seven stages: planning,
overview, preparation, meeting, rework, follow-up and casual analysis. For the best
effectiveness, the inspectors should be trained in the process of inspection and to be
able to carry out all the various roles. The inspections depend on the checklists of the
common defects found in various types of products of work on software, rules to
build products of work and various techniques to seek the bugs.
A primary characteristic of inspection is that participants other than the author of
product of work carry out the meeting (moderator), present the material to the team
of inspection (reader) and record issues as they are brought up (recorder). The
participants prepare for the meeting of the inspection by examining the material on
6
their own to understand it and to find problems. During the meeting, the reader
briefly presents the material to the other inspectors, who raise issues and point out
possible defects.
The reader helps the team to achieve the same understanding of each part of the
product, because the inspectors can compare their understanding, expressed by the
reader. At the end of the meeting, the team agrees on a review of the work product
and decides how to verify the changes that the author will make during rework.
The inspection is more effective in finding defects in comparison with informal
techniques of revision. An example is that, one of the telecommunications company,
detected an average of about 18 defects per thousand lines of code by inspection,
compared to only three defects per thousand lines of code using informal reviews.
The close analysis of an inspection provides a complete test of the understandability
and maintenance of a product, and indicates problems of programming. Various
inspectors mention various kinds of problems, but this involvement does not increase
linearly with additional participants. When one inspector notes a defect, it’s common
to hear another participant say, “I saw that, too.” The interactions between the
inspectors during the meeting can indicate new bugs while an observation of the
person stimulates the thought of another person. This bidirectional advantage can be
lost if your peer review does not include a meeting. On the other hand some studies
have questioned whether the synergy adds any real value.
2.1.2 Team Review (Structured Walkthrough)
The team reviews are a type of “inspection-lite”: although planned and structured,
they’re a bit less formal and less comprehensive. The team review goes by many
names, including “structured walkthrough” and simply “review.” Yourdon (1989)
describes in detail how to perform a structured walkthrough. A structured
walkthrough is a group peer review of the product, being composed of a series of
reviews, carried out in all the software development cycle. Each review has its proper
objectives. The common objective is detection of error and improvement of quality.
7
Participants have specific roles, such as coordinator, scriber, presenter, reviewer,
producer (author), user representative, standards bearer, and maintenance oracle.
Team reviews follow several of the steps found in an inspection. The participants
receive the materials of the review several days before the meeting of the revision,
and expect to study the materials on their own. The team gathers data on the effort of
review and the number of found defects. However, the overview and follow-up
inspection stages are simplified or omitted, and some participant roles may be
combined. The author may lead the team review, whereas in an inspection, the author
may not serve as the moderator. In contrast to most inspections, the reader role is
omitted. Instead of having one participant to describe a little of the product at a
moment, the review leader asks the participants if they have any comments on a
specific section or page. The team reviews cost more than having a single colleague
to carry out a peer deskcheck, but the various participants will find various sets of
errors. Like in any meeting, the team can get unfocused in tangential discussions, so
the leader of the revision must keep the meeting on the way. As the issues are raised
during the discussion, recorder or scriber captures them using standard forms the
organization has created.
This is a peer review process, so no managerial staff is involved. MacDonald (1998)
pointed out that the structured walkthrough has organizational, preparation,
walkthrough, rework, and follow-up phases. The coordinator contacts participants,
prepares and distributes documentation, and selects a schedule for the walkthrough.
Participants spend time preparing for the walkthrough by examining the product. The
walkthrough, which is a group activity, is the central component of the review
process where all participants meet and discuss or comment on the product. Then, a
list of the comments is made and passed to the producer(s). This list is used to make
changes to the product. A final management summary is also produced by the
coordinator. There is little data gathered during walkthroughs. No feedback is used to
control the development process. Due to the heterogeneity of (large) software
projects, walkthrough meetings are used as means to provide education pertaining to
8
software development to various participants of the software project. Freedman and
Weinberg (1982) explained that structured walkthroughs are in fact walkthroughs.
While few data are available, a study of industry noted that this type of peer review
discovered only two-third of as many defects per hour of effort compared to
inspections. The team reviews are appropriate a product which does not require the
full strictness of the process of inspection.
2.1.3 Walkthrough
The author of a work product describes it to a group of peers in a walkthrough, which
is an informal review. The author takes the dominant role; there are no other specific
reviewer roles, so, walkthroughs differ significantly from inspections. On the other
hand an inspection is intended to meet the team’s quality objectives; a walkthrough
principally serves the needs of the author. Table 1 points out some differences among
the ways that inspections, team reviews and walkthroughs are usually performed.
Walkthroughs that originated at IBM reviewed the methods to examine source code
to detect possible errors of codification. The achievement of the coding phase is
considered as an important stage in software development. It was clear that the
programming errors not discovered before the product was deployed led to the major
software improvement. These original walkthroughs were peer reviews performed in
informal fashion, as the term code “walk-through” may suggest. Later, walkthroughs
were also used for reviewing materials other than source code, including requirement
and design documents, structure charts and test data. It was done to prevent errors
from propagating in following phases of software development, which results in an
increase in the total cost. The majority of the forms of software review have a simple
common goal, i.e. to improve the quality of the software product. However,
otherwise similar techniques can be distinguished in specific terms of objectives.
Reviews and walkthroughs accentuate multiple objectives and may not have rigorous
means to achieve such objectives. Fagan (1976), a quality control engineer at IBM,
proposed additional rigorous activities (inspections) to be coupled with traditional
development activities from the requirement to the testing phases. Based on a series
9
of inspections performed at IBM, Fagan demonstrated a cost-benefit analysis
showing the cost-effectiveness of his software inspection. The method of Fagan was
published later extensively.
Walkthroughs typically do not follow a defined procedure, require no management
reporting and generate no metrics. They can be effective manner to review products
modified during maintenance because the author can draw the reviewers’ attention to
those portions of the deliverables, which were changed. However, this runs the risk
of overlooking other sections that should have been changed but could not. Since
records are seldom kept, there are few data about how effective walkthroughs are at
detecting bugs. The inspections at Ford Motor Company discovered 50 percent more
defects per thousand lines of code in comparison to walkthroughs.
When the walkthroughs do not follow a defined process, people carry out them in
various manners. In a typical walkthrough, the author presents a module of code or
component of design to the participants, describing what it does, how it is structured
and carries out its tasks, the flow of logic, and its inputs and outputs. Finding
problems is one goal; reaching a shared understanding of, and a consensus on, the
goal of the component, structure and implementation is another. The walkthroughs of
design provide a manner of evaluating if the design suggested is sufficiently robust
and is adapted to solve the problem.
If you do not fully understand some information presented in a walkthrough, you
could annotate it more and assume the author is right. It’s easy to be swayed into
silence by the rationalization of an author of his approach, even if you are not
convinced. Sometimes, though, a confusion of a reviewer is a hint that a defect is
likely to exist. Walkthroughs have a place in your peer review toolkit, but the
domination by the author and unstructured approach render them less valuable than
either inspections or team reviews as a defect filter.
10
Characteristics Inspection Team Review Walkthrough
Leader Moderator Moderator Author
Material Presenter Reader Moderator Author
Granularity Small chunks Pages or sections Author’s
discretion
Recorder used Yes Yes Maybe
Follows a documented
procedure Yes Maybe Maybe
Defined participant roles Yes Yes No
Defect checklists used Yes Yes No
Data collected and
analyzed Yes Maybe No
Product appraisal
determined Yes Yes No
Table 1 Comparison of Some Characteristics of Review Types
2.1.4 Pair Programming
Two developers work on the same product simultaneously at a single workstation in
pair programming. This creates communication and provides an occasion for the
continuous examination, incremental and informal review of ideas of each person.
Two brains driving a single set of fingers writing every line of code leads to better
work products by literally applying the idea “two heads are better than one”. Pair
programming can be used in order to create any software work product, not just
code. Culturally, pair programming improves collaboration and a common
engagement to the quality of every member. At least two team members become
familiar with every part of code so it reduces the acquaintance lost throughout the
turnover of the staff. The pair can quickly make corrections because of the real-time
review.
11
Pair programming can be classified as a type of informal review because it is
unstructured and involves no process, preparation or documentation. It lacks the
outside ideas of someone who is not personally attached to the code that a formal
review brings. It also does not include the author of the parent work product as a
separate perspective. Pair programming is not specifically a review technique, but
rather a strategy of software development, which is based on the synergy of two
focused minds to create higher products in design, execution and quality. However,
pair programming is an important culture change in the manner that a development
team actuates, thus it’s not a simple replacement for traditional peer reviews in the
majority of the situations.
2.1.5 Peer Deskcheck
In a peer deskcheck, only one person other than the author examines the work
product. The author could not know how the reviewer approached the task or how
largely the review was done. A peer deskcheck depends completely on the reviewer’s
knowledge, competence and self-discipline, thus to expect broad variability in the
results. Peer deskchecks can be practically formal if the reviewer uses checklists of
defect, methods specific of analysis and standard formats to keep records. On the
completion, the reviewer can provide a list of defect to the author or simply give to
the author his marked-up work product.
The peer deskcheck is the cheapest review approach. It implies only one reviewer’s
time plus perhaps a follow-up discussion with the author to explain the reviewer’s
findings. This method is appropriate for the products which are less risky, if you
have colleagues who have skills to find defects on their own or if you have serious
restrictions of time and resource. A peer deskcheck can be more comfortable for the
author than a group review. However, the only found errors will be those a reviewer
is best at spotting. Moreover, the author is not present to answer questions and hear
the discussions, which can help him to find additional defects that nobody else sees.
Peer deskchecks provide a good way to develop a review culture. Find a colleague
that you respect and trust personally, and offer for changing work products for peer
12
deskchecks. It is also a good method of supervision, providing it’s done with
sensitivity.
2.1.6 Passaround
A passaround is a multiple, parallel peer deskcheck. Instead of requesting an input
from a single colleague, you provide a copy of the product to several people and
assemble their feedback. The passaround decreases two major risks of a peer
deskcheck: the reviewer does not provide convenient feedback and the reviewer
doing a poor work. You can make more reviewers through a passaround than you
could conveniently meet at a meeting. However, a passaround still lacks the
advantage that a group discussion can provide.
As an alternative to distributing physical copies of the document to the reviewers, an
electronic copy of the document can be put in a shared folder on our server.
Reviewers contribute their comments in the form of document annotations, such as
Microsoft Word comments or PDF notes, for a set period of time. Later, the author
fixes conflicting inputs from the reviewers, doing obvious corrections and not paying
attention to useless suggestions. Then only a few issues remain that required the
author to sit down with a particular reviewer for clarification or brainstorming.
This passaround method makes it possible for each reviewer to see the comments
others have already written, which reduces the loss of work to the minimum and
indicates differences in interpretation. Watch out for debates that might take place
between reviewers in the form of document comments; those are better handled
through direct communication. The review of document of this type is a good choice
when you have reviewers who cannot hold a face-to-face meeting because of
geographical conditions or time limitations.
13
2.1.7 Ad Hoc Review
Spur-of-the-moment reviews should be a normal part of software team cooperation.
They provide fast manner to obtain another prospect, which often finds problems that
we just cannot see ourselves. The Ad hoc reviews are the most informal type of
review. They could solve the instantaneous problem, but they have little impact
beyond that.
Review Activity
Review Type Planning Preparation Meeting Correction Verification
Inspection Yes Yes Yes Yes Yes
Team Review Yes Yes Yes Yes No
Walkthrough Yes Yes Yes Yes No
Pair
Programming Yes No Continuous Yes Yes
Peer
Deskcheck,
Passaround
No Yes Possibly Yes No
Ad Hoc
Review No No Yes Yes No
Table 2 Activities Included in Different Types of Software Peer Reviews
2.2 CHOOSING A REVIEW APPROACH
One way to choose the appropriate method of review for a given situation is based on
the risk, the probability of a work product containing defects and the potential for
damage if it does. Cheaper techniques should be relied on for components that have
lower risk, and inspections should be employed for the high-risk work products.
Some factors that increase risk in any project include:
• Use of new technologies, techniques or tools
• Complex logic or algorithms that are difficult to understand but must be
accurate and optimized
14
• Excessive developer schedule pressure
• Inadequate developer training or experience
• Mission- or safety-critical sections of the product with severe failure modes
• Key architectural components that provide a base for subsequent product
evolution
• A large number of exception states or failure modes, which must be handled,
in particular if it is difficult to stimulate them during testing
• Components that are envisaged to be reused
• Components, which will be used as models or templates for other
components
• Components with multiple interfaces, which affect various parts of the
product.
A review technique can also be selected based on your objectives indicated for the
review. The review approaches that are best suited to achieve specific goals are
suggested in Table 3 (by Karl E. Wiegers). The best manner of comparing which
peer review method to use in a given situation is to keep records of review
effectiveness and efficiency in your own organization. It might be found that
inspections work well for code, but that team reviews or walkthroughs work best for
design documents. The most important is, a culture, which embraces peer review as a
powerful supplier of increased software quality; productivity and individual learning
should be developed.
15
Review Objectives Inspection Team Review Walkthrough Pair Programming Peer Deskcheck Passaround
Find implementation defects √ √ √ √ √ √
Check conformance to specifications √ √ √
Check conformance to standards √ √
Verify product completeness and correctness √ √
Assess understandability and maintainability √ √ √ √ √
Demonstrate quality of critical or high risk components √
Collect data for process improvement √ √
Measure document quality √
Educate other team members about the product √ √ √
Reach consensus on an approach √ √ √
Explore alternative approaches √ √
Ensure that changes or bug fixes were made correctly √ √ √
Simulate execution of a program √
Minimize review cost √
Provide progress check to management and customers √ √
Table 3 Suggested review methods for meeting various objectives
16
2.3 SOFTWARE INSPECTION
2.3.1 SOFTWARE INSPECTION: BACKGROUND
Software inspection became the most used method [Wheeler, Brykczynski, &
Meeson, 1996] since it was presented by Fagan (1976) at IBM. It is because of the
potential advantages to the development of software, the increased request for
certification of quality in software, (e.g. ISO9000 compliance requirements), and the
adoption of the Capability Maturity Model as a methodology of development.
Inspection proved that it’s cost-effective because it helps to discover defects before
they deviate to subsequent phases of the software life cycle. This reduces the delay in
discovering defects and the cost to fix them. The software systems, which are
missioning critical (air traffic control) or that impact human life and well being
(medical systems) are primary candidates for software inspection.
Software Inspection is the process in the development of software where the software
is examined for defects. It includes detection, definition and correction of errors
before the release of software.
Software inspection by Fagan requires that the detection of defect be continued at a
face-to-face meeting. Software inspection can be carried out asynchronously by the
use of computer support. Asynchronous software inspection tries to surmount the
same place/same time constraints of the traditional method. Inspectors privately
create observations about defects, and these observations can be posted on the screen
of computer and accessible by all members of the inspection team.
Software Inspection:
• Developed by Michael Fagan (Fagan, 1976)
• All kind of Documents (Requirements, Specifications, Designs, Code, Test cases,
etc...)
• 2 to 5 People
• Formal (check lists, process)
17
• Inspection group is responsible for final quality
• Estimation of savings
• Estimation of number of remaining errors
• Knowledge base – statistical based quality improvement
2.3.2 IMPACTS AND BENEFITS
The advantages of software inspection were studied and documented since its
introduction in 1976. Fagan (1986) brought back advances in software inspection and
the advantages occurring from it. Ackerman (1989) indicated that when software
inspections are carried out correctly, the quality of the total product and productivity
of development would increase significantly. Bisant and Lyle (1989) reported
advantages of two-person inspection teams. Rusell (1991) brought back the success
of inspection in ultra projects on a large scale of development. Kelly, Sherif., and
Hops (1992) brought back a three-years experience of Jet Propulsion Laboratory’s
software inspection practice that inspection reduced cost to fix defects from 5 – 15
hr/defect (using formal testing) to 0.5 hr/defect.
Inspections shorten delivery period by reducing the time spent in the integration and
system test and debug phases, since a cleaner product is passed into those late-stage
quality filters. A better quality in the product saves the maintenance time, too. The
reduction of the effort that you must spend fixing bugs after delivery, frees up time,
which can be employed for the new development work.
18
Figure 3 The effect of Inspection in Software Development life-Cycle
Source: Institut für Informatik-Systeme Universitat Klagenfurt
In Figure 3, the effect of inspections over the project schedule and people resource
can be seen. Until half of the coding phase, usage of people resource is high when
using inspections. The reason is that the inspection over plans/requirements, design
and code takes more resource of people. After this, the use of people resource
decreases in comparison with / “without inspections”. Moreover the schedule of the
project shortens, since the defects are discovered and solved in the earlier phases of
the project in the requirement, design and code inspections. Consequently the quality
of the product increases, the schedule of the project shortens and use of people
resource decreases in comparison to “without inspections”.
Benefits of Inspections can be listed as the following:
(Source: Gilb 1988, Principles of Software Engineering Management)
• Decrease development cost (25-35 %)
19
• Decrease development time (25-35%)
• Decrease maintenance cost (factor 10-30)
• Higher Quality (10-100 times less errors)
• Learning effect for developers
0
10
20
30
40
50
60
70
80
90
0 1 2 3 4 5 6
Experience as Checker
Defe
cts
Id
en
tifi
ed
Figure 4 Learning Effect for authors when participating at inspections
Source: Gilb1988, Principles of Software Engineering Management, Table 12.2
Fruhauf pointed out that, from a sight of a programmer; the inspection opens a
manner for feedback on a regular (real-time) basis. Programmers are well informed
which types of errors they tend to make and they can try to avoid them or change
their programming style. This improves the practice of prevention of defect, which
reduces development cost. This effect can be seen in Figure 4. It was noted that the
inspections are practices, which encourage standardized effort. Standards are
employed and forced when inspections become a part of practice. Documents of
standards are consulted as key components of development. Additional advantages of
inspection include more reliable measurement of project progress, a distribution of
knowledge, and a development of common values and improved team spirit fostering
[Fruhauf, 1996].
20
0
5
10
15
20
25
30
Requirements Preliminary
Design
Detailed Design Code and Unit
Testing
Integration and
System Testing
Rework Production
Figure 5 Rework vs. Production in development phases [Boehm87]
Figure 5 proves that the cost of defect correction increases significantly for each
successive software development stage and defects from the earlier stages have
greater costs of recovery later in the cycle of development than they are discovered.
Rework
44%
Code and Unit
Testing
12%
Detailed Design
16%
Requirements
6%
Integration and
System Testing
10%
Preliminary
Design
12%
Figure 6 Cost of rework in software development [Boehm87]
According to the indications of Figure 6, 44% of typical software development effort
is devoted to defect-correction (rework). The use of inspections can significantly
reduce the quantity of necessary rework because defects are found early in the life
21
cycle. Both figures are created from the data published by Boehm [Boehm87].
Project Inspection’s Results
IBM Santa Teresa Lab
[Remus84]
Ratio of the cost of fixing defects during inspection to
fixing them during formal testing is 1:20
Large real time software
project [Collofello89]
7.5 hours per defect for design inspection, 6.3 hours per
defect for code inspection, and 11.6 hours per defect for
testing
Jet Propulsion
Laboratory [Kelly92]
1.75 hours per defect of design inspection, 1.46 hours per
defect for code inspection, 17 hours per defect for testing
Bull HN Information
Systems [Weller93]
1.43 hours per defect in inspection and 6 hours in testing
Applicon [Gilb93] 0.9 hours to find and fix a major defect
IBM Rochester Lab
[Kan95]
Ratio of the cost of fixing defects during inspection to
fixing them during formal testing is 1:13
Jet Propulsion
Laboratory [Kaner98]
Ratio of the cost of fixing defects during inspection to
fixing them during formal testing range from 1:10 to 1:34
Table 4 Cost of Inspections with respect to human effort
In Table 4, it can be seen that the fixing a defect in the earlier phases of the project
requires less resource of people and cost is much less. The defects, which cannot be
detected in the subsequent phases of the development, i.e. formal testing, the cost to
fix a defect can be definitely high.
22
Project Inspection’s Results
Aetna Life Casualty [Fagan, 76] 82% of all detected defects
IBM Respond [Fagan, 86] 93% of all detected defects
Standard Bank of South Africa [Fagan,
86]
Over 50% of all detected defects
Large real time software project
[Collofello, 89]
Effectiveness for design inspections:
54%
Effectiveness for code inspections: 64%
Effectiveness for testing: 38%
Bull HN Information Systems [Weller,
92]
70% of all detected defects
Hewlett-Packard [Grandy, 94] Over 60% of all detected defects
AT&T Bell Laboratories [Barnard, 94] 30%-75% of all detected defects
Cardiac Pacemakers Inc [McGibbon, 96] 70% to 90% of all detected defects
Erickson[Conradi, 99] Average number of defects detected:3.41
Table 5 Results of Inspections with respect to quality
The effect and benefit of the inspection can be observed in Table 5. It has been seen
that by the inspection process approximately 70% of all defects are detected before
the delivery of the product. It can be decided that in the formal testing of the product,
the number of defects are decreased at least 70%, which will decrease the phase of
correction of errors or maintenance of errors of the project.
Another question is, in order to carry out an effective inspection, large files of
documentation, which will be inspected, should be divided into reasonable sizes.
Figure 9 shows the optimal inspection rates in order to carry out an effective
inspection.
23
Figure 7 Optimal Inspection Values
Text Documents: 1 to 5 pages/hour (page: ~300 words)
Source: Tom Gilb, Denise Leigh
24
2.3.3 INSPECTION PROCESS
Figure 8 Inspection Process
An inspection process can be described as in Figure 8. The main stages of the
process can be grouped as the following: Preparation phase, individual checking,
logging meeting, process brainstorming meeting and final phases. For each stage,
actions that should be considered are as the following,
2.3.3.1 Preparation Phase
• Entry:
– Check, whether inspection is feasible and reasonable
– Documents have to pass the entry criteria
* Quality, Maturity, Voluntarily participating persons
• Planning:
– Participants and roles
– Schedule
– Partitioning of the documents
25
– Master plan
• Kickoff Meeting (optional)
– Hand out documents, check lists, process description, role descriptions
– Author gives an overview
– Affirm or change roles
• Explain process changes
2.3.3.2 Individual Checking
• Check for major defects
• Generate individual log
• Use rules and checklist
• Consider Optimal Inspection Rate
– Programs 100-150 nloc/h
– Text documents: 1-5 pages/hour
• Individual Checking finds 80~85 % of all defects found during the inspection
• Effectiveness
– Inexperienced checkers find up to 30 % of all errors
– Experienced ones up to 90 %
2.3.3.3 Logging Meeting
• Goals:
– Log detected errors
– Find more errors (double checking) (~ 20 % new defects)
– Improve the development process
• Duration max. 2 hours
• Don‘t shoot at the author
• No discussion: logging rate > 1 issue / Minute
• Author decides afterwards, if major is major,...
• Each issue is described in max. 7 words
• Result: Issue Inspection Log
!= Defect Log
26
2.3.3.4 Process Brainstorming Meeting
• Identify reasons
– Why does this defect occur repeatedly?
– Why is this defect introduced only by this group?
– ...
• Develop solutions
– Result: process brainstorming log which is input for the software
– Process improvement group
2.3.3.5 Final phases
• Edit (rework) phase
– Author corrects identified majors
– Submits change requests
– Attention: every 6 reworked issue introduces one error!
• Follow up
– Moderator checks if all issues are handled
– updates the QA database (data summary)
• Exit
– Decision if product should be reworked or not
– Based on formal exit criteria
2.3.3.6 Exit criteria
Quality of document after inspection should match expectations
• Rules of Thumb
– Defects remaining in the document = defects found
– Depending on
* Document type
* Process maturity
* Experience of inspection group
• Re-editing problem
– One correction in six introduces a new defect
– Removal effectiveness between 30% and 88%
27
2.3.4 SOFTWARE INSPECTION STANDARDS
Fagan Inspection concentrates on the three main procedures - Preparation,
Collection, and Rework. Many later review methods are based on this pattern. Porter
(1997) reasserts the three-step process as:
• Individual analysis (preparation)
• Team analysis (collection)
• Repair
This formulation covers many variations of various software inspection techniques,
other than Fagan Inspection. Individual analysis is the step in which each individual
examines the material independently, whether to gain familiarity or to detect defects.
Team analysis is the activity where all participants discover true defects. Repair is
the post inspection activity concerned with defect removal. The standard of IEEE for
the Review software and auditent (1988, 1997) is based on this process of three
stages of inspection. The process is recapitulated in Figure 9.
NASA’s Software Formal Inspections Standard (NASA, 1993), suggested by the
Office of Safety and Mission Assurance; provides rules, procedures, and
requirements to ensure the quality of the formal software inspections carried out
during the development of software. All NASA software development must undergo
this inspection procedure to ensure the quality of software developed and used in the
NASA program.
28
Figure 9 IEEE software inspection process [Wheeler, 96]
29
2.3.5 SOFTWARE INSPECTION TECHNIQUES
Several inspection processes are in widespread use. The most popular are original
method of Michael Fagan [Fagan, 76] and the variation developed by Tom Gilb and
Dorothy Graham [Gilb, 93]. Although these methods employ different terminology
and define different manners to carry out various stages of inspection, their
similarities compensate their differences. One or the other method, as well as
additional inspection techniques described in the literature, can help any software
organization to improve its product quality and development productivity.
The later sections give a short description of the most important inspection
techniques.
2.3.5.1 FAGAN'S SOFTWARE INSPECTION
Software inspections aim at increasing the productivity and the quality of the product
and reduce the development cost, in terms of time and money. According to Fagan
[Fagan, 1976], the success of a software project depends on controlling the internal
processes. Each program development process requires inputs and outputs, which
satisfy a specific set of exit criteria. He emphasized on the use of detailed exit criteria
as process control checkpoints. He thought that this should be invariable through
projects. The later the rework is performed the higher the cost. Consequently, he
discovered that detecting and fixing errors should be made as soon as possible in
order to reduce the development cost of product. Thus, it is desirable that “clean”
(error-free) exit criteria be defined. Inspection ensures that exit criterion is
completely satisfied, reducing the total development cost to the minimum. Fagan’s
inspection underlines the use of the inspected data in order to control the
development process as well as the inspection process itself. Software inspection is a
review process to improve software development by focusing on team-based fault
detection in software artifacts. Fagan believes that meetings can increase the
effectiveness of inspections. Certain roles are assigned to participants in the
inspection process.
30
A typical Fagan’s inspection involves four to six participants, with each participant
having a well-defined role in the inspection. It includes author, moderator, reviewer,
reader and recorder. Fagan’s inspection method occupies strictly structured, meeting-
oriented procedures. [IEEE97].
As shown in Figure 10, it has three phases; organization, detection, and completion
and there are five operations in the inspection process as originally defined by Fagan
[1976]:
• Overview
• Preparation
• Inspection
• Rework
• Follow-up
- Overview - Preparation - Detection
- Rework - Follow-up
Figure 10 Phases in Fagan’s inspection process
The organization phase includes an overview stage, which is an overview meeting
that involves the entire team. The main activities include, defining the objectives of
the inspection, assignment of roles and distribution of inspection material.
The detection phase involves two stages; preparation and detection. In the
preparation stage, each team member carries out individual preparation by inspecting
the document and recording the potential defects. The use of checklists can help
inspectors focus on certain parts of the document. In the detection stage, the
31
moderator leads this meeting. The entire team meets and jointly identifies potential
defects. Defect detection should be the only purpose of this meeting; discussions
concerning causes or solutions of the defects should be avoided since they might
degrade the inspection results. The meeting should be rescheduled, if the moderator
notices that the participants are not prepared. This meeting should not be longer than
two hours. The moderator writes a report that includes the inspection details and the
detected defects. This report is submitted to the author.
The completion phase includes the rework stage and the follow-up stage. During the
rework stage, the author carries out the necessary modifications to handle the defects
and issues found during the inspection. In the last stage, the moderator ensures that
all the required modifications have been made. The moderator then decides whether
a partial or full re-inspection of the document is required.
2.3.5.2 Inspections in Software Development Life Cycle
Fagan Inspection is intended as an integral part of the software development life
cycle. Inspection activities are required at the completion of certain software
development phases as described in Figure 11. The names and all mnemonics are
listed in Table 6. These inspections are more structured and formal than regular
reviews. They maximize the correctness of the product.
Figure 11 Software inspections in development process
Requirement High-Level
Design Detailed
Design
Test Plan Test Case
Code Test Execution
=Software Inspection
32
(Ackerman, Fowler & Ebenau, 1984, p. 19)
PHASE INSPECTION TYPE MNEMONICS
Requirement Software Requirements
inspection
R0
High-level Design Architectural design inspection I0
Detailed Design Detailed design inspection I1
Implementation Source code inspection I2
Testing Test plan inspection IT1
Testing Test procedures & functions
inspection
IT2
Table 6 Fagan’s Inspection
Fagan says that the results of inspections should never be used as criteria for
assessment of programmers (Fagan, 1976, p.197). This is to prevent inadequate
inspections due to personal conflicts or loyalties. Inadequate inspections lead to
degradation of inspection processes. Inspections are a self-improvement process
because inspection results are used as a feedback for both defects removal and
professional development. The authors of the artifacts are informed of errors they
have made and perform corrections. Inspection statistics, such as inspection rates and
error density, are gathered and used for determining characteristics of inspections
and for process improvement as well as defect prevention. Inspections also provide
benefits to later phases of software development - for example, providing internal
knowledge about the system. Participants can become well informed about the
artifacts inspected, requirement documentation or source code. This knowledge will
be beneficial during the testing and the maintenance phases.
2.3.5.3 HUMPHREY'S INSPECTION MODEL
Humphrey (1989) developed structured techniques for inspection as a central part of
his software process model. A specialized team is organized and includes members
with specific roles. Inspection is performed in a process similar to Fagan’s three-step
33
inspection – preparation, inspection, and repair, but with additional activities. The
process includes planning, overview, preparation, analysis, inspection, rework, and
follow-up. The moderator prepares for the inspection by selecting participants and
entry criteria. Participants get together in an overview meeting in which the
inspection objectives are described. Each participant also receives inspection
materials, and then individual preparation starts. Reviewers individually inspect the
material and create defect lists. All defect logs are given to the moderator for
preliminary analysis, and its results are used during the inspection meeting. During
the inspection meeting, participants produce a final defect list. Rework and follow-up
are, then, performed as post-inspection activities.
2.3.5.4 FTARM
In order to overcome the ineffectiveness and inefficiency of conventional formal
technical reviews, as well as to promote their implementation in software
organizations, Philip Johnson developed an asynchronous inspection method called
Formal Technical Asynchronous Review Method (FTARM), which removes the
need to have face-to-face group meetings [Johnson93] [Johnson94]. FTArm is
designed to overcome weaknesses in the Fagan Inspection method, by leveraging
parallel activities and employing computer control. FTARM is a computer-based
method, which provides an asynchronous discussion environment. It allows the users
to access an online version of the document under inspection. Reviewers can add
their comments to the document that will be visible to other reviewers.
A review tool called Collaborative Software Review System (CSRS) has
implemented FTARM [Johnson93]. As with conventional inspection, FTARM
defines several roles: moderator, producer and reviewer. FTARM process consists
three phases and seven stages.
• Setup
• Orientation
• Private Review
• Public Review
• Consolidation
34
• Group Review
• Conclusion
Figure 12 Phases in FTARM’s inspection process
The organization phase involves setup and orientation stage. In the setup stage, the
inspection team is selected and the document for inspection is prepared via CSRS.
The document under inspection must be split into a set of hypertext nodes. The
orientation stage is same as the overview stage in Fagan’s inspection and involves
preparation of the participants. The private review stage is same as preparation stage
in Fagan’s inspection. Each participant examines the hypertext nodes of the
document. Participants can make new nodes containing new issues and comments.
Comment nodes are visible to the entire team, which allow inspectors to obtain
guidance from each other. Issue and action nodes do not allow public access. The
detection phase involves four stages. In the public review stage, all the nodes created
by reviewers allow public access to the entire team. Reviewers can asynchronously
vote for each comment. Voting can be one of three; confirm the issue, disconfirm the
issue, or neutral. This stage ends when all the nodes have been resolved or if the
moderator decides that further discussion would not be useful. During the
consolidation stage, the moderator summarizes the results of the inspection into a
report and submits it to the document’s author. The report should include reviewers’
opinion about resolved and unresolved issues, and their degree of consensus. In the
group review meeting stage, the moderator can decide to arrange a face-to-face group
meeting, if there are still any unresolved issues. During the completion phase, the
moderator produces the final inspection report and a metrics report.
- Setup - Orientation
- Private Review - Public Review - Consolidation - Group Review
- Conclusion
35
2.3.5.5 GILB INPECTION
Gilb and Graham (1993) developed a comprehensive inspection method. Inspection
steps include entry, planning, kickoff, checking, logging, brainstorming, edit, follow-
up and exit. While the names of the steps vary from those used by Fagan, they are
based on the steps in Fagan Inspection with additional pre- and post-inspection
activities that help ensure successful software inspection.
The inspection process begins when certain entry criteria are satisfied. Then, the
leader selects participants and generates an inspection plan. A kickoff meeting
provides an overview where the inspection goals and plan are described to all
participants. Documents are also distributed at the kickoff meeting. Next, each
inspector begins checking the documents for defects. All participants get together in
the defect-logging meeting to jointly log as many defects as possible. The leader
moderates discussions during the logging meeting. Optionally, a brainstorming
session may be held after the logging meeting. This is a causal analysis meeting for
improving development processes by preventing causes of discovered defects from
happening in the future. Recommendations resulting from the cause analysis meeting
may affect various aspects of development, including technical and organizational
aspects. After that, post inspection activities take place. The authors receive the
defect log, then edit (rework) the product. Upon completion of the rework, the leader
performs a follow-up and consults exit criteria before the inspection is declared
complete. The method also includes phase definitions, metrics and technical details
for conducting inspections in order to achieve the optimal benefits of inspection.
The phases in Gilb’s inspection are almost similar to Fagan’s inspection except the
brainstorming phase right after the inspection meeting. The organization phase is
divided into three stages; entry, planning, and kick-off. The entry stage is started
when certain entry criteria have been satisfied. In the planning stage, the team
members are selected and an inspection plan is prepared. The Kick-off stage gives an
overview of the documents and the inspection plan to all of the participants. The
normal duration for this stage is 30 minutes.
36
• Entry
• Planning
• Kickoff
• Checking
• Logging
• Brainstorming
• Edit
• Follow-up
• Exit
- Entry - Planing - Kickoff
- Checking - Logging - Brainstorming
- Edit - Follow-up - Exit
Figure 13 Phases in Gilb’s inspection process
In the detection phase, the stages of checking, logging, and brainstorming are
conducted. During the checking stage, each participant checks the documents for
defects. They should also note the points that need improvements, to make the
current work process more efficient. In the logging stage, the entire team meets and
jointly logs all the detected defects. The leader is responsible for leading this
meeting. Discussions should be avoided at the logging meeting because it takes up
all team members’ time and steals attention from defect detection. This meeting
should not be longer than 2 hours. The optional stage of brainstorming involves
causal root cause analysis meeting. The main purpose of this activity is to improve
the development processes by preventing causes of discovered defects from
happening in the future. The recommendations to improve the inspection procedure
and standards are also handled in this activity. The maximum duration for this
meeting is 30 minutes. The completion phase involves three stages; edit, follow-up,
and exit. During the edit stage, the author receives the defect log and makes
necessary changes in the document. In the follow-up stage, the inspection leader
37
ensures the correctness of the modified document. He should also check that edit
process has been properly followed. The exit stage is an objective form of approval.
The main purpose of this activity is to check if the product is economic to release.
The inspection leader should follow the determined exit criteria and not exit the
inspection, unless exit criteria are properly met.
2.3.5.6 TWO-PERSON INPECTION
Bisant and Lyle (1989) introduced that a two-person inspection is feasible. Its
advantage is that it is less costly than the conventional Fagan Inspection. The
inspection team consists of the author and one reviewer performing an individual
review and a one-session inspection meeting.
2.3.5.7 PHASED INSPECTION
This technique is an enhanced inspection technique (Knight & Myers, 1993) that
attempts to overcome weaknesses of other inspection techniques. It addresses defect-
free software but also maintainability, portability, and reusability. Phased
inspection’s aim is to ensure that the quality of the software product will be retained
after being inspected or reviewed. Applying reviews strictly and utilizing computer
support achieve this. Computer-supported software inspection allows reviews to be
carried out in parallel and increases group activity. The term phase is used to ensure
that each inspection is performed for either a single property or a small set of related
properties. A number of inspection phases are conducted in series. Phased inspection
can be performed in single-inspector or multiple-inspector mode. In the single-
inspector mode, a person reviews the product against an application-independent
binary checklist. A phase finishes when all checks are satisfied, and then a defect list
is produced. In contrast, the multiple-inspector mode aims to discover small and
hidden defects that may not be detected by a single person, such as performance
criteria and completeness. Inspection is carried out in three steps: examination,
inspection and reconciliation. Work products and reference documents are
distributed for inspectors to examine. Then, inspectors independently examine the
product against required properties defined for the phase (inspection). Domain-
38
specific and application-dependent checklists in part drive the inspection. The last
step is resolution where all participants meet to bring together their findings. The
completed checklist is produced as the result.
2.3.5.8 N-FOLD INSPECTION
The N-fold inspection method (Tripp, Struck, & Pflung, 1991; Schneider, Martin, &
Tsai, 1992) is a technique where multiple inspections on the same artifact are carried
out in parallel, by some number (N) of teams. Improved inspection performance is
achieved due to the hypotheses that a single inspection team can detect only a subset
of the total number of defects and that multiple inspection teams can detect (more)
unduplicated defects. Using inspection teams that employ different techniques also
results in improved overall inspection. The inspection begins with planning and
overview, performed by the head moderator who controls the overall process. Then
each team separately conducts its own defect detection using their preferred
technique. The head moderator performs an additional activity before the post-
inspection activities begin. The inspection results are brought together and a single
defect list that will be passed to the authors is created. The inspection ends after the
follow-up phase.
2.3.5.9 ACTIVE DESIGN REVIEW
Parnas and Weiss (1989) proposed that large meetings in conventional design
reviews limit interactions between reviewers and authors, leading to an ineffective
review process. ADR approach is carried out by conducting several smaller
(specialized) reviews, each of which focuses on certain parts of the artifact. The
process can be described as a three-step process, i.e. overview, review and discussion
(MacDonald, 1998). In the overview, designers present a design brief to reviewers,
and tasks are assigned. Next, reviewers independently review the design. Reviewers’
work is guided by questionnaires, provided by the authors in order to ensure a
thorough review. Then, each inspector meets with the author individually and
discusses the inspection results. Unlike passively reading through the artifact,
questions may require reviewers to take an active role in the inspection process such
39
as providing justification for their review decisions or writing programs for the
design specification being inspected. In order to gain optimal benefits, reviewers are
required to have specialized skills and knowledge significant to the content to be
reviewed. This allows important and subtle defects to be uncovered. In order to
achieve maximal inspection coverage, each reviewer has a different area of expertise.
The inspection ends when the reviewers and the designers agree upon the issues and
the designers make appropriate changes to the design documents.
2.3.5.10 MEETINGLESS INSPECTION
Meetings are considered an essential part of successful or effective inspection. Some
have argued that the primary benefit of meetings, particularly the collection
meetings, is that new faults, undetected by individuals working separately, may be
found (Fagan, 1976; Wheeler, 96). This is commonly referred to as the synergy of
meeting. Hence, inspection is believed to be more effective because of these
meetings and defects uncovered would justify their cost. Secondary benefits of
meetings include:
• General features of face-to-face meetings, e.g. team building, trust, group
communication, etc.
• Aspects of development or inspection can be elaborated in detailed
• Administrative or managerial issues may be conducted
• Education of novice participants by observation
Face-to-face meetings have been proven costly in terms of coordination overhead
(schedule conflicts) resulting in additional time required to complete the inspection,
and consequently the overall task. This is particularly true for large software
development projects. Delays due to inspection meeting scheduling can lengthen
development time. Meeting-based review has been shown to add approximately 20%
additional overhead cost (Russel, 1991; Ballman & Votta, 1994). Parnas and Weiss
(1989) argue that meetings make inspection unfocused and interrupt developers’
schedules. Votta (1993) found that scheduling inspection meetings lengthens the
inspection interval by almost one third. Humphrey (1989) found that the majority of
defects are found during preparation and meetings increase the development time
40
significantly affecting the cost of the product. “Meeting-gain” is defined as new
fault(s) detected during the detection (collection) meeting, i.e. by collective effort,
so-called synergy. Fagan Inspection advocates this benefit of the defect collection
meeting. “Meeting-loss” happens when individually detected fault(s) are not
subsequently recorded during the meeting. The net benefit of fault collection
meetings result from the combination of meeting-gain and meeting-loss. Votta
(1993) investigated the usefulness of inspection meetings in a working environment
at AT&T. The number of faults discovered by individual efforts was compared with
those discovered during team efforts, i.e. collection meetings. It was found that
meeting gain is approximately 5% (i.e. 95% of the total number of defects have been
already discovered during individual inspections). A number of studies also show
similar results. Porter, Siy, et al. (1995) reported meeting gains of 33%. They found,
however, that there were some cases where meeting-gains were as high as 80%.
Large variations in reported meeting-gains from inspection meetings led to a series
of controlled experiments being conducted to get more precise results.
Empirical results indicate that meetingless inspection is more effective than meeting-
based inspection. Porter and Johnson (1997) point out similar results of two
separately conducted controlled experiments one of which included professional
software developers as subjects. These studies suggest that meetings are not
necessary for making inspections successful or effective. Table 7 summarizes the
inspection methods discussed in this chapter.
41
Inspection
Method
Team
Size
Multiple
Sessions
Meeting Detection
Method
Fagan Large No Yes Ad-hoc
FTARM Large No Optional Ad-hoc / Checklist
Gilb Large No Yes Checklist
Two-person Small No Yes Ad-hoc
Phased Small Yes Yes Checklist
N-fold Small Yes Yes Ad-hoc
ADR Small Yes Yes Ad-hoc
Meetingless Large No No Any
Table 7 Comparison of Inspection Techniques
Meetingless inspection is suggested as an alternative to avoid the problems caused by
collection meetings. The Active Design Review (ADR) method is the first inspection
technique to address the weaknesses of meeting-based inspections. In ADR, there is
no large meeting where all participants meet to do defect collection. Instead, each
reviewer performs inspection systematically, i.e. guided by questionnaires posted by
the author of the design, and meets with the author individually to give feedback.
Votta (1993) suggests that using depositions, where the moderator meets with each
reviewer, after individual analysis, can eliminate disadvantages of inspection
meetings. McCarthy (1996) proposed a Detection-Detection inspection method that
uses multiple individual inspection efforts without a group meeting. Others also
report the success of asynchronous computer-support inspection in which no (face-
to-face) meeting is held. In summary, most defects are discovered during the
preparation step, or individual inspection (analysis). Successful inspection can be
carried out without a meeting, i.e. meeting is not an essential part of software
inspection as Fagan indicated.
42
Figure 14 Summary of activities within different software inspection techniques
(MacDonald, 1998)
43
Figure 15 Summary of Inspection tasks
44
2.4 COMPUTER SUPPORT IN SOFTWARE INSPECTION
2.4.1 PROBLEMS IN CONVENTIONAL METHODS
Conventional software inspection activities are manual and labor intensive. In the
preparation phase, the required documents are created. They are then distributed to
all participants. Participants must meet and should be assigned roles. After work
begins, the moderators must keep track of activities and roles of each participant.
During the individual inspection process, inspectors read through documents and
guidelines to discover defects and make annotations. Defects found, as well as other
related information, including their locations, defect types, source of defects, and
numbers of defects, must be recorded.
A meeting schedule must be made before the defect collection meeting can be held.
This can be an intricate task and may require many contacts with each participant to
resolve scheduling conflicts. In the collection meeting, a master defect list must be
created and individual defect lists must be updated. After the meeting is over, a
master defect list is completed and an action plan is created.
These documents are then distributed to the authors of the artifacts inspected and
others. The authors use them to remove defects. The moderator must keep track of
progress. Another meeting may be held to follow up the final results. Then the final
reports are produced and distributed. It is possible that tasks may be performed
repetitively, such as making multiple copies of documents, copying annotations from
one place to another place, making changes to multiple documents and distributing
correction documents. These kinds of processes are costly and can increase the
chance of errors.
The defect collection meeting is a face to face meeting, which suffers from the
typical weaknesses of face to face meetings. Inspection statistics, such as defects
plotted against inspected material (code modules) or authors, and inspection rates,
are necessary to improve the inspection process. These statistics are normally
45
calculated manually. In large development projects, an inspection history may be
needed. Searching through documents is an inefficient way to produce this history
and can lead to an overall decrease in productivity.
2.4.2 COMPUTER SUPPORT
There have been a number of commercial products and academic prototypes that
attempt to provide computer support for software inspection. Computer systems
ranging from simple database management systems to interactively integrated
systems have been developed. In a comparative analysis of computer supported
software inspection four important areas are described in need of support for
software inspection. These areas are document handling, individual preparation,
meeting support, and data collection. Each of these is described below.
2.4.2.1 Document handling
The benefits of electronic document handling include:
• access by multiple users without keeping multiple copies
• latest version of document is readily available
• documents can be browsed on-line, allowing anywhere-anytime access
• cross-referencing between documents can be created more easily
• document presentation can be tailored for individual needs
• coordinated document manipulations are supported, such as creation,
modification, distribution and deletion
• comments/annotations can be stored on-line and available for all participants
• loss of documents can be minimized (compared to multiple pieces of paper-
based documents)
• safeguards can be enforced (through authorization requirement and backup
copies)
• all issues addressed by participants are posted (reducing missing issues)
46
2.4.2.2 Individual preparation
Individual preparation can benefit from computer support in various ways, including:
• reviewing artifacts for defects, computer support by using tools such as
source code profilers can help discover certain types of defects
• inspectors can easily access checklists and other documents for referencing
• on-line checklists can be enforced by the tool to prevent incomplete defects
checking
• comments/annotations can be kept on-line allowing quick access and data
processing
• inspectors can concentrate more on difficult (inspection) tasks while certain
tasks are off-loaded to computers (agents)
2.4.2.3 Meeting support
According to Fagan (1976) the defect collection meeting should not exceed two
hours. This implies that a number of meetings must be held before the inspection is
finished; hence, meeting support becomes an important issue. While not specific to
defect collection meetings the potential benefits of electronic meeting support
include:
• allowing (human) parallel processing
• support and automation of meeting activities, such as brainstorming, idea
organization, balloting, and annotation
• providing equal opportunity
• reducing meeting time
• offering access to external information
• increasing process gains and reducing process losses
• anonymity
• In a distributed environment, asynchronous inspection support can eliminate
the need for synchronous meetings.
47
2.4.2.4 Data collection
Conventionally, data is collected manually. This can be time consuming and error-
prone. The inspection process suffers if inaccurate data is collected. Weller (1993)
pointed out that the manual effort required to collect (accurate) data was minimized.
Computer support allows:
• inspection metrics to be collected automatically
• data to be used for controlling the inspection process, for example the
moderator can monitor inspection progress of each participant and decide to
exclude those who have not spent enough time in preparation from
participating in the defect collection meeting
• detailed data analysis to be performed, once data is readily available,
providing additional useful information
Some computer support for software inspection provides automatic data collection
capability.
2.4.3 COMPUTER SUPPORT TOOLS
A wide category of computer support for software inspection can be described in
terms of system features. MacDonald (1995) suggests system features including:
document support, annotation support, automatic defect detection, checklists,
enforcement, meeting support, and decision support and metrics collection. There are
a number of implicit requirements for computer support for inspection, including
multiple user communication and coordination support. However, from software
inspection point of view, too much specific computer support is required to support
the software inspection process. At the same time, there are support mechanisms,
such as scheduling systems, document databases, hypertext systems, electronic
annotations, visualizations, etc, that are not specific to the software inspection
process.
48
There are three general approaches in adopting computer support for software
inspection. First, tools that provide the supporting features can enhance the
inspection process by facilitating particular inspection activities. In this approach,
generally available tools, such as generic project management, document
management, electronic meeting support, and electronic mail systems, can support
software inspection. Employing this approach, conventionally manual processes are
supported by electronic means, partially or totally. For example, documents and
checklists are stored in the system and can be retrieved by all participants. Defect
data entered into a database system can be traced and processed. Meetings are
scheduled with computer assistance. Electronic meeting systems are used during
defect-logging. Hypertext technologies enhance document database and user
interactions. Inspection processes can be streamlined resulting in better efficiency.
A second approach to computer support for software inspection seeks to improve the
inspection process through integrated computer support. FTArm and Phased
Inspection techniques were developed with total dependence on computer
technologies. Using FTArm allows software inspection to be conducted without a
meeting. Incorporating new tools as part of the process leads to a modified process,
and such a process leads to process improvement. Computer-mediated asynchronous
software inspection such as HyperCode Inspection Process [Perpich, 1997] can
streamline the inspection process.
The third approach to computer support for software inspection is the use of tools
specifically tailored to optimally support software inspection activities. These tools
tend to introduce new interfaces to users. GUI’s allows different media to be
displayed. Multiple-window systems allow annotations and materials to be in relation
to each other. Defect entry interfaces may be augmented with defect classification
tools so that every defect is classified once it is discovered. This is to ensure that no
required information is missing. Information displayed in a window is organized and
color-coded such that certain inspection activities are optimized, e.g. different types
of defects are shown in different colors. With process modeling capabilities
(MacDonald & Miller, 1997), computer tools can flexibly support multiple types of
49
inspection processes. These inventions lead to improvement in the efficiency and
effectiveness of the inspection. This approach may be considered as process and
interaction optimization. Different activities benefit from different types of computer
support so that it is possible to combine different approaches. Available technologies,
success stories in other application areas (such as communication, collaboration, and
authoring support), and standards greatly influence how such tools are designed and
realized. Changing requirements, such as geographic separation of development
teams, the need for integrated software development environments and automation,
also impact what kind of computer support is needed.
Finally, different organizations may have different preferences related to details of
the software inspection process. All of these factors play a role in the development of
an optimal computer supported software inspection system.
2.4.4 ASYNCHRONOUS SOFTWARE INSPECTION
Although different inspection processes have been developed, the basic process has
remained unchanged since Fagan first defined it in 1976. Inspection is composed of
series of coordinated activities. These activities include meetings, which all
participants must attend. Throughout software development, each inspection meeting
typically covers a manageable (small) portion of the whole software artifact. As a
result, for a large project, a large number of meetings are required. This makes
inspection the bottleneck of the software engineering process. To make software
inspection more efficient, synchronous activities, such as inspection meetings, must
be removed or replaced by more efficient alternatives, such as meetingless (or
asynchronous) inspection. With advances in computer technology, asynchronous
inspection becomes more efficient and effective. For example, the inspection process
can be improved by reducing the number of steps in the inspection process, allowing
inspector preparation, defect collection, defect resolution, and defect repair to be
performed concurrently.
Furthermore, software inspection gains several benefits from asynchronous
50
activities, such as flexible schedules, distributed inspection teams, decreased
inspection cost, better computer resource utilization, etc. There are disadvantages
when synchronous activities are eliminated. Nunamaker Dennis (1991) identify
several of these:
• Serial thread of execution
• Free riding
• Information overload
• Domination
• Coordination problems
• Socializing
• etc.
There are software inspection tools that support both synchronous and asynchronous
activities, such as Collaborative Software Review System (CSRS), Notes Inspector,
Asynchronous Inspector of Software Artifacts (AISA), InspectA, Asynchronous/
Synchronous Software Inspection Support Tool (ASSIST), HyperCode System
(WWW-based Software Inspection), and Web Inspection Prototype (WiP). In the
next sections, these tools are described in detail, and finally compared with each
other in the means of inspection functions, support functions, and availability, in
Table 8.
2.4.4.1 Collaborative Software Review System (CSRS)
CSRS (Johnson, 1994) is a computer mediated collaborative software review that
was designed to support FTArm (Formal Technical Asynchronous review method).
FTArm consists of series of activities: setup, orientation, private review, public
review, consolidation, and group review meeting. CSRS is a hypertext system with
business logic for controlling the inspection process. Each participant must inspect
(read) all nodes of the artifact and may create comment nodes. All private comment
nodes will become public nodes for reviewing and voting. The system keeps track of
51
the status of each node and controls whether inspection can proceed to the next
phase. The inspection is complete when there are no issues to be resolved. Review
and group decision-making activities are carried out asynchronously for the most
part. An e-mail notification feature is available for informing reviewers about newly
created (document or comment) nodes that need attention. FTArm’s process
modeling capability also allows other inspection techniques to be supported.
2.4.4.2 Notes Inspector
Notes Inspector (Thompsom & Riedl, 1995) uses a commercially available
groupware environment, Lotus Notes, as an asynchronous software inspection tool.
Inspection begins with making documents available in the system. Inspectors can
read documents and create annotations. Annotations are displayed symbolically
inside the documents. A defect collection meeting is conducted asynchronously via a
group discussion facility. Discussion threads lead to an agreement on an issue.
The discussion ends or the new issue begins with a voting mechanism. All
participants vote whether to reject or accept an issue. Lotus Notes provides built-in
features for supporting such activities so that the look-and-feel remains unchanged to
Lotus Notes users. Notes Inspector does not collect additional information to support
the inspection process.
2.4.4.3 Asynchronous Inspector of Software Artifacts (AISA)
AISA (Stein, 1997) is a WWW version of CSI, CAIS, and Notes Inspector. It was
originally implemented using the Mosaic WWW client. WWW technology was
chosen due to its popularity, familiarity and flexibility. The system handles both
textual and graphical documents, such as entity-relationship diagrams. AISA is
structured to allow inspection of graphical objects in a distributed environment.
Graphical documents are structured such that they can be viewed in hierarchical
fashion, i.e. there is a zoom operation. The user interface contains buttons offering
different functions such as annotation and listing. Hypertext links to help documents
are also provided. HTML template is used to allow defects to be entered along with
52
its classification and other information. The inspection process follows initialization,
preparation, discussion, and post-discussion phases. Asynchronous meetings are
conducted through discussion threads using the WWW client interface. Participants
can vote to accept or reject defects proposed by others. A defect summary report is
finally produced by the system.
2.4.4.4 InspectA
InspectA was developed by Murphy and Miller (1997) who advocates asynchronous
software inspection through computer supported communication. It claims to
overcome some weaknesses of AISA, a similar prototype. Messaging (e-mail) is the
only mechanism for supporting group processes. The moderator initially creates a list
of participants and documentation. Team members are notified and receive copies of
the documents through e-mail. InspectA provides user interfaces that help in
presenting messages in organized structures facilitating the inspection process.
During the inspection phase, each inspector is presented with four screens displaying
product document, source document, checklist, and defect entry form. Each inspector
reviews the product document and creates private comments, regarding defects
discovered, each of which has additional information associated, including defect
classification – Missing, Wrong, and Severity (Major or Minor). Later, all privately
created comments are merged into a list of comments that will be circulated among
team members. Then, discussion is started as threads of messages that are exchanged
asynchronously. New comments can be added to the list. The final defect list is
collated and produced by the moderator. Synchronous meeting support is omitted
from InspectA.
2.4.4.5 Asynchronous/Synchronous Software Inspection Support Tool (ASSIST)
ASSIST, developed by MacDonald and Miller (1997), was developed to be a flexible
software inspection tool that overcomes weaknesses of other inspection tools. Its key
feature is the process modeling capability allowing different software inspection
techniques to be used for controlling the inspection process. Both synchronous and
asynchronous activities are supported. With an open architecture, different types of
53
documents can be stored and viewed in the system. A simple browser is used for on-
line viewing of the documents required for inspection, i.e. checklists and
specifications. A code browser allows source code to be viewed and annotations to
be made. Users can view annotations, representing defects discovered, via a list
browser which also displays defect classification and votes associated with each
annotation. ASSIST employs several techniques that support group meetings. The
current focus of the meeting (the artifact under consideration) is controlled via a
shared browser. Conferencing tools, such as the MBone video tool, audio tool and
whiteboard tool, are provided. In addition, the list browser allows additional
comments to be added to an existing annotation. The discussion threads form group
meetings in asynchronous fashion. Useful information is automatically collected and
can be reported. MacDonald & Miller (1998), conducted an empirical study, which
observed the benefits of ASSIST (tool-based) inspection over paper-based
inspection.
2.4.4.6 HyperCode System (WWW-based Software Inspection)
Using the Common Gateway Interface (CGI) extensions to HTTP, the HyperCode
system, based on the HyperCode Inspection Process, allows inspectors to use WWW
browsers as a front-end for performing code inspection [Perpich, 1997]. Web
browsers provide a standard user interface for physically distributed participants. The
user interface is divided into frames. The largest frame displays line-numbered
source code with any hypertext links to annotations; the viewed annotation is
displayed at the bottom of the screen. All participants can view all annotations and a
list of annotations maintained by the system is available for viewing. Porter (1998)
has extended the HyperCode system to become a web-based workflow system
allowing better support for the inspection process.
2.4.4.7 Web Inspection Prototype (WiP)
Harjumma and Tervonen (1998) pointed out possible benefits of next generation
WWW technology to software inspection. These include widespread client
availability, platform-independence, add-on technology such as Java, and hypertext
54
capabilities. WiP is developed as a testbed for examining the use of WWW
technology for a collaborative inspection environment. The inspection method in
WiP is based on the Goal-Rule- Checklist-Metric (GRCM) quality model (Tervonen,
1996). This inspection process model includes setup, individual inspection, peer
inspection, public inspection, group inspection, consolidation and reporting. WiP is
the first WWW-based asynchronous software inspection tool that employs Java
technology (applets).
In the setup phase, documents are imported into the system and roles are assigned to
participants. Quality goals must also be specified. Some information is automatically
set up by the system such as hypertext links to inspectors’ homepages. Inspectors are
required to log-in before performing inspection. Each document to be inspected is
presented as a dynamically created HTML page. The document is displayed to the
user through a Java applet offering a graphical user interface that includes
highlighting and command buttons. An inspector can request related documents, e.g.
help documents and checklists. The checklists can be tailored to user’s preferences,
such as project quality goals or user roles. WiP’s annotation tool allows users to
make comments associated with any line of the inspected document. Each comment
can be classified as Major, Minor or a Question.
In the public inspection phase, comments are merged into a single HTML document,
available to all team members, with links to descriptions. The system provides
capabilities to generate dynamic inspection reports that summarize the statistics
about the inspection process. Statistics include time spent on inspection, reading rate,
the numbers of issues created and by whom, and a list of inspectors who have visited
each document. This allows the process to be continuously monitored and inspection
performance to be improved. The reports also include information for consolidation.
The benefits of WiP include on-line commenting and documentation, a familiar
(WWW) interface, integrated interface, and simple reports. There are some
weaknesses such as non-modifiable comments, an unstructured interface, very little
report information, and lack of security features. In summary, WiP
55
demonstrates distributed software inspection through WWW technology.
2.4.4.8 FEATURE-BASED CLASSIFICATION OF SOFTWARE INSPECTIONS
Siy (1996) identified six attributes of software inspection methodologies: (1) team
size, (2) number of sessions, (3) coordination between multiple sessions, (4)
collection technique, (5) defect detection method, and (6) use of post-collection
feedback.
Computer supported software inspection can be distinguished in terms of features
provided by the tools. MacDonald (1998) summarizes a list of features of software
inspection tools, including linked annotation, defect classification, cross-referencing,
automated analysis, checklists, supporting material, distributed meetings, decision
support, and data collection. The features can be classified into three broad
categories: inspection functions, support functions, and availability. Inspection
functions are features that support key activities of the inspection process. Support
functions are features that facilitate information access and group processes during
inspection. Availability of the tools includes multimedia support and cross-platform
capability. Table 8 summarizes which tools to use for the selected features of
software inspection tools.
56
Table 8 List of selected features in computer supported software inspection
57
3 DEVELOPING A NEW INSPECTION METHODOLOGY
3.1 ROLES AND RESPONSIBILITIES
Three important questions practitioners usually have about roles in software
inspections are (1) what roles are involved in an inspection, (2) how many people are
assigned to each role, and (3) how people should be selected for each role. For the
first question, a number of specific roles are assigned to inspection participants.
Hence, each inspection participant has a clear and specific responsibility. The roles
and their responsibilities are described below. There is not much disagreement
regarding the definition of inspection roles. In the following, each of these roles are
described in more detail:
Organizer
The organizer plans all inspection activities within a project or even across projects.
Moderator
The moderator ensures that inspection procedures are followed and that team
members perform their responsibilities for each phase. He or she moderates the
inspection meeting if there is one. In this case, the moderator is the key person in a
successful inspection as he or she manages the inspection team and must offer
leadership. Special training for this role is suggested.
Inspector
Inspectors are responsible for detecting defects in the target software product.
Usually all team members can be assumed to be inspectors, regardless of their other
roles in the inspection team.
Reader/Presenter
If an inspection meeting is performed, the reader will present the inspected products
at an appropriate pace and lead the team through the material in a complete and
logical fashion. The material should be paraphrased at a suitable rate for detailed
58
examination. Paraphrasing means that the reader should explain and interpret the
material rather than reading it literally.
Author
The author has developed the inspected product and is responsible for the correction
of defects during rework. During an inspection meeting, he or she addresses specific
questions the reader is not able to answer. The author must not serve as moderator,
reader, or recorder.
Recorder
The recorder is responsible for logging all defects in an inspection defect list during
the inspection meeting.
Collector
The collector collects the defects found by the inspectors if there is no inspection
meeting.
3.2 HYBRID INSPECTION PROCESS OVERVIEW
The hybrid inspection method is the combination of several inspection methods like
GILB, Humphrey and Fagan’s style and this is the improved version of the
inspection method that was being used in the industry in the previous years. Not all
features of one specific method, but different specifications that are more useful were
taken from these methods. However, the common property that is inherited from
many methods is being asynchronous (participants are not required to assemble
together at the same time and place) and having a meeting with reviewers, author and
the quality engineer.
Implemented method is based on reviewers’ effort. The ways for high quality
product are investigated while minimizing the inspection costs. In the previous years,
a method that has a cycle as prepare package, send package to reviewers, review
package by noting anomalies on a paper or document, organize meeting, perform
59
meeting and take action items, was being applied. During the meeting, all the
comments and anomalies were being gathered from each reviewer one by one. Then
a discussion was taking place on each comment of each reviewer. This was
increasing the meeting effort since generally 4-6 reviewers participate in each review
and reluctantly the meeting time was being multiplied with the number of reviewers.
This effort could sometimes be increased up to 10-15 hours per one page. This
method has advantages and disadvantages. Product is being reviewed well and gets
higher quality; this is the advantage. However spending more effort is the
disadvantage of this method. The cost of inspection effort can be higher than solving
the problems after releasing the product.
Currently an updated and improved version of that poor inspection method is being
used by automating the review and meeting processes. A tool is developed for
getting reviewer comments. While using the previous method, a package was being
prepared then this package was being sent to reviewers for review. The difference
between these two methods can be observed at this point. Reviewers review the item
but not the entire product, only the parts they are responsible from. This is one of the
new features in the new method. The product is divided into many responsibilities
and be shared between reviewers. Reviewers review the product and give their
comments to author and SQE via the new developed web-based tool. They write the
anomalies and comments one by one by selecting the location of the anomaly in the
tool for each anomaly. All of the reviewer comments are collected in anomaly
database that can be seen by only author and SQE. Author gives responses for each
comment by defining the decision of the anomaly as Agree, Disagree or Discuss with
the long description of response. Each reviewer can see the responses that are given
for his/her comments before the peer review meeting.
For the peer review meeting every reviewer come together in a meeting room. This is
the important point, because each reviewer should know about the decisions that are
taken about the product they reviewed. Otherwise this method would not be ethical
since other reviewers might not accept any comment that is given by one reviewer. In
the peer review meeting, action items are being taken from the comments that are
60
accepted by the author. Then the disagreed comments are being reviewed by making
a maximum 2 minutes long conversation between the author and the comment
owner. According to the result of this conversation an action item is being taken or
not. Finally the discuss comments are being reviewed again by making 2 minutes
conversation between the author, reviewer and the team leader. According to the
result of this conversation an action item is being taken or not.
In the peer review meeting, the comments are being reviewed, not the product
directly, therefore, time and resources are utilized in the most effective and efficient
way.
There are advantages and disadvantages of the hybrid inspection process. First of all,
the produced software products may have more defects. Defect removal efficiency
may reduce in comparison to previous method, and this is the disadvantage. But the
effort of inspection is reduced. So the cost of inspection is much more reduced in
comparison to the cost of correcting the defect later.
3.3 FORMALIZING THE INSPECTION PROCESS
The organization phase involves setup and orientation stages. The inspection
process begins when certain entry criterion are satisfied. Then, the team leader
selects participants and generates an inspection plan. In the setup stage, the
inspection team is selected and the document to be inspected is e-mailed to the
participants by the SQE. A sample e-mail can be as below.
Inspection team,
A peer review is planned for the Test Cases written for Marker Beacon and Attitude Background packages. Responsibilities are shared as stated below:
Attitude background
Author : Bulentt
Moderator : Osmany
Inspector : Tayland (All checklist Items)
Inspector : Gokhanb (All checklist Items)
Inspector : Bugras (All checklist Items)
Reader : Yusufa
61
Recorder: Mervec
Marker Beacon
Author : Yusufa
Moderator : Osmany
Inspector : Bulentt (All checklist Items)
Inspector : Selcuks (All checklist Items)
Inspector : Mervec| (All checklist Items)
Reader : Yusufa
Recorder: Tayland
Deadline for entering comment (Inspectors): 25 June 06, 10:00 Deadline for giving response (Author): 25 June 06, 14:00
The orientation stage is similar to the overview stage in Fagan’s inspection method
and involves preparation of the participants. Participants get the correct version of
the product and get related materials from the author.
The private review phase is same as preparation stage in Fagan’s inspection. Each
participant examines the document and enters their comments on a web-based tool.
Inspection is done according to the checklists appropriate for the inspected product,
like code review checklist, requirements review checklist or design review checklist.
Comments are not visible to the entire team, which may cause misguidance in the
comments or wrong inspection by other inspectors. Inspectors inform the author by
sending a warning mail that they finished entering their comments about the product.
In the control phase author checks the comments of the inspectors on the web-based
tool and write comments about the issue like agree, disagree, discuss. After that all
participants get together in the inspection meeting and go over all the items using
the tool to jointly inspect the target at a moderately fast pace. Agreed comments are
read but they are passed as fast as possible since the author and the inspector agree
on the comment. Comments disagreed by the author are discussed, disagree reason is
explained to the other inspectors. The discuss comments are discussed within the
inspection team and agreed on an action. These discussion sessions should be
performed in control in order not to increase the review time.
62
Once defects have been identified, the author of the product performs a rework over
the materials to correct them. The moderator performs follow-up to ensure that
every issue is addressed and every defect is corrected.
Figure 16 Phases in Developed Inspection model
3.4 PROCESS SEQUENCE OF THE HYBRID INSPECTION MODEL
• Author informs the SQE and team leader for the product that will be
inspected, and place the product under control with SCM tool.
• Team leader prepares the inspection team, reviewers, moderators, and
informs the SQE about the team members.
• SQE informs the inspection team about the date of the review and path of the
file that will be inspected in the controlled library.
• Reviewers enter their comments and issues they find via the web-based tool.
The comments and issues are entered such that the commented product and
its commented parts are clearly identified. All the comments and issues are
entered in a reasonable time before the start of the review meeting so that the
author can check the comments and give a comment like agree, disagree, or
discuss in the meeting.
• Reviewers inform the author by the help of the tool, that he or she has
finished commenting on the product. Then author immediately starts reading
the entered comments before inspection meeting.
- Setup - Orientation
- Private Review - Control Review
- Edit - Follow-Up - Exit
- Inspection meeting
63
• Reviewing the comments before the inspection meeting will help the author
to analyze and examine the comments at his desktop. He or she can have time
and chance to check the other related technical materials or documents from
his desk in order to agree or disagree on the comments.
• With the information gathered before review meeting about comments, in the
meeting the comments that the author agreed are not discussed again. The
issues that the author disagreed or commented like discuss, are discussed
among the inspection team.
• Action items are recorded and delivered to SQE. Then SQE mails the action
items to the author, author updates the product according to the action items
and takes notes next to the action item that explains what changes are done
and where they are done.
• This form is again mailed to the SQE and the final product is controlled by
the SQE according to action items and their explanations.
3.5 ADVANTAGES/DISADVANTAGES OF THE MODEL
Advantages:
• Just “disagreed” comments are discussed
• Less time is spent on the comments
• Author is informed as soon as the reviewer finishes his comments
• The author gives more accurate answers to comments. Every single comment
is analyzed by the author
• Total review and meeting time is reduced, people resource is saved.
• Paper usage is reduced
Disadvantages:
• No brain storming
• Domination
64
• Coordination problems
• Information overload
3.6 A SIMPLE TOOL FOR THE INSPECTION METHOD
3.6.1 Overview
In the overall process there is a need to use a tool to support the inspection process.
So, a simple tool is written to complete the introduced process. Tool is developed
with PHP, MySQL, and Apache Server.
3.6.2 Use Case Diagrams
In this section, the primary elements and processes that form the system are
identified. The primary elements are termed as “actors”, and the processes are termed
as “use cases”.
The following diagrams show which actor interacts with each use cases. As shown in
the Diagram 1, Reviewer is the actor and the use cases of the actor are listed.
Diagram 1: Reviewer
• Reviewer adds a comment for the specific part of the product
• Reviewer lists the comment that he has entered for the specific product
• Reviewer deletes a selected comment form the list of comments window
• Reviewer informs the author that he/she finished entering comments.
65
The other actor of the system is the Author. In Diagram 2, the use cases of the
Author are listed.
Diagram 2: Author
• Author responds to the comments from the list of comments page
• Author lists the comments for his product
• Author adds the agreed comments to the Action Item Form
3.6.3 Classes
In the previous section, the primary actors and use cases are identified by using the
use case diagrams. Apart from those, the classes and the relationships of the system
are identified according to the importance of their roles in the system. Since the
classes reflect the important entities, at the first glance, the actors specified in section
3.6.2 appear to be potential classes. Instead of creating two different classes for the
specified actors, a class “User” is created. The list of classes in the system appears to
be:
• User
• Product
• Project
• Comment
• Data
66
Enumeration: User Type
Class: User
Class: Product
Class: Project
67
Class: Comment
Class: Data
3.6.4 Class Diagram
Class Relations:
The above diagram defines the design of the developed web-based tool by the help of
the relationships between the specified classes. The relationship between the classes
Project and Product points out that; a project has 0 or more products. But if there is at
least one product in the medium, then it has to be related with a project. Likewise,
the relationship between the classes Product and User specifies that; a product has 0
68
or more users. But if there is at least a user in the medium, then the user has to use at
most one product. A user can enter 0 or more comments. And, if there appears at
least a comment, then it has to be entered by a user. A product has 0 or more
comments. If there is at least one comment, then it has to be related with a product.
Sequence Diagrams:
Reviewer:
The Reviewer who is an actor invokes the Select functionality in order to select a
project. After the project is selected, the getProducts functionality of the Project class
is invoked in order to list the products under the project with the specified project
identification number. After the information of product and project identification
numbers are gathered, reviewer can add comments about the product. Finally, the
reviewer invokes the listComment functionality to list all the comments.
69
Author:
The Author who is an actor invokes the Select functionality in order to select a
project. After the project is selected, the getProducts functionality of the Project class
is invoked with the project identification number parameter, in order to list the
products under the specified project. After the information of product and project
identification numbers are gathered, the author invokes the listComment
functionality to list all comments. Finally, the author invokes the ResponseComment
functionality to give responses to the comments.
70
3.6.5 Usage of the Inspection TOOL
When first executed the login page is displayed to the user as in Figure 17. User
enters his user name, such as author or reviewer, according to his participation
responsibility in the review meeting.
Figure 17 Login Page
71
Figure 18 Project and Product Selection Page
After login, user selects the project and the product name. According to the user type,
if he is a reviewer, he will be directed to the comment entrance and listing page or if
he is an author he will be directed to the list of comments that are given on the
selected product.
72
Figure 19 Comment Entrance Page
Reviewer can enter his comments as shown in Figure 19; in the location part,
reviewer should give information that his/her comments are related to which part of
the product (section, line number, step number, paragraph number or name).In the
comment part, he will enter his comments, he can also select a priority for his
comment, and can select the Check List item number after viewing the Check List as
shown in Figure 20.
73
Figure 20 Check List
74
Reviewer can also be able to list his comments to see as a whole as shown in Figure
21.
Figure 21 List Of Comments Page (reviewer’s)
After reviewer entered all the comments, by pressing the “Inform the Author” button,
he informs the author that he has finished entering all his comments. Author gets this
information by mail and he logins to the tool. He can list and see all the comments
from the screen as shown in the Figure 22. He can give his response to the comments
like agree, disagree, discuss or other information that explains what actions he will
do as shown in Figure 23. Then, in the review meeting participants go through the
comments, and comments agreed by the author is just read and not discussed. Other
issues that need to be discussed are also tried to be solved or a new action is taken as
fast as possible. The author finally adds the comments, which the author and all the
inspection team agree to the action item form. The action item form appears as
shown in Figure 24.
75
Figure 22 Review Comments Page (author’s)
Figure 23 Add Author’s Comment Page
76
Figure 24 Action Item Form
77
4 CONCLUSION
According to the studies performed by myself as a Software Test Engineer with the
Software Quality Assurance team at AYESAS (Aydın Yazılım ve Elektronik Sanayi
A.Ş., located at ODTU Technopolis), by using the hybrid inspection process
described in this thesis, the total time spent by the reviewers is decreased with
respect to other methods, and the usage of the sources is improved as described in
more detail in section 3.2. By submitting comments with the help of the tool on an
electronic environment, the speed of the comment entering process is increased and it
is ensured that the author and participants read every comment carefully. Also the
usage of hardcopies is minimized. Another important improvement is to enable the
author to review the comments before inspection meeting. In this way author is able
to study and check the correctness of the comments in detail when required from
his/her desktop, so that he can be able to agree or disagree with the comment safely.
This practice also prevents the wrong or misunderstanding in communication
between the participants that can be encountered in classical review techniques. It
also prevents the need of referring to other documents in the review that may
increase the total review time. Since most of the agreed comments passed rapidly,
this process also decreases the total review time.
As described in section 2.4.3, MacDonald (1995) summarizes a list of features of
software inspection tools, which can be classified into three broad categories:
inspection functions, support functions, and availability. Table 9 compares recent
tools with the introduced web based software inspection tool in this study called “T”
according to the selected features of software inspection tools. As it can be seen in
the table, this tool is developed and still in the process of improvement to provide
most of the features of the recent tools that could benefit the hybrid inspection
process.
Work on the web-based tool that has been developed to visualize the hybrid
inspection process is continuing within a large-scale software project at AYESAS
(Aydın Yazılım ve Elektronik Sanayi A.Ş., located at ODTU Technopolis) in regard
to process control and document management. New features about these topics and
version of the tool information will be added to improve the maturity of the tool.
78
Furthermore, as a future work, the tool may also be improved to provide real-time
videoconferencing or audio conferencing for a team in a distributed environment.
In addition, the proposed hybrid inspection methodology and the developed web
based tool can be experimented on different structures (such as large or small teams;
single or multiple teams working in parallel or in sequence) concerning the
inspection effort (person hours), inspection interval (working days), defect density,
and repair statistics in order to improve the process of the methodology or to find out
and add new features to the tool. Different reviewer characteristics such as inspection
experience, (programming) language familiarity, and application experience should
also be considered in these experiments.
Moreover, a simulator could be designed and developed which behaves or operates
like the web based software inspection tool introduced in this study and any of the
recent tools such as InspeQ, CSI, Scrutiny, ASIST, etc. By this way, these tools
could be compared with each other with respect to the efficiency and effectiveness of
the software inspection.
79
Table 9 Comparison between the proposed tool “T” and the recent tools
80
5 REFERENCES
[Gilb, 93] Tom Gilb, Dorothy Graham: Software Inspection; Addison
Wesley 1993
[Fagan, 86] M. E. Fagan. Advances in Software Inspections. IEEE
Transactions on Software Engineering, 1986.
[Fagan, 76] Fagan, M. (1976). Design and code inspections to reduce errors
in program development. IBM System Journal 15(3).
[Humphrey, 89] Humphrey, W. (1989). Managing the Software Process. NY:
Addison Wesley.
[Ackerman, 83] A.F. Ackerman, P.J. Fowler and R.G. Ebenau, Software
inspections and the industrial production of software, in Hans-
Ludwig Hausen (Ed.),
Software validation: Inspection, testing, verification,
alternatives, Proceedings of the Symposium on Software
Validation, Darmstadt, FRG, 1983.
[Ackerman, 89] Ackerman, A.F., Buchwald, L.S., & Lewski, F.H. (1989).
Software inspections: an effective verification process. IEEE
Software.
[Bisant & Lyle, 89]
Bisant, D.B., & Lyle, J.R. (1989). A two-person inspection
method to improve programming productivity. IEEE
Transactions on Software Engineering.
[Boehm, 81] Barry Boehm, Software Engineering Economics, Prentice
Hall, 1981.
81
[Boehm, 87] B. W. Boehm, Improving Software Productivity, Computer, Vol.
20.
[Collofello, 89] J. S. Collofello and S. N. Woodfield, Evaluating the
Effectiveness of reliability-assurance techniques. Journal of
Systems and Software, 1989.
[Conradi, 99] R. Conradi, A. S. Marjara and B. Skatevik, Empirical Studies of
Inspection and Test Data, 1999.
[Daly, 77] Daly, E.B. (1977). Management of Software Development.
IEEE Transaction on Software Engineering.
[Freedman&Weinberg,
82]
Freedman, D. P., & Weinberg, G.M. (1982). Handbook of
Walkthrough, Inspections and Technical Reviews. Boston, MA:
Little, Brown and Company.
[Fruhauf, 96] Fruhauf, K. (1996). Ten obstacles inspections face in an
organisation.
[Gilb, 88]
Tom Gilb, Susannah Finzi: Principles of Software Engineering
Management, Addison-Wesley 1988
[Grandy, 94] R. B. Grady and T. Van Slack, Key Lessons in Achieving
Widespread Inspection Use. IEEE Software, 1994.
[Hollocker, 87] C.P. Hollocker, The standardization of software reviews and
audits, Van Nostrand-Reinhold Company, NY, 1987.
[Hops, 92] Kelly, J., Sherif, J., & Hops, J. (1992). An analysis of defect
densities found during software inspections. Journal of Systems
and Software 17(2).
82
[Johnson, 93]
Philip M. Johnson and Danu Tjahjono. CSRS users guide,
University of Hawaii, 1993.
[Johnson, 94] Johnson, P. (1994). An instrumented approach to improving
software quality through formal technical review. Los Alamitos,
CA: IEEE CS Press.
[Kan, 95] S. H. Kan, Metrics and Models in Software Quality
Engineering. Addison-Wesley, 1995.
[Kaner, 98] C. Kaner, The Performance of the N-Fold Requirement
Inspection Method, Requirements Engineering Journal, vol. 2,
1998.
[Nunamaker, 91] Nunamaker, J. F., A. R. Dennis, et al. (1991). “Electronic
meeting systems to support group work.” Communications of
ACM 33(2): 40-61.
[Kelly, 92] J. C. Kelly, J. S. Sherif and J. Hops, An Analysis of Defect
Densities found during Software Inspections. Journal of Systems
and Software, 1992.
[Knight&Myers, 93] Knight, J.C., & Myers, E.A. (1993). An improved inspection
technique. Communications of ACM..
[MacDonald & Miller,
98]
Macdonald, F., & Miller, J. (1998). A comparison of tool-based
and paper-based software inspection. Empirical Software
Engineering.
[MacDonald, 98] MacDonald, F. (1998). Computer Supported Software
Inspection. Department of Computer Science, University of
Strathclyde, UK.
[MacDonald & Miller, MacDonald, F. and J. Miller (1997). Automated generic support
83
97] for software inspection.
10th International Quality Week.
[MacDonald & Miller,
95]
MacDonald, F (1995). A review of tool support for software
inspection.
Software Manage. Network, IEEE Comput. Soc. Press Los
Alamitos CA USA: 340-49.
[McGibbon96] T. McGibbon, A Business Case for Software Process
Improvement, 1996.
[Parnas&Weiss, 89] Parnas, D.W., & Weiss, D.M. (1989). Active design review,
IEEE CS Press.
[Perpich, 97] Perpich, J.M., Perry, D.E., & Porter, A.A. (1997). Anywhere,
anytime, code inspections: Using the Web to remove inspection
bottlenecks in large-scale software development.
[Porter, 97]
[Porter97] A. A. Porter and P. M. Johnson. Assessing software
review meetings: Results of a comparative analysis of two
experimental studies. IEEE Transactions on Software
Engineering, 1997.
[Porter Siy, 95] Porter, A., H. Siy (1995). “An experiment to assess the cost-
benefits of code inspections in large scale software
development.” SIGSOFT Software Engineering
Notes 20(4): 92-103.
[Porter, 98]
Porter, A. and L. G. Votta (1998). “Comparing detection
methods for software requirement inspections: A replication
using professional subjects.” Empirical Software Engineering
Journal 21(6): 355-79.
[McCarthy, 96] McCarthy, P., A. Porter, et al. (1996). An experiment to assess
cost-benefits of inspection meetings and their alternatives: A
pilot study. Proceeding of the International Metrics
Symposium, Berlin, IEEE CS Press: 100-11.
84
[Remus, 84] Remus, H. (1984). Integrated software validation in the view of
inspections/reviews.
[Rusell, 91] Russel, G.W. (1991). Experience with inspection in ultra large-
scale developments. IEEE Software.
[Schneider,
Martin&Tsai, 92]
Schneider, G.M., Martin, J., & Tsai, W.T. (1992). An
experimental study of fault detection in user requirements
documents. ACM Transaction on Software
Engineering.
[Standard for Software
Review and Audit,
88,97]
IEEE (1988). IEEE Standard for Software Reviews and Audits
(IEEE STD 1028-1988), IEEE Computer Society.
IEEE (1997). IEEE Standard for Software Reviews and Audits
(IEEE STD 1028-1997), IEEE Computer Society.
[Tripp, Struck, &
Pflung, 91]
Tripp, L., Struck, W., & Pflung, B. (1991). The application of
multiple team inspections on a safety-critical software standard.
[Weller93] E. F. Weller, Lessons from Three Years of Inspection Data.
IEEE Software, 1993.
[Wheeler,
Brykczynski, &
Meeson, 96]
Wheeler, D.A., Brykczynski, B, & Meeson, R.N. (1996).
Software Inspection: An Industry Best Practice. Los Alamitos,
CA: IEEE CS Press.
[Thompsom & Riedl,
95]
Thompsom, C. and J. Riedl (1995). Collaborative asynchronous
inspection of software using Lotus Notes. Technical Report 95-
047. Computer Science Department, University of Minnesota.
[Harjumaa &
Tervonen, 98]
Harjumaa, L. and I. Tervonen (1998). A WWW-based tool for
software inspection. Proceedings of the Thirty-first Hawaii
International Conference on System Sciences (HICSS'98), Vol.
85
3: 379-88.
[Tervonen, 96] Tervonen, I. (1996). “Support for quality-based design and
inspection.” IEEE Software
13(1): 44-54
[Siy, 96] Siy, H. (1996). Identifying the Mechanisms to Improve Code
Inspection Costs and Benefits. PhD Thesis. Department of
Computer Science. Colledge Park, University of Maryland,
USA.
[Stein, 97] Stein, M., J. Riedl (1997). A case study of distributed,
asynchronous software inspection. Proceeding of the 19th
International Conference on Software Engineering
(ICSE'97), ACM Press: 107-17.
[Ballman & Vott, 94] Ballman, K. and L. G. Votta (1994) Organizational congestion
in large-scale software development. Proceeding of the Third
International Conference on the Software Process Applying the
Software Process, IEEE CS Press: 123-34
[Votta, 93] Votta, L.G. (1993). Does every inspection need a meeting?
Proceeding of the First ACM
SIGSOFT Symposium on the Foundations of Software
Engineering: 107-114.
[Yourdon, 89]
Yourdon, E. (1989). Structured Walkthroughs. NY: Yourdon
Press.
86
6 APPENDICES
6.1 SAMPLE CHECKLIST FOR DESIGN INSPECTION
Design Review Checklist
Accuracy and Consistency
YES NO N/A
ITEM 1 Two objects or functions are referred to by one
unique name.
ITEM 3 Each requirement is uniquely identified..
ITEM 4 Low-level requirements are free of timing conflicts.
ITEM 5 Each requirement is specified only once.
ITEM 6 Low-level requirements are free of logical conflicts
ITEM 7 Data is specified only once.
ITEM 8 All acronyms and abbreviations are defined and used
consistently.
ITEM 9 Mathematical equations are defined consistently.
ITEM 10
The architecture and low-level requirements are
consistent and accurate with respect to control flow
and data flow.
ITEM 12 Algorithms are accurate.
ITEM 13 Low-level requirements are deterministic.
ITEM 14 All design elements conform to the design
constraints.
87
Verifiable (V&V)
YES NO N/A
ITEM 15 The Low-level requirements are specified
quantitatively, not qualitatively.
ITEM 16 All low-level requirements can be verified.
ITEM 17 The software architecture is specified in the design in
such a way that it can be verified.
Understandable (Developer)
YES NO N/A
ITEM 18 The requirements are stated clearly.
ITEM 19 There is only one interpretation of the requirement.
ITEM 20 The wording is understandable or defined.
General
YES NO N/A
ITEM 22 Design is traceable to the high level requirements
ITEM 23
The design implements all functional, interface,
performance, sizing, timing and adaptation
requirements.
ITEM 26
Design addresses issues with Control Flow, Run
Time Support, Real Time Predictability, and
Deactivated Code.
ITEM 27 Design avoids high complex class hierarchies.
ITEM 28 Public class data has been minimized and analyzed
for design errors.
88
6.2 SAMPLE CHECKLIST FOR CODE INSPECTION
Code Review Checklist
Requirements-Design
YES NO N/A
ITEM 1 Each procedure and function is modular and cohesive.
ITEM 2 The implemented code consistent with the architecture.
ITEM 3 The implemented code consistent with low level
requirements.
ITEM 4 The implemented code and comments consistent.
ITEM 5 Interrupts are handled correctly.
Variables
YES NO N/A
ITEM 6 All variables are initialized before usage.
ITEM 7 Variables are used for only one purpose.
ITEM 8 All used variables in the code are also defined.
ITEM 9 All defined variables in the code are also used.
ITEM 10 Data is retrieved and stored according to the specified
format
ITEM 11 Minimized usage of hard-coded constants is ensured.
ITEM 13 Variable ranges are defined in the range of data type
being used.
Computations
YES NO N/A
ITEM 14 Boundary conditions are handled correctly.
ITEM 15 Implemented code makes efficient use of computer
resources.
89
ITEM 16 Implemented equations are correct.
ITEM 17 Required accuracy is maintained during rounding.
ITEM 18 Types are consistent in expressions.
ITEM 19 Operators in equations are correct
ITEM 20 Sign conventions are correct and consistent.
Interfaces
YES NO N/A
ITEM 21 Interface variables are used for only one purpose.
ITEM 22 Return types are declared.
ITEM 23 All calling arguments are consistent by parameter and
type.
Logic
YES NO N/A
ITEM 24 There is only one entry and exit point.
ITEM 25 Loop conditions are correctly controlled.
ITEM 26 It is possible to execute all branch paths.
ITEM 26 There is code for every possible condition.
ITEM 28 Code does not depend on evaluation order.
ITEM 29 Expressions involving mixed case operators are
properly parenthesized.
ITEM 30 Parentheses are used correctly for determining
precedence.
ITEM 31 Array subscripts are properly used.
ITEM 32 Multiple inheritance is not used in the implemented
code
ITEM 33 Class hierarchies, that involves high complexity, have
been avoided
90
Error Checking
YES NO N/A
ITEM 34 Constraints are identified and handled correctly.
ITEM 35
Functions check for input parameter values in range,
or calling functions prevent calling with out of range
parameters.
ITEM 36 Divide by zero is prohibited, trapped, and handled.
General
YES NO N/A
ITEM 37 Implemented code is traceable to the design.
ITEM 38 Code does not generate non-justifiable warnings
when compiled.
ITEM 39 Control coupling is handled correctly.
ITEM 40 Data coupling is handled correctly.
91
6.3 SAMPLE CHECKLIST FOR REQUIREMENT BASED TESTING
Test Case Review Checklist
YES NO N/A
ITEM 1 The test environment (versions of software and tool)
is correct.
ITEM 2 Tests conditions are designed to reveal potential
errors.
ITEM 3 Implementation of the software requirements is
verified by the requirements-based low-level testing.
ITEM 4 Implementation of the software requirements is
verified by the requirements-based integration testing.
ITEM 5 Traceability between the requirement and test cases
are correct.
ITEM 6 Each requirement is traced to at least one test case.
ITEM 7 Expected results are defined clearly.
ITEM 8 Tested requirements are present on the test case.
ITEM 9 Test inputs are clearly defined.
ITEM 10 The test steps are clear and repeatable.
ITEM 11
Each test case traces to a requirement or clearly states
in the procedure/test case why it does not trace to a
requirement.
ITEM 12 Variables are tested using equivalence class
partitioning.
ITEM 13 Time related functions are tested.
92
Robustness Tests (V&V)
YES NO N/A
ITEM 14 Out of range conditions and computations are
exercised.
ITEM 15 Robustness Test Criteria have been met with regards
to time-related functions.
ITEM 16 Robustness Test Criteria have been met with regards
to invalid data inputs.
ITEM 17 General Robustness Test Criteria have been met with
regards to the creation of illegal states.
ITEM 18 Arithmetic overflow conditions are tested and trapped.
ITEM 19 Robustness Test Criteria have been met with regards
to System initialization under abnormal conditions.
ITEM 20
Robustness Test Criteria have been met with regards
to Input of corrupted and failure mode data from
external sources.