SOFTWARE QUALITY ASSURANCEREVIEWS & CHECKLISTS

Seminar: Oana FEIDIQuality Manager – Continental Automotive

INTRO

Software Quality Assurance: a systematic and planned pattern of actions that are demanded to guarantee software quality

Activities to check software quality: Software reviews

objective: finding analysis and design defects in SW artifacts produced in the initial phases of SW development

Testing Patterns and formal procedures Change control Software metrics Registering and record keeping

PREREQUISITES

Source: http://cronos.cos.ufrj.br/publicacoes/reltec/es55601.pdf

REVIEW A software review is

"A process or meeting during which a software product is [examined by] project personnel, managers, users, customers, user representatives, or other interested parties for comment or approval” [IEEE 1028:1997 – “IEEE Standard for software reviews”]

German company: a defect caught by testing costs 14.5 time as

much to correct compare to one discovered by formal inspection

a defect discovered by a customer costs 68 times as much to fix

REVIEWS: BENEFITS Decrease the costs of fixing a defect Shorten the delivery time Reduce the rework, thus increasing

productivity

Group reviews provide an opportunity to see how other team members are working (Knowledge is automatically exchanged:

programming language features coding and commenting style program architecture design notations ways to document requirements)

provide an effective “forum” for less experienced team members

REVIEW: COMMON BUGS

Error Type Review Testing

Module interface errors X

Excessive code complexity X

Un-required functionality present X

Usability problems X

Performance problems X X

Badly structured code X

Failure to meet requirements X X

Boundary value errors X X

Source: http://www.processimpact.com/articles/inspects.html

REVIEW TYPE: (FAGAN) INSPECTION

REVIEW TYPE: WALKTHROUGH “ a designer or programmer leads members of the

development team and other interested parties through a software product, and the participants ask questions and make comments about possible errors, violation of development standards, and other problems" [IEEE 1028:1997 – “IEEE Standard for software reviews”]

Objectives to gain feedback about the technical quality or content of the

document to familiarize the audience with the content

IEEE 1028 recommends three specialist roles in a walkthrough:

author, who presents the software product in step-by-step manner at the walk-through meeting, and is probably responsible for completing most action items;

walkthrough leader, who conducts the walkthrough, handles administrative tasks, and ensures orderly conduct (and who is often the Author); and

The recorder, who notes all potential defects, decisions, and action items identified during the walkthrough meeting.

SUCCESSFUL REVIEW: TIPS & TRICKS Check your egos at the door

“we prefer to have a peer find a defect, rather than the customer”

Critique the products, not the producers Find problems during the review; don’t try to fix them Limit inspection meetings to a max 2 hours

Optimum balance: 150-200 lines of code/hour Optimum balance: 4-6 pages of design or text/hour

Avoid style issues unless they impact the performance or understandability Use coding standards or code “beautifiers” Focus on uncovering errors in logic, function or

implementation Inspect early and often, formally & informally

FAILED REVIEW: AVOIDING TIPS & TRICKS Developers fear that the review result will be

used against them during evaluation Authors are unable to separate their egos from

their work products; claim they don’t need reviews; become defensive during the review

Participants don’t prepare adequately prior to the review meeting

Documents under review are not self-explanatory

Author does not take the result of the review seriously

Review meetings loose their focus → trap to fall into technical discussions rather than covering the document under review

Project schedule doesn’t leave room for reviews

REVIEW: DOCUMENTATION Documentation of review results is the major

distinction between informal and informal review Each error found is described in enough detail (line

of code reference, function name, (sub)chapters) Each company customizes its own classification for

defects/issues (errors/remarks); severity; development phase in which the error was introduced

Documentation of review means: Record defects during review meeting Collect data from multiple inspections Analyze defect trend

Assess review effectiveness Identify ways to improve the software development process

to prevent common types of defects

Example review template

CHECKLIST “hunt for anticipating types of errors” Sample for code inspections (

http://www.processimpact.com/articles/inspects.html)

"Walking on water and developing software from a specification are easy if both are

frozen.” Edward V Berard

SPECIFICATION REVIEW

What to look for: InconsistenciesMissing “else” branch of a specificationMisinterpretation – unclear requirementsRequirements that are excluding one

another

Exercise (working groups): document the review of the specification

received (including checklist)Requirements

review checklistSpecification

review template

THANK YOU!