Software Quality AssuranceSoftware Quality AssuranceActivitiesActivities
Standard DefinitionStandard Definition
Software Quality - Conformance to: 1. explicitly stated functional and performance
requirements,
2. explicitly documented development standards, and
3. implicit characteristics that are expected of all professionally developed software.
The Big Question
How do we assure quality?How do we assure quality?
SQA Activities
1. Pre-Project Activities
2. Development and Maintenance Activities
3. Error Reduction Infrastructure
4. SQ Management Components
5. SQA System Assessment
6. Human Components
Software Quality Assurance by Galin
SQA Activities
1. Pre-Project Activities1. Contract Review
2. Development Plan
3. Quality Plan
2. Development and Maintenance Activities
3. Error Reduction Infrastructure
4. SQ Management Components
5. SQA System Assessment
6. Human Components
Contract Review ProcessContract Review Process
RFP
1st Draft
Revisions
Final ContractFinal Contract
SOW
Contract
Reality Check...
Q: Why should the software geeks worry about the contract?
A: Because the software team must do the work and assure the product's quality. loosely defined requirements unrealistic budgets unrealistic schedules
A: Contract review is required by ISO 9001
What to look for in 1What to look for in 1stst Draft Draft customer reqs clarified and documented?
alternative approaches examined?
risks identified?
costs and time estimates reasonable?
both customer and creator have capacity?
subcontractor participation clear?
proprietary rights?
relationship between customer and creator specified? Galin section 5.3
Development PlanDevelopment Planthat Includes Quality Assurance
RequirementsSpecification
Design
Coding
UnitTesting
Installation &Training
IntegrationTesting
Maintenance
Review the SRS
Design Reviews
CodingStandards
Validation
Co
nfig
uratio
n C
on
trol
Test Proceduresand Tolerances
Do
cum
entatio
n
Defect T
racking
Components of the Quality Plan Quality Goals
hopefully the SRS is some help
Review Activities schedule, type of reviews, scope, responsible persons
Software Test Plan type of unit tests and coverage, integration plan
Acceptance Tests for Sub-contract Software Configuration Tools and Procedures
Galin section 6.3
SQA Activities
1. Pre-Project Activities
2. Development and Maintenance Activities1. Formal Technical Reviews
2. Code Inspections
3. Unit Testing, Integration Test, etc. - earlier lecture
3. Error Reduction Infrastructure
4. SQ Management Components
5. SQA System Assessment
6. Human Components
Types of EvaluationsTypes of Evaluations Formal Reviews
conducted by senior personnel or outside experts uncover potential problems
Inspections and Walkthroughs done by peers detect problems, adherence to standards, etc.
Verification Unit Test Integration Test Usability Test
SQA Activities
1. Pre-Project Activities
2. Development and Maintenance Activities
3. Error Reduction Infrastructure1. work procedures - more details in a later lecture
2. templates and checklists - later lecture
3. staff training - at end of course
4. preventive actions - later lecture
5. configuration control - earlier lecture
6. document control
4. SQ Management Components
5. SQA System Assessment
6. Human Components
DefinitionsDefinitions
procedure - general methodology
work instructions - details of how to implement those procedures for this project
template - a standard Table of Contents
checklist - list of items that are part of an activity
Example Types of Work InstructionsExample Types of Work Instructions
audit process for subcontractors
design documentation templates
C++ programming instructions
coordination and cooperation with the customer
follow-up of beta site reporting
monthly progress reporting
etc...
Galin: page 316
SQA Activities
1. Pre-Project Activities
2. Development and Maintenance Activities
3. Error Reduction Infrastructure
4. QA Management Components Project Progress
schedules, budgets, risk analysis, … - earlier lectures Quality Metrics - earlier lecture Quality Costs
5. SQA System Assessment
6. Human Components
Statistical SQAStatistical SQA1. Info about defects are collected and categorized.
2. Defects are traced to their root cause.
3. Isolate the vital 20%
4. Move to correct problems that caused defects.
Pareto Principle: 80% of defects can be traced to 20% of causes.
SSQA is similar to"Six Sigma"
How much SQA is cost effective?
Software Quality
Costs
InitialCost of SQA
Cost of Failure
SQA+
Failure
Optimal Quality Level
EventualEventualCost of SQACost of SQA
Real NumbersCost of Software Quality for 15 Projects at Raytheon’s Equipment Division
0
10
20
30
40
50
60
70
Pe
rce
nta
ge
of to
tal p
roje
ct co
st
Year
CMM level 3Start of intiative CMM level 1
TCoSQ
PreventionRework
Appraisal
Cost ofConformance
Rework
87 88 89 90 91 92 93 94 95 96
htt
p:/
/ww
w2
.um
ass
d.e
du
/sw
pi/c
ost
mo
de
ling
/pa
pe
rs/s
coq
pa
p1
.do
c
What are the SQA Costs Contract Reviews SRS Reviews Design Reviews Code Walkthrough Checklists
creating the checklists training people to use the checklists filling out the checklists reviewing the checklists data auditing the checklist process
even more SQA Costs Tools o' Plenty
progress tracking tool estimation and scheduling tools testing tools
Metrics time spent creating forms and gathering the data time spent analyzing performance data
Yadda yadda yadda etc etc etc
but don't forget to ask
What are the costs of not conducting SQA? inaccurate proposal => inaccurate time
estimate => unhappy customer poor SRS => expensive and time
consuming changes to design etc etc etc
SQA Activities
1. Pre-Project Activities
2. Development and Maintenance Activities
3. Error Reduction Infrastructure
4. SQ Management Components
5. SQA System Assessment Quality Management Standards
SEI CMM ISO 9001
6. Human Components
Details o'plenty at end of semester
SQA Activities
1. Pre-Project Activities
2. Development and Maintenance Activities
3. Error Reduction Infrastructure
4. SQ Management Components
5. SQA System Assessment
6. Human Components Management SQA Unit
The SQA GroupThe SQA Group Participates in the development of the project's software
process description. Reviews software engineering activities to verify compliance
with the defined process. Audits designated software work products to verify compliance
with those defined as part of the software process. Ensures that deviations in software work and work products are
documented and handled according to a documented procedure.
Records any noncompliance and reports to senior management.
Coordinates and controls change. Helps collect and analyze metrics.
SummarySummary
"Quality" includes explicit and implied properties as well as the development process.
Reviews are a key part of SQA.
An SQA Group is highly recommended.
QA must be part of the plan and the schedule.
SSQA is a commonly used formal approach to process improvement.