software quality engineering for managers training · pdf fileon process organization,...

SEMATECHTechnology Transfer 92111389B-TRG

Software Quality Engineering forManagers—Training Materials

© 1995 SEMATECH, Inc.

SEMATECH and the SEMATECH logo are registered service marks of SEMATECH, Inc.

Semiconductor Equipment Technology Center (SETEC), the SETEC logo, Sandia National Laboratories, andthe Sandia National Laboratories logo are trademarks of Sandia National Laboratories.

Software Quality Engineering for Managers—Training MaterialsTechnology Transfer # 92111389B-TRG

SEMATECHMay 31, 1995

Abstract: This document contains presentation and instructional materials for a four-hour seminar on theessential elements for developing high-quality, reusable software. The seminar includes sessionson process organization, managing software development, the software requirements specification(SRS) document, software reviews and inspections, software configuration management (SCM),and testing. The target audience for this course includes senior and middle managers, and projectand quality managers with little or no software quality engineering background. These trainingmaterials were developed at the Semiconductor Equipment Technology Center (SETEC) at SandiaNational Laboratories.

Keywords: Software Process Improvement, Software Development, Software Reliability, Training

Approvals: Harvey Wohlwend, Project ManagerDan McGowan, Technical Information Transfer Team Leader

iii

Technology Transfer # 92111389B-TRG SEMATECH

Table of Contents

1 EXECUTIVE SUMMARY........................................................................................................11.1 Introduction .......................................................................................................................11.2 Process Organization .........................................................................................................11.3 Project/Risk Management..................................................................................................11.4 Requirements .....................................................................................................................11.5 Reviews and Inspections (Formal In-Process Reviews)....................................................11.6 Configuration Management ...............................................................................................11.7 Testing ...............................................................................................................................21.8 Support ..............................................................................................................................2

2 COURSE SYLLABUS ..............................................................................................................2Course Schedule: .......................................................................................................................2Course Instructors: .....................................................................................................................2Target Audience:........................................................................................................................2Objectives: .................................................................................................................................2Course Material:.........................................................................................................................2

3 SCHEDULE...............................................................................................................................2

4 OUTLINE ..................................................................................................................................3Introduction................................................................................................................................3Process Organization .................................................................................................................3Project/Risk Management ..........................................................................................................3Requirements .............................................................................................................................4Reviews and Inspections (Formal In-Process Reviews) ............................................................4Configuration Management .......................................................................................................4Testing........................................................................................................................................4Support.......................................................................................................................................4

5 INTRODUCTION......................................................................................................................5

6 SOME WORKING DEFINITIONS.........................................................................................17

7 PROCESS ORGANIZATION.................................................................................................18

8 PROJECT / RISK MANAGEMENT.......................................................................................30

9 REQUIREMENTS...................................................................................................................40

10 REVIEWS AND INSPECTIONS............................................................................................47

11 CONFIGURATION MANAGEMENT ...................................................................................65

12 FORMAL SOFTWARE CONFIGURATION MANAGEMENT AND TIME-PHASEDVERSION (BASELINE) RELEASE .......................................................................................72Overview..................................................................................................................................72Structure...................................................................................................................................72Change Request/Problem Reporting........................................................................................74

iv

SEMATECH Technology Transfer # 92111389B-TRG

Change Control/Configuration Management...........................................................................75Software Engineering Activities ..............................................................................................77Summary ..................................................................................................................................77

13 GLOSSARY OF SOFTWARE CONFIGURATION MANAGEMENT TERMINOLOGY ..78

14 TESTING.................................................................................................................................81

15 SUPPORT................................................................................................................................90

16 QUESTIONS TO ASK ............................................................................................................98

17 ACTION PLAN FOR SOFTWARE QUALITY .....................................................................99

18 SEI MATURITY MODEL: LEVEL 2 – REPEATABLE PROCESS ...................................101

19 FOLLOW-UP QUESTIONS .................................................................................................104

20 SOME FURTHER READING ..............................................................................................105Articles Specific to Sessions..................................................................................................105On Software Project Management .........................................................................................105On Software Metrics ..............................................................................................................105

v


List of Figures

Figure 1 SCM Context ..................................................................................................................73

Figure 2 Normal Release Cycling .................................................................................................74

List of Tables

Table 1 Change Profiles................................................................................................................75

Table 2 Contents Of Change Request Analysis ............................................................................76

Table 3 Contents of Change Implementation Package .................................................................76

1


1 EXECUTIVE SUMMARY

This document contains presentation and instructional materials for a seminar on the essentialelements necessary to develop high-quality, reliable software. Included are several key questionsthat managers and executives can (and should) ask about a software development project. Asmall case study is used to illustrate important elements in the software development process andas a focus for discussion. The seminar is structured as follows:

1.1 Introduction

The seminar begins with an introduction and a query of participants’ expectations. Althoughcourse goals are presented, emphasis is placed on areas of common interest to the class. Softwareis defined and two views of software quality are offered. The following sessions all relate toactivities that denote a level of software engineering maturity that is well defined. Targetaudience includes project and quality managers with little or no background in software qualityengineering.

1.2 Process Organization

This session focuses on the importance of viewing software development and softwareengineering from a process perspective. Software life cycle activities and their interrelationshipsare discussed, with emphasis on devoting sufficient time to pre-implementation activities. Alsodiscussed are opportunities for quality improvement and use of some core metrics to trackcurrent processes and measure improvement.

1.3 Project/Risk Management

This session addresses a reasonable approach to the management aspects of softwaredevelopment. It teaches that each participant in a software development project must be aware ofthese activities, accompanying management planning, and what management planning does forthe project.

1.4 Requirements

This session describes and discusses a software requirements specification (SRS) as the mostimportant product definition document in software development. The reasons for an SRS and thecontent of an SRS also are presented.

1.5 Reviews and Inspections (Formal In-Process Reviews)

This segment covers the nature and purpose of software reviews, which satisfy both managementand technical aspects of software development. Levels and types of reviews are discussed, withappropriate life cycle review points highlighted.

1.6 Configuration Management

Software configuration management (SCM) is an essential aspect of the software developmentprocess. This session covers the elements and benefits of SCM, and answers common questionsabout SCM.

2


1.7 Testing

Regardless of the care taken in developing software, the need remains to test software at alllevels to validate it to requirements and ensure that it is as defect-free as possible before deliveryto the customer. This session stresses the concept of testing to detect defects. Test approachesand activities for testing are presented and discussed.

1.8 Support

This concluding session recaps the elements of software engineering that promote high qualityand high reliability. The session and the course is concluded by reviewing the softwareengineering Capability Maturity Model (CMM).

2 COURSE SYLLABUS

Course Schedule:

8:30 a.m. to 12:30 p.m.

Course Instructors:

Eric Tomlin, Sandia National Laboratories, Primary

Dave Peercy, Sandia National Laboratories, Alternate

Target Audience:

Senior and middle managers. Project and quality managers with little or no software qualityengineering background.

Objectives:

• Develop a framework for software engineering (and the software life cycle).

• Recognize the benefits of integrating software engineering techniques into a designfor quality and reliability.

• Outline an action plan for the manager to apply toward increasing the quality ofsoftware produced.

Course Material:

1. Course Notebook

2. Case Study Material - (Handouts at Seminar)

3 SCHEDULE

Seminar Length: 4 hours

8:30 AM

Introduction

3


Process Organization

Break

Project/Risk Management

Requirements

Review and Inspections (Formal In-Process Reviews)

Break

Configuration Management

Testing

Support

12:30 PM

Lunch

4 OUTLINE

Introduction

The course in introduced along with a query of the class participant's expectations. Althoughthere are course goals presented, emphasis will be placed on areas of common interest to theclass. A working definition of software is presented as well as two views of software quality. Thesoftware engineering CMM from the Software Engineering Institute (SEI) is introduced to causethe participants to consider where their organizations might be in their ability to consistentlyproduce high quality software. The following sessions all relate to activities that denote a level ofsoftware engineering maturity that is well defined.


This session focuses on the importance of viewing software development and softwareengineering from the perspective of a process. Software life cycle activities and theirinterrelationships are discussed with an emphasis on sufficient time spent on activities prior toimplementation. Opportunities for quality improvement are discussed along with some coremetrics that can be used to track current processes and measure improvement. This session startsa series of questions that managers could (and should) ask about software projects under theirdirection. An outline to action plan items also begins with this session.


This session addresses a reasonable approach to the management aspects of softwaredevelopment. Each participant in a software development project must be aware of theseactivities, the accompanying management planning, and what management planning does for theproject.

4


Requirements

An SRS is the most important product definition document in software development. The SRS isdescribed and discussed in this session. The reasons for an SRS and the content of an SRS arealso presented.

Reviews and Inspections (Formal In-Process Reviews)

Reviews satisfy both management and technical aspects of software development. This sessioncovers the nature and purpose of software reviews. Levels of reviews and types of reviews arediscussed with appropriate life cycle review points highlighted.

This session focuses on one particularly important type of review, the software inspection. Itcovers the background of software inspections, discusses pros and cons of inspections, andpresents reported information of its return on investment. This session also describes theinspection process and roles and responsibilities of participants.

Configuration Management

SCM is an essential aspect of the software development process. This session covers theelements of SCM, benefits of SCM, and answers some common questions about SCM.

Testing

Regardless of the care taken in developing software, there is still a need to test software at alllevels to validate the software to requirements and to ensure that the software is as defect-free asit can be before delivery to the customer. This session stresses the concept of testing to detectdefects. Test approaches and activities for testing are presented and discussed.

Support

This concluding session recaps the elements of software engineering that promote high quality,high reliability software. The session and the course is concluded by revisiting the softwareengineering Capability Maturity Model.

5


5 INTRODUCTION

Sof

twar

e Q

ualit

y E

ngin

eeri

ngfo

rM

an

ag

ers

Spo

nsor

ed B

y

Dev

elop

ed B

yS

emic

ondu

ctor

Equ

ipm

ent

Tec

hnol

ogy

Cen

ter

(SE

TE

C)

San

dia

Nat

iona

l Lab

orat

orie

sC

opyr

ight

© 1

994

SE

MA

TE

CH

, In

c

Rev

isio

n D

ecem

ber

1994

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56In

trod

uctio

n -

1 (

12/9

4)

6


Who

Are

We?

W

hat

do W

e D

o?

•S

andi

a N

atio

nal L

abor

ator

ies

−O

pera

ted

by M

artin

Mar

iett

a fo

r th

e D

epar

tmen

t of

Ene

rgy

- ".

. .

in t

hena

tiona

l int

eres

t."

−P

art

of t

he N

ucle

ar W

eapo

ns C

ompl

ex

Hig

h R

elia

bilit

y -

Hig

h C

onfid

ence

Pro

duct

s

Ext

ensi

ve u

se o

f em

bedd

ed s

oftw

are

−R

esea

rch

exte

nds

to r

obot

ics,

mat

eria

ls s

cien

ce,

ener

gy s

ourc

es,

com

puta

tiona

l mec

hani

cs,

cryp

togr

aphy

and

man

y ot

her

area

s

•S

emic

ondu

ctor

Equ

ipm

ent

Tec

hnol

ogy

Cen

ter

(SE

TE

C)

−Id

entif

icat

ion

of c

ross

-sec

tion

of L

abor

ator

y as

sets

to

prov

ide

assi

stan

ce t

o S

EM

AT

EC

H a

nd t

he S

EM

I/S

EM

AT

EC

H m

embe

rco

mpa

nies

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56In

trod

uctio

n -

2 (

12/9

4)

7


Wha

t D

o M

anag

ers

Do?

"Man

agem

ent's

job

is t

o cr

eate

an

envi

ronm

ent"

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56In

trod

uctio

n -

3 (

12/9

4)

8


Why

are

YO

U h

ere?

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56In

trod

uctio

n -

4 (

12/9

4)

9


Why

is T

here

a

Pro

blem

?

Sof

twar

e as

% o

f

Dev

elo

pmen

t W

ork

05

1015202530354045

Pro

blem

Cre

ated

Rqm

ts

Des

ign

Cod

e

Tes

t

Sup

port

05

10152025

Clo

sure

Rqm

ts

Des

ign

Cod

e

S/W

Int

Sys

Int

Bet

a T

est

Sup

port

Ada

pted

fro

m S

oftw

are

Logi

stic

s N

atio

nal W

orks

hop,

Aug

ust

15-1

6, 1

989

0

255075

100

19

6019

7019

8019

90

Sm

all A

mou

ntof

Em

bedd

ed S

oftw

are

Ext

ensi

ve U

seof

Em

bedd

ed S

oftw

are

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56In

trod

uctio

n -

5 (

12/9

4)

Def

ects

: W

hen

Cre

ated

- W

hen

Fou

nd

10


Sem

inar

Obj

ectiv

es

•D

evel

op a

fra

mew

ork

for

soft

war

e en

gine

erin

g

−B

asis

for

: T

rack

ing

cost

& s

ched

ule,

(S

oftw

are)

Sys

tem

Dev

elop

men

tP

lan

•R

ecog

nize

the

ben

efits

of

inte

grat

ing

soft

war

een

gine

erin

g co

ncep

ts,

tool

s an

d te

chni

ques

into

ade

sign

for

qua

lity

and

relia

bilit

y

−R

educ

es r

isk,

impr

oves

vis

ibili

ty in

to t

he s

ched

ule,

and

impr

oves

the

qual

ity o

f de

liver

ed p

rodu

cts

•O

utlin

e a

n ac

tion

plan

for

YO

U t

o ap

ply

to in

crea

se t

hepr

obab

ility

of

prod

ucin

g hi

gh q

ualit

y so

ftw

are

−R

esul

ts c

an b

e pa

rt o

f a

Sof

twar

e Q

ualit

y P

lan

and

Sof

twar

eD

evel

opm

ent

Pla

n

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56In

trod

uctio

n -

6 (

12/9

4)

11


Sem

inar

For

mat

•P

robl

em/D

ilem

ma

Dia

san

Mar

ine

Har

dwar

e C

ompa

ny

•D

iscu

ssio

n

•P

rese

ntat

ion

on P

repa

red

Mat

eria

l

•Q

uest

ions

Enc

oura

ged

!

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56In

trod

uctio

n -

7 (

12/9

4)

12


Wh

at

is s

oft

wa

re?

• D

oc

um

en

tati

on

(P

rod

uc

t D

efi

nit

ion

)

--

So

urc

e,

Ob

ject

s, E

xecu

tab

les

--

Co

mp

reh

en

siv

e w

ritt

en

de

scri

pti

on

- D

efi

ne

s re

qu

ire

me

nts

co

nte

nt,

co

mp

osi

tio

n,

de

sig

n,

pe

rfo

rma

nce

, te

stin

g,

use

an

d s

up

po

rt.

Incl

ud

es

pla

ns

as

ap

pro

pri

ate

(e

.g.,

de

velo

pm

en

t, t

est

, co

nfi

gu

rati

on

ma

na

ge

me

nt)

.

• C

od

e

Pro

gra

m

Lin

ke

rs

Ed

ito

rs

Ma

nu

als

Te

st

Pla

ns

De

ve

lop

me

nt/

Su

pp

ort

To

ols

Pro

ce

du

res

Op

era

tin

g S

ys

temD

iag

no

sti

c S

oft

wa

re

Inte

rfa

ce

Re

qu

ire

me

nts

So

ftw

are

De

sig

n D

es

cri

pti

on

s

Sp

ec

ific

ati

on

s

Co

mp

ile

rsS

oft

wa

re R

eq

uir

em

en

ts

"Ah

, it

's o

nly

co

de

!"

13


Sof

twar

e Q

ualit

y

Dep

ends

on

One

's P

ersp

ectiv

e!

•If

I h

ave

to u

se it

. .

.

•If

I'm

the

one

who

has

to

fix it

. .

.

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56In

trod

uctio

n -

9 (

12/9

4)

14


Sof

twar

e Q

ualit

yU

ser

Per

spec

tive

Usa

bilit

yT

he e

ffor

t re

quir

ed t

o le

arn,

ope

rate

, pr

epar

e in

put

and

inte

rpre

t ou

tput

of

a pr

ogra

m

S

afet

yT

he s

oftw

are

desi

gn c

onta

ins

posi

tive

mea

sure

s to

prev

ent

or r

educ

e to

an

acce

ptab

le le

vel o

f ri

sk,

syst

em

failu

res

that

cou

ld c

ause

inju

ry o

r de

ath,

or

seri

ous

dam

age

to c

ritic

al e

quip

men

t

Eff

icie

ncy

The

ext

ent

to w

hich

sof

twar

e pe

rfor

ms

its in

tend

ed

func

tions

with

a m

inim

um o

f co

mpu

ting

reso

urce

s (e

.g.,

Res

pons

e T

ime)

Rel

iabi

lity

The

pro

babi

lity

that

sof

twar

e w

ill n

ot c

ause

the

fai

lure

of

a sy

stem

for

a s

peci

fied

time

unde

r sp

ecifi

ed c

ondi

tions

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56In

trod

uctio

n -

10

(12/

94)

15


Sof

twar

e Q

ualit

yM

aint

aine

r's P

ersp

ectiv

e

Mai

ntai

nabi

lity

The

eas

e w

ith w

hich

sof

twar

e ca

n be

cha

nged

- "S

uppo

rtab

ility

"

P

orta

bilit

yT

he e

ase

with

whi

ch s

oftw

are

can

be

tran

sfer

red

from

one

com

pute

r sy

stem

or

envi

ronm

ent

to a

noth

er

Impl

ies

Reu

sabi

lity

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56In

trod

uctio

n -

11

(12/

94)

16


Whe

re is

YO

UR

Org

aniz

atio

n?(I

n its

Goa

l to

Pro

duce

Hig

h Q

ualit

y S

oftw

are)

Sof

twar

e E

ngin

eeri

ng C

apab

ility

Sof

twar

e E

ngin

eeri

ng I

nstit

ute

(S

EI)

, C

arne

gie

Mel

lon

Uni

vers

ity

RE

SU

LT

SP

RO

BLE

M A

RE

AS

AD

DR

ES

SE

D

5 -

OP

TIM

IZIN

G[F

eedb

ack]

4 -

MA

NA

GE

D[Q

uant

itativ

e]

3 -

DE

FIN

ED

[Qua

litat

ive]

2 -

RE

PE

AT

AB

LE[In

tuiti

ve]

1 -

INIT

IAL

[Ad

Hoc

]

Acq

uire

d C

apab

ilitie

s -

Ada

ptab

le t

o C

hang

e -

Aut

omat

ion

Pro

cess

Con

trol

- S

tatis

tical

Pro

cess

Con

trol

- C

hang

ing

Tec

hnol

ogy

- P

robl

em P

reve

ntio

n

Pro

cess

Mea

sure

men

t -

Fau

lt P

roje

ctio

n -

Tes

t &

Rev

iew

Cov

erag

e -

Pro

cess

Met

rics

Dat

abas

e

Pro

cess

Def

initi

on -

Qua

lity

Poi

nt R

evie

ws

- S

oftw

are

Eng

inee

ring

Tra

inin

g

Bas

ic M

anag

emen

t C

ontr

ol -

Pro

ject

Pla

nnin

g (C

ost,

Sch

edul

e) -

Con

figur

atio

n M

anag

emen

t -

Reg

ress

ion

Tes

ting*

- S

oftw

are

Qua

lity

Ass

uran

ce

Pro

duct

ivity

&Q

ualit

y

Ris

k

MA

TU

RIT

Y

LEV

EL

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56In

trod

uctio

n -

12

(12/

94)

* R

egre

ssio

n T

estin

g:

Sel

ectiv

e r

etes

ting

of a

sys

tem

or

com

pone

nt t

o ve

rify

tha

t

mod

ifica

tions

hav

e no

t ca

used

uni

nten

ded

effe

cts

and

that

the

sys

tem

or

com

pone

nt s

till c

ompl

ies

with

its

spec

ified

req

uire

men

ts

17


6 SOME WORKING DEFINITIONS

Baseline A baseline is the documented identification of a software productitscode and all its related documentation at some specific point in time. It isthe basis for all Software Configuration Management activities.

Error A human action that results in one or more faults being inserted into awork product.

Failure A departure of a computer program's operation from the user'srequirements.

Failure Intensity The number of failures occurring in a given time period.

Fault Defect in the software that can cause a failure. Defect and fault are termsthat are often interchanged.

Inspection A formal evaluation of work products or software elements (requirements,design, code, etc.) to detect faults, violations of development standards andother problems Participants have defined roles, minutes recorded andretained, results reported.

Review Any number of techniques where a work product, or a set of workproducts, are presented for comment or approval. Reviews includemanagement reviews.

Regression Testing Selective retesting of a system or component to verify that modificationshave not caused unintended effects and the system of component stillcomplies with its specified requirements.

Risk The probability (chance) of an unwanted event or outcome.

Version A version is a software product with a defined set of capabilities. A newversion is a variation of the previous version, in that it has a change in itsfunctionality or performance characteristics. An example of this is if thesystem is changed to include an additional process or new feature.

18


7 PROCESS ORGANIZATION

Pro

cess

Org

aniz

atio

n

Pro

cess

Org

aniz

atio

n -

1 (1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

19


Ses

sion

Goa

ls

•T

o es

tabl

ish

a fr

amew

ork

for

the

rem

aini

ng s

essi

ons

•T

o di

scus

s th

e im

port

ance

of

view

ing

soft

war

een

gine

erin

g fr

om t

he p

ersp

ectiv

e of

a p

roce

ss

•T

o di

scus

s qu

ality

impr

ovem

ent

in li

ght

of t

he s

oftw

are

engi

neer

ing

proc

ess

Pro

cess

Org

aniz

atio

n -

2 (1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

20


Rqm

ts.

Des

ign

Cod

e

Tes

t

Sup

port

6%5%

7%

15%

67%

Typ

ical

Allo

tmen

tfo

r Li

fe C

ycle

More

Id

eal A

llotm

en

tin

Deve

lop

me

nt

50%

10%

15%

25%

Req

mts

Des

ign

Cod

eT

est

Typ

ical

Allo

tmen

tin

Dev

elop

men

t

15%

35%

20%

30%

Req

uire

men

ts Des

ign

Impl

emen

tatio

n

Tes

t Pro

duct

ion Sup

port

Ret

irem

ent

Con

cept

Exp

lora

tion

21


Rel

atio

nshi

p of

Pro

cess

Act

iviti

es

Use

rs /

Cus

tom

ers

Pro

duct

ion

/ S

uppo

rt

(Use

rs /

Cus

tom

ers)

Req

uire

men

ts

Pre

limin

ary

Des

ign

Cod

ing

Det

aile

dD

esig

n

Sys

tem

Tes

tP

lann

ing

Inte

grat

ion

Tes

tP

lann

ing

Uni

tT

est

Pla

nnin

g

Sys

tem

Tes

ting

Inte

grat

ion

Tes

ting

Uni

tT

estin

g

Inte

grat

ed S

oftw

are

Man

agem

ent

Str

uctu

re

Pro

cess

Org

aniz

atio

n -

4 (1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

22


Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56

Em

bedd

ed S

oftw

are

Life

Cyc

le M

odel

Ada

pted

from

"A

Spi

ral M

odel

of S

oftw

are

Dev

elop

men

t and

Enh

ance

men

t", B

. W. B

oehm

, IE

EE

Com

pute

r, M

ay 1

988.

Eva

luat

e A

ltern

ativ

es,

Iden

tify

& R

esol

ve R

isks

Dev

elop

, Ver

ifyN

ext-

Leve

l Pro

duct

s

Det

erm

ine

Obj

ectiv

es,

Alte

rnat

ives

, Con

stra

ints

Pla

n N

ext P

hase

s

Exp

lore

Con

cept

/C

usto

mer

Inp

utR

qmts

Ana

lysi

s, H

W-

SW

Tra

de

off

sP

roto

type

Bet

aS

yste

m

Pro

duct

ion

Sys

tem

Pol

icy/

Pro

c.fo

r S

W Q

ualit

y

Ge

n.

SW

D

ev.

Ap

pro

ach

Ris

k A

naly

sis

-C

ost,

Sch

edul

e, T

echn

ical

Ris

k A

naly

sis

-C

ost,

Sch

edul

e, T

echn

ical

Des

ign

Tra

deof

fs

Ref

ine

Sof

twar

eD

eve

lop

me

nt

Pla

n

Sof

twar

e S

yste

mT

est

Pla

nnin

g

Ref

ine

Sof

twar

eR

equi

rem

ents

Req

uire

men

tsIn

spec

tion

For

mal

ize

Sof

twar

eP

rodu

ctD

esig

n

Des

ign

Rev

iew

/In

spec

tion

Cod

e In

spec

tion

Sof

twar

e In

tegr

atio

nT

est

Pla

nnin

g

Har

dwar

e/

Sof

twar

e

Inte

grat

ion

T

est

Pla

nnin

g

Tes

t P

lan

Insp

ectio

n/R

evi

ew

Inte

grat

ion

and

Tes

t

Sys

tem

Tes

t

Acc

epta

nce

Tes

tN

ew F

eatu

res

:F

old

into

Pro

toty

pe,

Bet

a.or

Pro

duct

ion

Sys

tem

dep

endi

ngon

siz

e an

d co

mpl

exity

Pro

cess

Org

aniz

atio

n -

5 (1

2/94

)

23


Pro

cess

(M

odel

) D

efin

ition

- W

hy B

othe

r?

•A

ll w

ork

is a

pro

cess

−H

as in

put(

s) &

out

put(

s)

•D

efin

ition

is n

eede

d to

bas

elin

e pr

oces

s

−P

rovi

des

fram

ewor

k fo

r de

finin

g de

velo

pmen

t ac

tiviti

es

−P

rovi

des

the

foun

datio

n fo

r m

easu

ring

the

pro

cess

•D

efin

ition

req

uire

d fo

r re

peat

abili

ty

−P

oint

s to

are

as o

f pr

oces

s fo

r im

prov

emen

t

Pro

cess

Org

aniz

atio

n -

6 (1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

24


Sof

twar

e Q

ualit

y M

etri

csM

easu

ring

the

Pro

cess

and

Its

Pro

duct

s

Pro

cess

% o

f P

rodu

ct-D

efin

ition

Doc

umen

tatio

n In

spec

ted

% o

f P

rodu

ct-D

efin

ition

Doc

umen

tatio

n U

nder

Con

figur

atio

n M

anag

emen

t

% o

f R

equi

rem

ents

Im

plem

ente

d à

Tes

ted

(Tes

t C

over

age)

Def

ect

Det

ectio

n/C

orre

ctio

n R

ates

Pro

duct

•

Cor

rect

ness

Fau

lts p

er t

hous

and

lines

of

code

- F

ault

(Def

ect)

Den

sity

•R

elia

bilit

yF

ailu

re I

nten

sity

(F

ailu

res/

Hr

of O

pera

tion)

+

MT

BF

**

•A

vaila

bilit

yO

pera

tiona

l Upt

ime

•S

uppo

rtab

ility

$$$;

Per

cent

age

of O

rigi

nal C

ost

to P

rodu

ce

** M

TB

F -

Mea

n T

ime

Bet

wee

n F

ailu

res

Pro

ject

vs

His

tory

Pla

nned

vs

Act

ual (

Cos

t—C

ON

C*,

S

ched

ule

Mile

ston

es,

Pro

gram

Siz

e)

Man

agem

ent

(Res

ourc

es)

* C

ON

C

- C

ost o

f Non

Con

form

ance

Pro

cess

Org

aniz

atio

n -

7 (1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

25


Sof

twar

e Q

ualit

y M

etri

cs

•T

hree

vie

ws—

Col

lect

ing

the

righ

t m

easu

re a

t th

e ri

ght

leve

l . .

. a

nd f

or t

he r

ight

pur

pose S

trat

egic

â

Cus

tom

er B

ased

Tac

tical

â

Pro

ject

Man

agem

ent

Pro

cess

& P

rodu

ct

â E

ngin

eeri

ng

•In

tegr

atin

g lo

wer

leve

l mea

sure

s in

to t

he a

ppro

pria

tevi

ew

Pro

cess

Org

aniz

atio

n -

8 (1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

26


Pro

cess

Def

initi

onF

ound

atio

n fo

r Q

ualit

y Im

prov

emen

t

Rel

ies

On:

•F

ocus

on

proc

ess,

rat

her

than

the

pro

duct

•R

epea

tabi

lity

of p

roce

ss

•M

easu

rem

ent

Mea

suri

ng d

egre

e of

impr

ovem

ent

agai

nst

som

e st

anda

rd

•E

mph

asis

on

cont

inuo

us im

prov

emen

t

Pro

cess

Org

aniz

atio

n -

9 (1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

27


Pro

cess

Org

aniz

atio

n S

umm

ary

•S

oftw

are

deve

lopm

ent

can

(and

mus

t) b

e ap

proa

ched

in a

dis

cipl

ined

, en

gine

erin

g m

anne

r

•K

ey t

o im

prov

ing

soft

war

e qu

ality

is t

o im

prov

e th

epr

oces

s us

ed t

o de

velo

p so

ftw

are

•A

str

uctu

red

appr

oach

alo

ne d

oesn

't gu

aran

tee

qual

ityim

prov

emen

t

−W

e ne

ed a

litt

le m

ore

than

just

the

fra

mew

ork!

W

e ne

ed t

o fil

l in

the

activ

ities

•Q

uest

ion

you

shou

ld a

sk

−H

ave

we

real

ly d

efin

ed O

UR

pro

cess

?

Pro

cess

Org

aniz

atio

n -

10 (

12/9

4)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

28


Pro

cess

Org

aniz

atio

n S

umm

ary

Act

ion

Pla

n

Mak

e an

app

oint

men

t w

ith y

our

soft

war

e m

anag

er t

osk

etch

out

the

str

uctu

re o

f Y

OU

R s

oftw

are

deve

lopm

ent

proc

ess

−U

se b

lank

Em

bedd

ed L

ife C

ycle

Mod

el a

s ba

sis

or d

evel

op s

impl

er m

odel

−C

ompa

re t

o m

odel

in t

his

cour

se

−A

sk:

Is

ther

e el

emen

ts o

f th

e co

urse

mod

el t

hat

we

wan

t to

impl

emen

t?If

so:

W

hen

can

we

begi

n to

impl

emen

t th

em?

Wha

t ca

n w

e do

in t

hene

xt 6

mon

ths?

In

the

nex

t ye

ar?

Pro

cess

Org

aniz

atio

n -

11 (

12/9

4)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

29


Em

bedd

ed S

oftw

are

Life

Cyc

le M

odel

Ada

pted

from

"A

Spi

ral M

odel

of S

oftw

are

Dev

elop

men

t and

Enh

ance

men

t", B

. W. B

oehm

, IE

EE

Com

pute

r, M

ay 1

988.

Eva

luat

e A

ltern

ativ

es,

Iden

tify

& R

esol

ve R

isks

Dev

elop

, Ver

ifyN

ext-

Leve

l Pro

duct

s

Det

erm

ine

Obj

ectiv

es,

Alte

rnat

ives

, Con

stra

ints

Pla

n N

ext P

hase

s

Pro

toty

peB

eta

Sys

tem

Pro

duct

ion

Sys

tem

Pro

cess

Org

aniz

atio

n -

12 (

12/9

4)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

30


8 PROJECT / RISK MANAGEMENT

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56P

roje

ct/R

isk

Man

agem

ent

- 1

(12

/94)

Pro

ject

/Ris

k M

anag

emen

t

31


Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56P

roje

ct/R

isk

Man

agem

ent

- 2

(12

/94)

Ses

sion

Goa

ls

•T

o un

ders

tand

tha

t pl

anni

ng,

and

docu

men

ting

the

plan

ning

, re

duce

s ri

sks

in s

oftw

are

deve

lopm

ent

•T

o di

scus

s ap

prop

riat

e le

vels

of

soft

war

e pl

anni

ng

•T

o sh

ow t

ypes

of

plan

ning

doc

umen

ts f

or s

oftw

are

deve

lopm

ent

32


Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56P

roje

ct/R

isk

Man

agem

ent

- 3

(12

/94)

Wha

t D

o W

e N

eed

To

Pla

n F

or?

Wha

t A

ctiv

ities

Inf

luen

ce Q

ualit

y?

•R

equi

rem

ents

Ana

lysi

s, D

esig

n, C

ode,

and

Tes

ting

•R

evie

ws

and

Sof

twar

e In

spec

tions

(F

orm

al I

n-P

roce

ss

Rev

iew

s)

•C

onfig

urat

ion

Man

agem

ent

and

Cha

nge

Con

trol

•P

roto

typi

ng a

nd B

eta

Tes

ting

•S

uppo

rt P

roce

dure

s

•S

tand

ards

, P

ract

ices

and

Con

vent

ions

Reg

ardi

ng

Dev

elop

men

t an

d S

uppo

rt

33


Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56P

roje

ct/R

isk

Man

agem

ent

- 4

(12

/94)

Pla

nnin

g D

ocum

ents

Onl

y 25

% o

f S

EM

I/S

EM

AT

EC

H C

ompa

nies

hav

eQ

ualit

y P

lans

for

Sof

twar

e

Sof

twar

e W

ithin

SE

MI/S

EM

AT

EC

H, 1

989

Sof

twar

e E

ngin

eerin

g S

urve

y (S

ET

EC

90-

003)

, SE

MA

TE

CH

Doc

umen

t 900

8031

4A-T

R, A

ugus

t 15,

199

0

•S

oftw

are

Qua

lity

Pla

n

−Li

st c

over

ed s

oftw

are

prod

ucts

(sc

ope

of p

lan)

−D

efin

e m

inim

um d

ocum

enta

tion

(pro

duct

def

initi

on)

requ

irem

ents

−D

efin

e m

inim

um r

evie

w r

equi

rem

ents

−Id

entif

y or

ref

eren

ce r

evie

w/

insp

ectio

n

crite

ria

−S

peci

fy s

tand

ards

, pr

actic

es,

and

conv

entio

ns

−S

peci

fy m

inim

um t

estin

g le

vels

- r

equi

red

test

typ

es

−D

efin

e co

nfig

urat

ion

man

agem

ent

polic

ies

−R

equi

re p

rodu

ct-l

evel

pla

ns a

s ne

cess

ary

Inst

itute

of E

lect

rical

and

Ele

ctro

nics

Eng

inee

rs (

IEE

E)

Std

s 73

0-19

89 a

nd 9

83-1

986

34


Pla

nnin

g D

ocum

ents

(con

tinue

d)

Sof

twar

e D

evel

opm

ent P

lan

(Pro

ject

Man

agem

ent P

lan)

-- P

roje

ct m

anag

emen

t - p

roce

ss m

odel

, org

aniz

atio

nal s

truc

ture

-- M

anag

emen

t obj

ectiv

es -

Ris

k m

anag

men

t--

Tec

hnic

al p

roce

sses

-- W

ork

pack

ages

, sch

edul

e, b

udge

t

Sch

edul

ing

expe

rienc

e

10 0203040506070

Pgm

1P

gm 2

Pgm

3P

gm 4

Pgm

5

36

4648

63

13

27

15

25

11

25

Pro

pose

d

Act

ual

Sch

edul

e(M

onth

s)

Sof

twar

e Lo

gist

ics

Nat

iona

l Wor

ksho

p, A

ugus

t 15-

16, 1

989

35


Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56P

roje

ct/R

isk

Man

agem

ent

- 6

(12

/94)

Sof

twar

e D

evel

opm

ent

Pla

nnin

g

•P

roje

ct M

anag

emen

t−

Def

ine

Pro

cess

Mod

el -

Maj

or m

ilest

ones

−H

ow a

re t

he g

roup

s or

gani

zed

with

in t

his

fram

ewor

k?

- D

istin

ctha

rdw

are/

soft

war

e gr

oups

? -

S

ome/

all h

ighl

y in

tegr

ated

?

•M

anag

emen

t O

bjec

tives

- R

isk

Man

agem

ent

−R

equi

rem

ents

, S

ched

ule,

Bud

get

prio

ritie

s

−R

isks

- T

rade

offs

: C

ontr

actu

al a

nd T

echn

olog

ical

•T

echn

ical

Pro

cess

es

−T

ools

- T

echn

ique

s:

Ana

lysi

s an

d D

esig

n M

etho

ds -

Com

pute

r-A

ided

Sof

twar

e E

ngin

eeri

ng (

CA

SE

) to

ols

•W

ork

Pac

kage

s, S

ched

ule,

Bud

get

−D

epen

denc

y of

wor

k pa

ckag

es -

PE

RT

*, G

antt

Cha

rts

−C

ost

and

Sch

edul

e M

odel

s -

CO

CO

MO

* ba

sed

good

sta

rtin

g po

int

* P

ER

TP

rogr

am E

valu

atio

n an

d R

evie

w T

echn

ique

* C

OC

OM

OC

Ons

truc

tive

CO

st M

Ode

l

36


Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56P

roje

ct/R

isk

Man

agem

ent

- 7

(12

/94)

Pla

nnin

g D

ocum

ents

(Con

clud

ed)

•S

oftw

are

Con

figur

atio

n M

anag

emen

t P

lan

–

Man

agin

g

the

soft

war

e pr

oduc

ts

−Id

entif

y so

ftw

are

prod

ucts

; co

ntro

l, im

plem

ent,

rep

ort

chan

ge

−C

hang

e au

thor

ity r

espo

nsib

ilitie

s (C

onfig

urat

ion

Con

trol

Boa

rd)

−S

uppo

rt t

ools

and

tec

hniq

ues

−S

oftw

are

Con

figur

atio

n M

anag

emen

t Li

brar

ies

- R

elea

se L

ibra

ry,

Pro

ject

Libr

ary,

. . .

.

•T

est

Pla

n (S

oftw

are

Por

tion)

Sof

twar

e In

tegr

atio

n,

Har

dwar

e -

Sof

twar

e in

tegr

atio

n,

Sys

tem

37


Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56P

roje

ct/R

isk

Man

agem

ent

- 8

(12

/94)

Wha

t A

ffec

ts P

lan

Con

tent

?

•Y

our

own

corp

orat

e qu

ality

ass

uran

ce p

olic

y

•In

tern

al e

ngin

eeri

ng p

roce

dure

s &

gui

delin

es

•IE

EE

* st

anda

rds

and

guid

es

•S

ET

EC

* -

SE

MA

TE

CH

gui

des

•C

omm

on s

ense

−C

an u

se le

ss t

han

stan

dard

May

nee

d m

ore

than

sta

ndar

d

•In

form

atio

n ca

n be

com

bine

d in

one

doc

umen

t

* IE

EE

Inst

itute

of E

lect

rical

and

Ele

ctro

nics

Eng

inee

rs

* S

ET

EC

Sem

icon

duct

or E

quip

men

t Tec

hnol

ogy

Cen

ter

(San

dia

Nat

iona

l Lab

orat

orie

s)

38


Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56P

roje

ct/R

isk

Man

agem

ent

- 9

(12

/94)

Ris

k M

anag

emen

t S

umm

ary

•P

lann

ing

is n

eces

sary

to

effe

ctiv

ely

and

cons

iste

ntly

prod

uce

qual

ity s

oftw

are

- R

educ

es R

isk

•A

dapt

a s

tand

ard

appr

oach

to

your

par

ticul

ar

envi

ronm

ent

-

Doc

umen

t th

is a

ppro

ach

•S

oftw

are

man

ager

s an

d de

velo

pers

nee

d yo

ur in

tere

st

and

supp

ort

to p

lan

effe

ctiv

ely

•Q

uest

ions

you

sho

uld

ask

−W

hat

is o

ur p

olic

y fo

r so

ftw

are

qual

ity?

−D

o w

e ha

ve s

tand

ards

for

sof

twar

e de

velo

pmen

t?

- W

hat

are

they

?

−D

o w

e ha

ve a

Sof

twar

e D

evel

opm

ent

Pla

n?

- I

s it

curr

ent?

−D

o w

e ha

ve a

cos

t an

d sc

hedu

le e

stim

ate?

−H

ow a

re w

e tr

acki

ng t

o th

e de

velo

pmen

t sc

hedu

le?

39


Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56P

roje

ct/R

isk

Man

agem

ent

- 10

(1

2/94

)

Ris

k M

anag

emen

t S

umm

ary

Act

ion

Pla

n

•W

rite

out

a p

olic

y st

atem

ent

defin

ing

YO

UR

com

pany

's

conc

ept

of q

ualit

y fo

r so

ftw

are

–

Suc

h as

:

(Sam

ple

Pol

icy

Sta

tem

ent)

−"Q

ualit

y so

ftw

are

shal

l be

an o

bjec

tive

of t

his

com

pany

's q

ualit

y pr

ogra

m.

Thi

s ob

ject

ive

appl

ies

to t

he a

cqui

sitio

n, d

evel

opm

ent,

use

, an

d su

ppor

t of

soft

war

e"

"The

pro

gram

sha

ll ad

dres

s, a

s a

min

imum

:

Org

aniz

atio

n, ta

sks,

and

res

pons

ibili

ties;

sof

twar

e do

cum

enta

tion;

con

figur

atio

n m

anag

emen

t;te

stin

g, a

nd; r

evie

ws

and

audi

ts."

•M

ake

an a

ppoi

ntm

ent

with

you

r so

ftw

are

man

ager

to

outli

ne s

oftw

are

deve

lopm

ent

goal

s fo

r yo

ur

next

new

feat

ure

(or

next

gen

erat

ion

of e

quip

men

t)

−In

tegr

ate

Pro

cess

Mod

el f

rom

pre

viou

s ac

tion

item

- T

echn

ical

ris

ks -

Ana

lysi

s an

d D

esig

n m

etho

ds -

Ini

tiate

for

mal

sch

edul

e an

d bu

dget

estim

atio

n, d

efin

e w

ork

pack

ages

40


9 REQUIREMENTS

Req

uire

men

ts -

1

12/9

4T

echn

olog

y T

rans

fer

# 92

1113

89B

-T

RG

/ S

ET

EC

94-0

56Req

uire

men

ts

41


Req

uire

men

ts -

2

12/9

4T

echn

olog

y T

rans

fer

# 92

1113

89B

-T

RG

/ S

ET

EC

94-0

56

Ses

sion

Goa

ls

•T

o un

ders

tand

wha

t a

Sof

twar

e R

equi

rem

ents

Spe

cific

atio

n is

•T

o un

ders

tand

the

impo

rtan

ce o

f re

quir

emen

ts a

naly

sis

and

the

Sof

twar

e R

equi

rem

ents

Spe

cific

atio

n in

soft

war

e de

velo

pmen

t (a

nd s

uppo

rt)

42


Req

uire

men

ts -

3

12/9

4T

echn

olog

y T

rans

fer

# 92

1113

89B

-T

RG

/ S

ET

EC

94-0

56

Wha

t is

a S

oftw

are

Req

uire

men

ts S

peci

ficat

ion?

WH

AT

: •C

onta

ins

a co

mpl

ete

desc

ript

ion

of

wha

t is

to

be d

one,

no

t ho

w t

odo

it

•A

n ag

reem

ent

betw

een

Sup

plie

r an

d C

usto

mer

(T

op le

vel -

Par

tner

ing

for

Tot

al Q

ualit

y is

sue)

, an

d,

Sys

tem

and

S

oftw

are

Dev

elop

ers

(Low

leve

l - R

equi

rem

ents

allo

catio

n to

em

bedd

edso

ftw

are)

•T

he

mos

t im

port

ant

docu

men

t de

scri

bing

the

sof

twar

e

A S

oftw

are

Req

uire

men

ts S

peci

ficat

ion

is

not

a D

esig

n D

ocum

ent

. .

. an

d W

HY

:

•P

rote

cts

both

sof

twar

e de

velo

per

(or

supp

lier)

and

sys

tem

dev

elop

er(o

r cu

stom

er)

from

mis

unde

rsta

ndin

g th

e ot

her's

inte

ntio

n

•H

ow c

an "

corr

ectn

ess"

be

dete

rmin

ed w

ithou

t a

spec

ifica

tion?

•S

erve

s as

the

ba

selin

e f

or s

ubse

quen

t de

velo

pmen

t an

d ve

rific

atio

n

activ

ities

43


Why

Bot

her

with

Req

uire

men

ts?

25-8

0% o

f fau

lts tr

acea

ble

to e

rror

s in

def

inin

g th

e re

quire

men

ts

Onl

y 23

% o

f SE

MI/S

EM

AT

EC

Hco

mpa

nies

alw

ays

writ

e re

quire

men

ts

Cos

t of F

ix a

Fau

lt(G

iven

the

faul

t was

cre

ated

dur

ing

requ

irem

ents

tim

e fr

ame)

Dur

ing

Req

mts

$

195

Dur

ing

Des

ign

48

9D

urin

g C

odin

g

997

Dur

ing

Tes

t

7,13

6D

urin

g S

uppo

rt

E

ven

mor

e!

Rog

er F

ujii,

Log

icon

, Inc

.

45 40 35 30 25 20 15 10 5 0

Req

mts

Des

ign

Cod

eT

est

Sup

port

Whe

n D

efec

ts In

trod

uced

NS

IA J

oin

t C

on

fere

nc

e,

19

87

25 20 15 10 5 0W

hen

Def

ects

Fou

nd/F

ixed

Req

mts

Des

ign

Cod

eTe

st (S

/W In

t)Te

st (S

ys In

t)B

eta

Test

Sup

port

44


Req

uire

men

ts -

5

12/9

4T

echn

olog

y T

rans

fer

# 92

1113

89B

-T

RG

/ S

ET

EC

94-0

56

Sof

twar

e R

equi

rem

ents

Spe

cific

atio

n

•W

hat

does

an

SR

S c

onta

in?

−G

ener

al D

escr

iptio

n of

the

Req

uire

d S

oftw

are

−C

onst

rain

ts,

Ass

umpt

ions

, an

d D

epen

denc

ies

−S

peci

fic R

equi

rem

ents

−S

uppo

rtin

g In

form

atio

n

•IE

EE

Gui

de t

o S

oftw

are

Req

uire

men

ts S

peci

ficat

ions

,

IEE

E S

TD

830

-198

4

−A

n in

dust

ry-w

ide

cons

ensu

s do

cum

ent

−R

ecom

men

ded

outli

ne f

or a

n S

RS

−T

empl

ates

ava

ilabl

e (D

esig

n an

d S

yste

m T

est

tem

plat

es a

lso!

)

45


Req

uire

men

ts -

6

12/9

4T

echn

olog

y T

rans

fer

# 92

1113

89B

-T

RG

/ S

ET

EC

94-0

56

Req

uire

men

ts S

umm

ary

•S

RS

is t

he

MO

ST

impo

rtan

t so

ftw

are

docu

men

t

−E

mbo

dies

the

res

ults

of

requ

irem

ents

ana

lysi

s

•S

RS

spe

cifie

s W

HA

T is

to

be d

one,

but

N

OT

HO

W

•Q

uest

ions

you

sho

uld

be a

skin

g

−D

o w

e kn

ow w

hat

the

requ

irem

ents

are

?

−W

hat

anal

ysis

tec

hniq

ues

are

we

usin

g to

def

ine/

refin

e re

quir

emen

ts?

−A

re t

he r

equi

rem

ents

doc

umen

ted?

−W

ho i

s o

ur c

usto

mer

? -

Who

is t

he u

ser?

−H

ave

we

draf

ted

the

Sys

tem

Tes

t P

lan?

•D

on't

ask!

−H

ow m

uch

code

hav

e yo

u w

ritt

en?

46


Req

uire

men

ts -

7

12/9

4T

echn

olog

y T

rans

fer

# 92

1113

89B

-T

RG

/ S

ET

EC

94-0

56

Req

uire

men

ts S

umm

ary

Act

ion

Pla

n

•M

eet

with

(in

terv

iew

) cu

stom

er p

rodu

ctio

n m

anag

ers.

Ask

: W

hat

are

thei

r to

p fiv

e re

quir

emen

ts f

or y

our

com

pany

's

equi

pmen

t.

•M

eet

with

you

r sy

stem

/pro

ject

and

sof

twar

e m

anag

ers

Ask

: H

ow a

re w

e m

eetin

g th

ese

requ

irem

ents

?

47


10 REVIEWS AND INSPECTIONS

Rev

iew

s &

Ins

pect

ions

- 1

12

/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Rev

iew

s an

d In

spec

tions

48


Rev

iew

s &

Ins

pect

ions

- 2

12

/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Ses

sion

Goa

ls

•Id

entif

y ch

arac

teri

stic

s of

a r

evie

w

•D

iscu

ss t

ypic

al li

fe c

ycle

rev

iew

poi

nts/

type

s of

re

view

s

•C

onvi

nce

you

that

for

mal

rev

iew

s ar

e im

port

ant

49


Rev

iew

s &

Ins

pect

ions

- 3

12

/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056S

oftw

are

Rev

iew

s

•W

hat

is r

evie

wed

?

−S

oftw

are

elem

ents

or

proj

ect

stat

usP

rodu

ct

- P

lans

- P

roce

sses

Sch

edul

e -

Cos

t -

Res

ourc

es A

vaila

ble

•W

hy a

re r

evie

ws

held

?

−Id

entif

y di

scre

panc

ies

(A

s th

ey o

ccur

! -

Ins

tead

of

findi

ng t

hem

late

r,

e.g.

, du

ring

sys

tem

tes

t or

lett

ing

the

cust

omer

fin

d th

em)

Def

ects

- N

on-c

onfo

rman

ce -

Inc

onsi

sten

cies

/Am

bigu

ities

•W

hat

are

the

resu

lts o

f th

e re

view

s?

−R

ecom

men

datio

ns f

or im

prov

emen

tsP

riorit

ize

- O

wne

rshi

p -

Val

idat

e w

ith fo

llow

-up

revi

ew (

as n

eces

sary

)

50


Rev

iew

s &

Ins

pect

ions

- 4

12

/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Typ

ical

Sof

twar

e Li

fe C

ycle

Rev

iew

Poi

nts

•R

equi

rem

ents

Rev

iew

•D

esig

n R

evie

w

•C

ode

Rev

iew

•T

est

Rev

iew

•R

elea

se R

evie

w

51


Rev

iew

s &

Ins

pect

ions

- 5

12

/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Req

uire

men

ts R

evie

ws

SU

CC

ES

S T

IP

* P

roje

ct s

ucce

ss d

epen

ds o

n a

good

und

erst

andi

ng

of t

he r

equi

rem

ents

•T

echn

ical

pro

ject

rev

iew

of

Sof

twar

e R

equi

rem

ents

Spe

cific

atio

n –

B

asel

ine

•F

orm

al R

evie

w (

Insp

ectio

n) o

f S

oftw

are

Req

uire

men

ts

Spe

cific

atio

n fo

r di

scre

panc

ies

•M

anag

emen

t w

alkt

hrou

gh w

ith c

usto

mer

Onl

y 18

% o

f S

EM

I/S

EM

AT

EC

H C

ompa

nies

alw

ays

form

ally

rev

iew

Req

uire

men

ts -

8%

nev

er d

o

52


Rev

iew

s &

Ins

pect

ions

- 6

12

/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Des

ign

Rev

iew

•T

echn

ical

pro

ject

rev

iew

of

Sof

twar

e D

esig

n D

ocum

ent

−C

ritic

al D

esig

n R

evie

w;

Bas

elin

e

•P

eer

wal

kthr

ough

of

mod

ule

desi

gn

•In

spec

tion

of S

oftw

are

Des

ign

Doc

umen

t fo

r de

fect

s

•M

anag

emen

t w

alkt

hrou

gh w

ith c

usto

mer

Onl

y 19

% o

f S

EM

I/S

EM

AT

EC

H C

ompa

nies

alw

ays

hold

Des

ign

Rev

iew

s -

3% n

ever

do

53


Rev

iew

s &

Ins

pect

ions

-7#

12

/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Cod

e R

evie

w

•P

eer

wal

kthr

ough

of

mod

ules

•P

eer

wal

kthr

ough

of

unit

test

pla

n/re

sults

•In

spec

tion

of s

ourc

e co

de f

or d

efec

ts

−S

tron

gly

Rec

omm

end!

Onl

y 8%

of

SE

MI/

SE

MA

TE

CH

Com

pani

esal

way

s fo

rmal

ly I

nspe

ct C

ode

- 29

% n

ever

do

54


Rev

iew

s &

Ins

pect

ions

- 8

12

/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Tes

t R

evie

ws

Pee

r w

alkt

hrou

gh o

r In

spec

tion

of T

est

Pla

ns a

nd R

esul

ts

Onl

y 5%

of

SE

MI/

SE

MA

TE

CH

Com

pani

esal

way

s fo

rmal

ly r

evie

w T

est

Pla

nnin

g -

40%

nev

er d

o

Sta

tus

Re

vie

ws

•T

rans

ition

poi

nts

- M

inim

um

•P

lann

ed v

s A

ctua

l

Cos

t, s

ched

ule,

and

tec

hnic

al p

rogr

ess

•R

esul

ts o

f te

chni

cal r

evie

ws

Tec

hnic

al p

roce

sses

in c

ontr

ol -

Too

man

y/to

o fe

w d

iscr

epan

cies

55


Rev

iew

s &

Ins

pect

ions

- 9

12

/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Sof

twar

e In

spec

tions

*(F

orm

al I

n-P

roce

ss R

evie

ws)

•D

efin

ition

:

−A

for

mal

eva

luat

ion

tech

niqu

e in

whi

ch s

oftw

are

prod

ucts

(req

uire

men

ts,

desi

gn,

code

, et

c.)

are

exam

ined

in d

etai

l by

a gr

oup

to

dete

ct f

aults

, vi

olat

ions

of

deve

lopm

ent

stan

dard

s, a

nd o

ther

pro

blem

s

−A

str

uctu

red

pee

r re

view

req

uiri

ng p

lann

ing,

adv

ance

pre

para

tion,

an

d

poss

ible

rew

ork

and

follo

w-u

p

−A

sta

tic t

est

of

the

soft

war

e

•P

urpo

se:

−P

reve

nt t

he p

ropa

gatio

n of

def

ects

into

the

op

erat

iona

l pro

duct

−K

ey f

acto

r in

roo

t-ca

use

anal

ysis

to

impr

ove

proc

ess

−P

roce

ss c

ontr

ol m

etri

c

* T

he te

rm S

oftw

are

Insp

ectio

n ha

s re

mai

ned

the

com

mon

nam

e fo

r th

is ty

pe o

f For

mal

Rev

iew

.

56


Rev

iew

s &

Ins

pect

ions

- 1

0 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Insp

ectio

ns

•C

reat

ed in

197

2 at

IB

M -

Mic

hael

Fag

an

−In

stitu

tiona

lized

by

larg

e so

ftw

are

deve

lopm

ent

orga

niza

tions

(e.

g.,

IBM

, H

P,

AT

&T

)

•P

roce

ss I

mpr

ovem

ent

inpu

t

−W

hen/

Why

did

err

or o

ccur

tha

t cr

eate

d th

e de

fect

?

−H

ow c

an t

his

be p

reve

nted

in t

he f

utur

e?

•T

HE

P

roce

ss C

ontr

ol M

echa

nism

for

sof

twar

e

−A

n ai

d to

pro

duct

ivity

as

wel

l as

qual

ity

−F

indi

ng a

nd f

ixin

g de

fect

s as

the

y oc

cur

inst

ead

of t

ryin

g to

tes

t th

em

out

57


Rev

iew

s &

Ins

pect

ions

- 1

1 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Com

pari

son

of D

efec

t Id

entif

icat

ion

Tec

hniq

ues

Met

hod

Cos

tE

ffic

ienc

y

Insp

ectio

nsM

ediu

m-H

igh

>60

%

Wa

lkth

rou

gh

Me

diu

m<

50

%

Pee

r R

evie

wLo

w<

35%

Sel

f C

heck

ing

Low

<20

%

Sou

rce:

Cap

ers

Jone

s,

Sof

twar

e M

easu

rem

ent a

nd E

stim

atio

n

Why

?

−M

ore

form

ality

and

rig

or

−D

efin

ed m

etho

dolo

gy f

or in

spec

tions

−R

ecor

ds c

aref

ully

kep

t

−A

ll pa

rtic

ipan

ts a

re a

ctiv

e an

d re

spon

sibl

e

−R

epea

tabl

e -

re

peat

-

Rep

eata

ble!

58


Rev

iew

s &

Ins

pect

ions

- 1

2 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Sof

twar

e In

spec

tions

•C

ON

S

−M

IST

AK

EN

AS

A "

FIN

AL

INS

PE

CT

ION

" IN

TH

E D

EM

ING

SE

NS

E

−C

an a

dd 5

-15%

to

net

reso

urce

s up

fro

nt*

−R

equi

res

som

e tr

aini

ng

−M

ista

ken

as t

oo "

low

tec

h" t

o be

so

effe

ctiv

e

•P

RO

S

−H

igh

retu

rn o

n in

vest

ed t

ime

and

effo

rt

−F

eedb

ack

to d

evel

oper

s -

avoi

d in

ject

ing

defe

cts

in f

utur

e w

ork

−S

erve

s as

che

ckpo

ints

to

faci

litat

e pr

oces

s m

anag

emen

t

−M

easu

re p

erfo

rman

ce o

f to

ols

and

tech

niqu

es

−T

rain

ing

aid

to n

ew o

r le

sser

cap

able

peo

ple

*A

bout

the

sam

e am

ount

of t

ime

is s

pent

pre

parin

g fo

r th

e in

spec

tion

as th

e in

spec

tion

itsel

f —R

ates

are

appr

oxim

atel

y on

e ho

ur p

er 1

0 do

cum

ent p

ages

, or

one

hour

per

100

line

s of

non

-com

men

ted,

non

-bla

nk

sour

ce c

ode

59


Rev

iew

s &

Insp

ectio

ns -

13

12/

94T

echn

olog

y T

rans

fer

# 92

1113

89B

-T

RG

/ S

ET

EC

94-0

56

Ben

efits

of

Insp

ectio

ns

•D

efec

t R

educ

tion

50 -

90%

of

all d

efec

ts d

isco

vere

d by

insp

ectio

n

•C

ost

Impr

ovem

ent

10-2

5% r

educ

tion

in d

evel

opm

ent

cost

s

95%

red

uctio

n in

cor

rect

ive

mai

nten

ance

cos

ts

•S

taff

-Hou

rsO

vera

ll re

duct

ion

by 1

0-40

%

Sho

rten

s ta

il en

d of

sch

edul

e

Mye

rs,

Com

pute

r,

Aug

ust

1990

60


Rev

iew

s &

Ins

pect

ions

- 1

4 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Insp

ectio

n E

xper

ienc

e S

umm

ary

•In

spec

tions

are

thre

e tim

es m

ore

effe

ctiv

e th

an o

ther

met

hods

•S

ave

appr

oxim

atel

y $1

600

for

ever

y de

fect

bef

ore

test

(C

ost t

o fix

late

r -

cost

to fi

nd &

fix

in

insp

ectio

n: $

1700

- $

105)

•A

vera

ge in

spec

tion

disc

over

s 16

def

ects

(4

maj

or, 1

2 m

inor

) fo

r $2

5,00

0 sa

ving

s

•S

ome

defe

cts

cost

as

muc

h as

$10

,000

eac

h to

fix

late

r

JPL*

Exp

erie

nce

IEE

E,

Exp

erie

nce

Rep

ort,

Aug

ust 1

990

* Je

t Pro

puls

ion

Labo

rato

ry

The

diff

eren

ce in

the

are

aun

der

thes

e tw

o cu

rves

afte

r th

ey in

ters

ect

repr

esen

ts th

e "F

inal

Insp

ectio

n" to

det

ect a

llth

e de

fect

s in

trod

uced

durin

g a

deve

lopm

ent

proc

ess

in w

hich

the

softw

are

insp

ectio

nte

chni

que

is n

ot u

sed.

Peo

ple

Res

ourc

es

Pla

nnin

g

Req

uire

men

ts

Des

ign

Shi

pC

odin

gT

estin

g

With

out

Sof

twar

eIn

spec

tions

With

Sof

twar

eIn

spec

tions

Sch

edul

e

(Fag

an '8

6)

61


Rev

iew

s &

Ins

pect

ions

- 1

5 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Insp

ectio

ns

•P

artic

ipan

ts h

ave

defin

ed r

oles

Mod

erat

orpl

ans

agen

da, d

eter

min

es a

dequ

ate

prep

arat

ion,

run

mee

ting,

set

tone

and

pac

e, a

ct a

s in

spec

tor

Aut

hor

prov

ides

cla

rific

atio

n, a

ct a

s in

spec

tor(

usua

lly th

e be

st)

Re

ad

er

para

phra

ses

line-

by-li

ne c

onte

nt, a

ct a

s in

spec

tor

Rec

orde

rre

cord

s pr

oble

m a

nd p

robl

em c

lass

ifica

tion,

act

s as

insp

ecto

r, c

an

be m

oder

ator

Insp

ecto

rspr

ovid

es e

xper

tise

in r

elat

ed m

atte

rs; r

epre

sent

ativ

es o

f pre

viou

s

and

subs

eque

nt p

hase

s

•In

spec

tions

hav

e de

fined

ste

ps

1.

Pla

nnin

g4.

In

spec

tion

Mee

ting

2.

Ove

rvie

w

(if

nece

ssar

y)5.

R

ewor

k

3.

Pre

para

tion

6.

Fol

low

-up

62


Rev

iew

s &

Ins

pect

ions

- #

16 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056W

hat

Can

You

Do?

•D

o no

t a

tten

d!

Insp

ectio

ns a

re n

ot a

pla

ce f

or m

anag

ers

•A

sk f

or a

Sum

mar

y R

epor

t

•R

EJO

ICE

whe

n lo

ts o

f de

fect

s ar

e di

scov

ered

in

insp

ectio

ns!

Don

't us

e in

spec

tion

resu

lts a

s a

basi

s fo

r m

erit

or c

ompe

nsat

ion

revi

ew

- U

se t

est

resu

lts

•E

ncou

rage

you

r st

aff

63


Rev

iew

s &

Ins

pect

ions

- 1

7 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Rev

iew

s an

d In

spec

tions

Sum

mar

y

•R

evie

ws

mar

k tr

ansi

tion/

clos

ure

•S

oftw

are

Insp

ectio

n is

the

rec

omm

ende

d T

otal

Qua

lity

Man

agem

ent

tech

niqu

e fo

r so

ftw

are

proc

ess

cont

rol

•Q

uest

ions

you

sho

uld

ask

Wha

t re

view

s ar

e pl

anne

d?

- W

hich

one

s sh

ould

I a

tten

d?

How

hav

e w

e in

sure

d th

e re

quir

emen

ts s

peci

ficat

ion

(des

ign)

is c

orre

ct

(com

plet

e)?

How

muc

h of

the

req

uire

men

ts s

peci

ficat

ion

(des

ign,

cod

e, t

est

plan

)

has

been

(w

ill b

e) in

spec

ted?

-

May

I s

ee t

he I

nspe

ctio

n M

anag

emen

t

Rep

ort?

64


Rev

iew

s &

Ins

pect

ions

- 1

8 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B -

TR

G /

SE

TE

C94

-056

Rev

iew

s an

d In

spec

tions

Sum

mar

yA

ctio

n P

lan

•D

eter

min

e w

hat

kind

of

revi

ews

are

curr

ently

bei

ng

cond

ucte

d to

con

trol

sof

twar

e qu

ality

Dis

cuss

with

sof

twar

e m

anag

er -

Thi

s ca

n be

don

e th

is a

fter

noon

!

•In

itiat

e an

eva

luat

ion

of s

oftw

are

insp

ectio

ns

−H

ave

soft

war

e m

anag

er s

ched

ule

soft

war

e de

velo

pers

(an

d m

anag

er)

for

trai

ning

in s

oftw

are

insp

ectio

ns

Cla

sses

taug

ht r

egio

nally

(ea

st &

wes

t coa

st)

and

in-h

ouse

−A

pply

insp

ectio

n to

a c

urre

nt w

ork

prod

uct

Tw

o im

med

iate

ben

efits

:

(1)

Impr

oved

wor

k pr

oduc

t

(2)

Insi

ght i

nto

nece

ssar

y im

prov

emen

ts in

dev

elop

men

t act

iviti

es le

adin

g up

tocu

rren

t wor

k pr

oduc

t

65


11 CONFIGURATION MANAGEMENT

Con

figur

atio

n M

anag

emen

t -

1 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56

Con

figur

atio

n M

anag

emen

t

66


Con

figur

atio

n M

anag

emen

t -

2 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56

Ses

sion

Goa

ls

•In

trod

uce

you

to e

lem

ents

of

the

Sof

twar

e

Con

figur

atio

n M

anag

emen

t pr

oces

s

•H

elp

you

unde

rsta

nd t

hat

Sof

twar

e C

onfig

urat

ion

Man

agem

ent

is a

n es

sent

ial a

spec

t of

the

sof

twar

e

deve

lopm

ent

proc

ess

67


Con

figur

atio

n M

anag

emen

t -

3 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56

Sof

twar

e C

onfig

urat

ion

Man

agem

ent

(SC

M)

•Id

entif

icat

ion

−La

bel,

num

ber,

and

cat

alog

pro

cedu

res

•C

ontr

ol

−P

rodu

ct c

hang

e au

thor

ity a

nd c

ontr

ol p

roce

dure

s

−S

oftw

are

Con

figur

atio

n C

ontr

ol B

oard

•S

tatu

s A

ccou

ntin

g

−P

rodu

ct c

hang

e re

cord

ing

and

repo

rtin

g

•A

udits

−P

hysi

cal a

nd f

unct

iona

l com

plia

nce

with

req

uire

men

ts p

rior

to

rele

ase

Onl

y 28

% o

f S

EM

I/S

EM

AT

EC

H C

ompa

nies

alw

ays

use

Con

figur

atio

n M

anag

emen

t

68


Con

figur

atio

n M

anag

emen

t -

4 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56Ben

efits

of

SC

M

•C

ontr

ol a

nd m

anag

e so

ftw

are

thro

ugho

ut it

s lif

e cy

cle

•R

educ

e ef

fort

to

iden

tify

and

impl

emen

t ch

ange

s

•E

nsur

e on

ly n

eces

sary

cha

nges

will

be

mad

e

•E

xten

d th

e us

eful

life

of

soft

war

e pr

oduc

ts

•R

educ

e de

fect

s in

sof

twar

e re

leas

es

•Im

prov

e cu

stom

er s

atis

fact

ion

•P

rovi

de a

nat

ural

fee

dbac

k m

echa

nism

69


Con

figur

atio

n M

anag

emen

t -

5 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56

Som

e C

omm

on Q

uest

ions

for

SC

M

•W

hich

sof

twar

e pi

eces

/par

ts d

o w

e co

ntro

l?

−In

tern

ally

dev

elop

ed p

lans

, sp

ecifi

catio

ns,

sour

ce,

obje

cts,

tes

ts,

data

,

envi

ronm

ent

−P

urch

ased

sof

twar

e do

cum

enta

tion

and

code

•W

hen

does

Sof

twar

e C

onfig

urat

ion

Man

agem

ent

begi

n?

−D

urin

g ex

plor

atio

n -

S

omet

imes

Pla

nnin

g, in

form

al c

ontr

ol

−D

urin

g de

velo

pmen

t -

Alw

ays

Pla

nnin

g, p

erso

nal &

dev

elop

men

t lib

rarie

s* -

con

trol

boa

rds

−D

urin

g su

ppor

tP

rodu

ct o

r re

leas

e lib

rarie

s -

rele

ase

cont

rol

* L

ibra

ries

are

def

ined

in a

ttac

hed

pape

r

70


Con

figur

atio

n M

anag

emen

t -

6 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56

Som

e C

omm

on Q

uest

ions

for

SC

M(C

ontin

ued)

•H

ow d

o w

e co

ntro

l cha

nge/

rele

ase

of t

he s

oftw

are?

Bas

elin

e* c

ontr

ol p

oint

s -

S

oftw

are

Con

figur

atio

n C

ontr

ol B

oard

-

Cha

nge

log

mec

hani

sm

- S

oftw

are

libra

ry-m

anag

emen

t to

ols

•W

hat

do w

e re

port

abo

ut c

hang

es?

Wha

t ch

ange

s w

ere

mad

e -

W

ho a

ppro

ved

-

Who

mad

e ch

ange

-

Whe

n ch

ange

was

mad

e -

H

ow m

uch

time

was

req

uire

d

•H

ow d

o w

e ch

eck

for

com

plet

enes

s, c

orre

ctne

ss?

Aud

it -

B

efor

e te

st,

befo

re r

elea

se

* B

asel

ine

- T

he d

ocum

ente

d id

entif

icat

ion

of a

pro

duct

(w

ith a

kno

wn

set o

f ele

men

ts)

afte

r so

me

perio

d of

dev

elop

men

t

71


Con

figur

atio

n M

anag

emen

t -

7 1

2/94

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56

Con

figur

atio

n M

anag

emen

t S

umm

ary

•S

oftw

are

Con

figur

atio

n M

anag

emen

t

−Is

an

impo

rtan

t m

anag

emen

t pr

oces

s

−Is

mor

e th

an v

ersi

on c

ontr

ol

−C

an b

e ta

ilore

d to

the

pro

duct

line

or

proj

ect

•Q

uest

ions

you

sho

uld

ask

−D

o w

e ha

ve a

Sof

twar

e C

onfig

urat

ion

Man

agem

ent

Pla

n?

−C

an w

e re

build

the

cur

rent

rel

ease

(ba

selin

e)?

−C

an w

e re

build

the

pre

viou

s re

leas

e (b

asel

ine)

?

•A

ctio

n P

lan

−A

sk f

or a

n ov

ervi

ew o

f ho

w p

robl

em r

epor

ting,

and

req

uest

s fo

r ne

w

feat

ures

, ar

e re

ceiv

ed,

revi

ewed

, an

d so

ftw

are

chan

ges

inco

rpor

ated

in

your

com

pany

72


12 FORMAL SOFTWARE CONFIGURATION MANAGEMENTAND TIME-PHASED VERSION (BASELINE) RELEASE

Overview

SEMI/SEMATECH companies should adopt a more formal software configuration management(SCM) methodology to manage change activity and control the system software configuration.Part of the methodology should be to adopt a time-phased version (baseline) release processunder the more formal SCM approach. (A glossary is attached that provides some commondefinitions of SCM terms.)

The general benefits to SEMI/SEMATECH companies of a more formal SCM approach includean increased confidence in the ability to preserve the integrity of their software configuration; anability to provide complete identification of prior, current release, and developmental baselines;and a ready accounting of the status of changes completed, in progress and awaiting work atscheduled times. The benefits to developers include a more stable development environment. Adefined set of changes can be integrated as a common development objective and adequatetesting can be completed to validate the reliability of the system. The benefits to managementinclude insight into the software engineering process. This allows management to effectivelyaddress the long term quality of the equipment software while attending to customer prioritiesand other market considerations. The benefits to the customer include more accurate informationas to when corrections to problems can be expected or when new features will be available.

Structure

An outline of a formal SCM approach is shown in Figure 1. Highlights of this approach include(1) establishment of a six month normal release cycle; (2) an automated change request log basedon the current problem report format; (3) establishment of an automated change controlmechanism to include a hierarchy of SCM Libraries (Personal or Developmental, Project orIntegration, and Release) for control of requirements, design information, code, and testdocumentation; and (4) establishment of a change control authority (Software ConfigurationControl Board or SCCB) consisting of representatives from the subsystem groups.

73


TestCCBChair

1

Analysis

Field Log

CustomerChangeAuthority

2

Developmentor New Engineering

Originator

TechComm

Evaluation Personal Lib

3

SoftwareInspection

Development Lib

ReleaseVersion

ReleaseControl

Release Lib

Test Failure

Field Report

CustomerRequest

Report

Report

Defer

AcceptReject

Change Activity

Promotion

Implementation

CopyImplementation

Promotion

Version ReleaseControl

Next VersionNext Version

AnalysisResults

Copy

Promotion

User Info

Figure 1 SCM Context

Establishment of a release cycle timeframe is essential to the overall success of maintaining orimproving software quality and reliability. Every release of the software, even a “patch,” incurs alevel of overhead not only in preparing and shipping the release, but in other activities, such asintegration and system testing. Normally, each change has some individual testing performedagainst it. However, if a release is made for a small number of changes, and the releases arefrequent, the system (and regression) testing to support each release becomes a significanceportion of the overall support cost. By maintaining a scheduled release cycle, integration andsystem (regression) testing of the combined release package reduces these overhead costs whilesignificantly increasing the confidence in the delivered software. The increased confidence isattributable to being able to test more thoroughly since (1) each system test can directly orindirectly exercise more of the changed software (many times, the same test is performed toactivate two or more components of the software in a sequential process), and (2) there is time toperform more tests because such activities as test setup can be combined. Note that a time-phasedrelease cycle does not mean that no work is performed on the next release until the previousrelease is shipped. There is usually some overlap if resources will allow it. A six-month releasecycle could actually produce releases five months apart as shown in Figure 2.

74


Release 1.26 Months

Release 1.36 Months

Release 1.46 Months

1Mon

1Mon

5Mon

Figure 2 Normal Release Cycling

An automated change request log is necessary to accurately keep track of change requests andreduce the burden on personnel to manually mark, retrieve, sort, store, etc., the change requestsin hardcopy form.

Establishment of a hierarchy of libraries provides a more secure mechanism to maintain theintegrity of released software through promotion from one library to the next. Promotion is acontrol mechanism that requires changed software to be subjected to an established level ofscrutiny before it is formally approved for entry into a higher level library. (The SCCB or aninspection team acting for the SCCB may promote software components). This control, coupledwith the ability to manage the configuration of the documentation defining the software system,reduces the defects because of incorrect or outdated support documentation, and increases thepotential to reuse significant portions of the software when developing the next generationequipment.

The change control authority would normally consist of a few software developers, systemengineers, and test/quality engineers (if they exist). These would be personnel most capable ofrendering accurate decisions about the mix of change activity that should be attempted for anygiven release cycle. Completing this decision process sometimes forces issues to be addressedand resolved that would perhaps go unresolved (or even unnoticed) until the consequencesbecome unrecoverable. Customer requests (demands) for new features may or may not be moreimportant than the correction of defects affecting operational uptime and throughput.

Change Request/Problem Reporting

All change activity is processed through a Log function. A Librarian would log the changerequest. The change request could be for a problem report (e.g., field failure report) a request fora new feature or a request for a conversion effort to accommodate new hardware or software(e.g., extensive update to operating system impacting application programs). Table 1 contains asummary of change types appropriate for software.

75


Table 1 Change Profiles

Correction New Feature Conversion

Principle InputSource(s)

Field Report, TestFailures

Customer Request,SEMATECH Initiative,Internal Initiative

Internal Initiative-(Hardware TechnologyDriven), Vendor Driven- (Hardware or SupportSoftware)

Origin Requirement NotCorrect, Non-conformance toRequirements (Design,Code Defect) - (Bug orPossible Bug)

New/ChangedRequirement -(Suggestion, New Item,or Improvement)

New/ChangedRequirement - (NewItem or Improvement)

Priority Level Normal, Important(Urgent), or Critical(Emergency)

Usually Normal orImportant (Urgent)

Usually Normal

Complexity Low, Medium, High Low, Medium, High Low, Medium, High

The change request/problem report log becomes the basis for tracking status (open, closed,deferred, or rejected) and implementation in releases.

Change Control/Configuration Management

The activities of change control would be performed by a group commonly referred to as theSCCB. The SCCB should have a Chairperson, a Librarian (maybe one for each engineeringgroup or a single person if dedicated to Librarian functions), and sufficient representation anddecision authority to determine the change activity for each release cycle. This may requirecoordination with other groups within the company or within the development project, withindividual customers, and with SEMATECH. The coordination would establish thatfeatures/customer requests and initiatives can be accommodated within a given release cyclealong with corrections needed based on field reports and internal test failures. The SCCB shouldplan to meet on a regular basis, and whenever emergency requests must be processed.

Preliminary analysis (Figure 1) is accomplished to provide the SCCB with sufficient informationto make decisions quickly as to accept, defer, or reject changes. The information necessary forpreliminary analysis is highlighted in Table 2. It is essential that the SCCB not get sidetracked ontroubleshooting or design issues at this point in the process. The purpose is to weigh the analysisinformation against the resources available and establish a status for the change. Depending onthe nature of the change, the board may have to suspend activity on one or more changes toaccommodate an extremely sensitive change. The value of the preliminary analysis in thissituation is to provide enough insight into that sensitive change to determine if it is even possibleand really prudent to take such action at that time. A decision to reject a change is usuallyaccompanied by the reasoning used to make that decision, which could be that insufficientinformation precluded the ability to analyze the problem. A decision to defer is usually based oninsufficient analysis and the change request is returned to the analyst for more information. Adecision to accept is accompanied by a decision as to which release the change should bescheduled for. The change implementation package is then sent to the developer forimplementation. The decision is also communicated back to the originator.

76


Table 2 Contents Of Change Request Analysis


Estimated Time:(Person-Months)

Estimated Time Estimated Time

Possible causes/Most likely Sizing/Timing Estimate -Addition computational resourcesrequired?

Relevant impact of new hardware -new operating system/application

Hardware/Software interfaceimpact?



Software only to correct? Test Outline

Modify existing tests? - New testsrequired?

Modify existing tests? - New testsrequired?

Changes required to Userinformation?

New User information required? New User information required?

The mechanism to handle a safety-related (personnel or equipment) critical or "emergency"change (perhaps even a market-strategic change) is to send the change directly into development,apply all available resources to make the change, test it (usually limited testing), and issue an“interim release.” Changes of this nature are usually folded back into the normal SCM processfor a thorough review and complete testing before inclusion into the next normal version release.

The development activity proceeds through testing of the change and completion ofdocumentation. As soon as practical, user information is passed to any documentation group suchas a Technical Communications group for inclusion into user publications and feedback to theoriginator or a representative user group. The SCCB evaluates the results of each completedchange package (Table 3) for promotion into the Release Library. The change package itself canconsist of the output and reporting of very structured verification and validation activitiesdepending on the nature of the change. The entire SCCB or a portion of it may function in thisevaluation activity. Promotion from the Personal Library to the Project (Integration) Library maybe the result of an inspection of the change. Small groups (3 to 5 people) have been shown tovery effective at discovering defects prior to test by using the inspection technique. Acting as thechange control authority, inspection teams form to approve changes (singularly or in groups) forpromotion to the Project Library for integration and system testing.

Table 3 Contents of Change Implementation Package


Actual Time

(Person-Months)

Actual Time Actual Time

Old & New files or New file and"Diff List"

Requirements and Designinformation - Source Code

Old & New files or New file and"Diff List"

Inspection Report, Peer Reviewcomments and/or Walkthroughresults

Inspection Report Inspection Report and/orWalkthrough Results

Tests completed - New Test Cases -Results

Test Cases - Results Tests completed - New Test Cases -Results

Updated documentation Updated documentation

77


Prior to final release, the SCCB insures sufficient system testing has been completed (either in-house or at a Beta site) to establish the overall integrity and reliability of the version to bereleased.

Note the metric of estimated time (Table 2) and actual time (Table 3). This information and anestimate of the complexity of the change (Table 1) could provide the basis of a longer term effortto more accurately estimate the resources required to perform support activities.

Software Engineering Activities

The software engineering activities center on analysis, development and some of the promotionmethods. Analysis is performed by a person knowledgeable in the area of the change request. Thesoftware developer benefits from this exercise because it provides him or her a preliminary lookat the request and the opportunity to begin thinking about the problem without the pressure toimmediately start solving it.

Development is of course the application of the same engineering activities in the supportenvironment that occurred in the original development. Taken as a group, the approved changesfor any particular release are in fact, a whole new development effort. Part of the engineeringactivity should include software inspections also mentioned as part of the change control activity.Software inspections are an engineering activity that serves management issues as well. Becausesoftware inspections are performed on completed products (at least a minimum number of TBDsin the case of requirements), they herald milestones and transitions as well as being the mostefficient defect removal mechanism in the software developer's inventory.

The SCM Library hierarchy must not be confused with other libraries that are used to groupsimilar software entities such as support tools, scripts, include files, test files, etc. Some of theselibraries may be found at multiple levels of the SCM Library structure depending on whetherthey have files being modified as part of a change package.

Summary

The proposed Software Configuration Management philosophy and Time-Phased version releaseprocess are not readily separable. They are both key components to improving the quality andreliability of equipment software.

As mentioned earlier, a software package specifically designed to automate much of the reporttracking and control of the SCM Libraries is important to the success of this concept.

78


13 GLOSSARY OF SOFTWARE CONFIGURATION MANAGEMENTTERMINOLOGY

Baseline: A baseline is the documented identification of a software product CI -- itscode and all its related documentation at some specific point in time. It is thebasis for all SCM activities.

CCB: Configuration Control Board. A Configuration Control Board is a group ofpeople responsible for evaluating and approving or disapproving proposedchanges to configuration items, and for ensuring implementation of approvedchanges. A CCB is sometimes referred to as a Change Control Board.

CI: Configuration Item. A collection of (software) elements (or components)treated as a unit for the purpose of configuration management. ConfigurationItems can vary in size, complexity, and type. A CI may also be called aComputer Software CI (CSCI) or Computer Program CI (CPCI).

CM: Configuration Management.

CPC: Computer Program Component, same as CSC.

CSC: Computer Software Component. A lower level entity of a CI. A CSC may bethe requirements or a collection of the design information for a CSCI.

Inspection: (1) A formal evaluation technique in which software requirements, design, orcode are examined in detail by a group to detect faults, violations ofdevelopment standards, and other problems. (2) A structured peer reviewrequiring advance preparation and planning. (3) Software inspections areformal peer reviews of code or associated documentation includingrequirements specifications, design descriptions, and test plans. Any productgenerated during the course of the software life-cycle can be inspected,although it may not always be advantageous to do so. Software inspectionsare formal in that there is a set agenda. A limited amount of material iscovered in a set amount of time. Those attending the review have clearlydefined roles. All issues raised at the inspection are rigorously recorded andresolved. Software inspections are peer reviews in that management isgenerally not invited to attend.

Personal Library: The Personal Library (also termed the dynamic or programmer's library) isused for holding newly created or modified software entities. This libraryconstitutes a software developer's workspace for writing new code ordocumentation, and may take any form suitable to the developer's needs, butshould have a degree of order to it that allows the status of its entities to bedetermined easily. Each software developer may have a Personal Libraryfrom which project entities can be linked and/or copied. Or, each small teammay have a Personal Library assigned to the team with lower level personallibraries assigned to each individual, especially in a local area network (LAN)or mainframe environment. Access to the Personal Library is controlled by,and usually limited to, the software developer.

79


Project Library: The Project Library (also termed the controlled or "master" library) is a libraryused for managing the current internal developmental baselines and forcontrolling changes made to them. This library represents the latest internally-approved version of the software product being developed. Changes tosoftware entities in the Project Library should have gone through formalapproval procedures established in a configuration management plan. Code inthis library should have been tested sufficiently to assure that it is ready forintegration. Copies may be freely made for use by the software developers andothers, but changes must be strictly controlled and documented in order toascertain at any given point its exact configuration. Even for simple projects inwhich there is only one code developer, there still should be functionalseparation between the Personal Library and the Project Library.

Promotion: A promotion is an action taken with a software component to increase thelevel of authority needed to approve changes to it. For example, a top-levelsoftware design description is promoted or moved into the Project Librarywhere all developers can view, but not modify it without proper authority.This allows the developers to work on issues that may concern detaileddesigns and implementation.

Release: A release is a copy of the software CI that is turned over to the customer oruser. It is a promotion of that CI outside of the development organization.

Release Library: The Release Library (also termed the static or "software repository" library) isa library used to archive the various baselines released for general use. Thislibrary is never changed (except to add a new release baseline), since it mustbe able to duplicate results from software that has been released foroperational use by other organizations. Access should be limited to "readonly" for the purpose of making copies.

Revision: A revision is a formal change to a software CI that does not alter itsdocumented functional or performance capabilities. An example of this is whencode is changed to correct a fault.

SCCB: Software Configuration Control Board. A Software Configuration ControlBoard is a group of individuals who oversee the software change process, withultimate authority for approving a change and promoting a software unit fromone library to another. The individuals may be from the project, relatedorganizations and management levels, the customer, or some combination.During the development process, the SCCB controls promotions into theProject Library from the Personal Library and changes to the products in theProject Library. During the support phase, the project CCB and SCCB providethe authority to make changes to products already promoted to the ReleaseLibrary, and to promote software products from the Project Library to theRelease Library. May also be referred to as a Software Change Control Board.The SCCB and the CCB may be the same in many applications.

SCM: Software Configuration Management.

SCMP: Software Configuration Management Plan.

80


Software Component: A distinct part of a Software Configuration Item (CI). A softwarecomponent may also be called a Computer Software Component (CSC)or Computer Program Component (CPC). CSCs may also be furtherdecomposed into other CSCs or individual units. If a large analysisprogram is called out as a CI, some of the CSCs could be arequirements document or major groupings of the software modules.

Version: A version is a software CI with a defined set of capabilities. A new version is avariation of the previous version, in that it has a change in its functionality orperformance characteristics. An example of this is if the system is changed togenerate a different output.

81


14 TESTING

Tes

ting

- 1

(12

/94)

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56

Tes

ting

82


Tes

ting

- 2

(12

/94)

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56

Ses

sion

Goa

ls

•T

o un

ders

tand

the

pur

pose

of

soft

war

e te

stin

g

•T

o in

trod

uce

test

ing

appr

oach

es a

nd a

ctiv

ities

83


Tes

ting

- 3

(12

/94)

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56

Bas

ic G

oal o

f S

oftw

are

Tes

ting

The

bas

ic g

oal o

f so

ftw

are

test

ing

is t

he e

limin

atio

n of

defe

cts

in t

he s

oftw

are

prod

uct

An

addi

tiona

l ben

efit

is t

hat

we

show

tha

t it

wor

ks in

the

oper

atio

nal e

nvir

onm

ent

84


Tes

ting

- 4

(12

/94)

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56

A T

estin

g P

hilo

soph

y

•T

estin

g is

the

act

of

perf

orm

ing

test

(im

plie

s fo

rmal

ity)

•S

oftw

are

requ

irem

ents

are

ess

entia

l to

test

ing

•T

estin

g is

pre

dom

inat

ely

an e

ffor

t to

sho

w t

hat

the

soft

war

e do

es

not

mee

t its

spe

cific

atio

n

Em

phas

is o

n "N

egat

ive

Tes

ting"

-

But

the

re is

an

addi

tiona

l ben

efit!

•E

ffec

tive

test

ing

dete

cts

def

ects

•D

efec

ts c

an o

ccur

at

man

y st

ages

−R

equi

rem

ents

Spe

cific

atio

n

C

Des

ign

C

Im

plem

enta

tion

C

Inte

grat

ion

C

S

uppo

rt

−M

ust

test

at

each

ste

p (r

evie

ws/

insp

ectio

ns a

re a

lso

test

act

iviti

es)!

85


Tes

ting

- 5

(12

/94)

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56A T

est

Phi

loso

phy

(Con

tinue

d)

•T

estin

g to

det

ect

all

the

defe

cts

is m

uch

like

usin

g "F

inal

Insp

ectio

n" t

o se

para

te g

ood

prod

uct

from

bad

•R

egre

ssio

n te

stin

g sh

ould

be

used

for

cha

nge

pack

ages

and

criti

cal o

r m

ajor

cha

nges

•S

oftw

are

cann

ot b

e co

mpl

etel

y te

sted

—S

elec

t su

bset

of

all p

ossi

ble

test

cas

es t

hat

have

the

hig

hest

pro

babi

lity

of d

etec

ting

the

mos

t de

fect

s

Fun

ctio

nal (

Fea

ture

) -

Str

uctu

ral -

Per

form

ance

and

Str

ess;

com

bine

d

care

fully

to

perf

orm

Rel

iabi

lity

test

ing

Reg

ress

ion

Tes

ting:

Sel

ectiv

e r

etes

ting

of a

sys

tem

or

com

pone

nt to

ver

ify th

at m

odifi

catio

ns h

ave

not

caus

ed u

nint

ende

d ef

fect

s an

d th

at th

e sy

stem

or

com

pone

nt s

till c

ompl

ies

with

its

spec

ified

req

uire

men

ts

86


Tes

ting

- 6

(12

/94)

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56Tes

ting

App

roac

hes

•S

tatic

Tes

ting

−T

estin

g ac

tiviti

es c

ondu

cted

dur

ing

soft

war

e de

velo

pmen

t th

at d

o no

t

invo

lve

the

exec

utio

n of

the

sof

twar

e -

Sta

tic a

naly

zers

, In

spec

tions

•D

ynam

ic T

estin

g

−T

estin

g pr

oces

ses

that

are

per

form

ed o

n ex

ecut

able

or

exec

utin

g

soft

war

e

−"W

hite

Box

" T

estin

g:

Req

uire

s kn

owle

dge

of c

ode

stru

ctur

e -

Impl

emen

ts lo

gic,

sta

tem

ent,

and

pat

h co

vera

ge

−"B

lack

Box

" T

estin

g:

Cle

ar,

com

plet

e sp

ecifi

catio

ns n

eces

sary

-

Kno

wle

dge

of p

rogr

am s

truc

ture

not

nec

essa

ry

−B

oth

requ

ire:

Inge

nuity

- In

tuiti

on -

Exp

erie

nce

Kno

wle

dge

of p

revi

ous

defe

cts

and

wea

knes

ses

87


Tes

ting

- 7

(12

/94)

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56

Dev

elop

ing

/ R

ecor

ding

Tes

ts

•T

est

Pla

n

•T

est

Des

ign

•T

est

Cas

e S

peci

ficat

ion

•T

est

Pro

cedu

re S

peci

ficat

ion

•T

est

Rec

ords

Onl

y 8%

of

SE

MI/

SE

MA

TE

CH

Com

pani

esal

way

s pr

oduc

e a

Tes

t P

lan

-

15%

nev

er d

o

Onl

y 3%

of

SE

MI/

SE

MA

TE

CH

Com

pani

esal

way

s m

aint

ain

a te

st lo

g -

20

% n

ever

do

88


Tes

ting

- 8

(12

/94)

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56Tes

ting

Sum

mar

y

•T

estin

g sh

ould

det

ect

defe

cts;

pro

per

func

tioni

ng is

an

addi

tiona

l ben

efit

•R

equi

rem

ents

are

cri

tical

•A

dequ

ate

test

ing

requ

ires

pla

nnin

g

•T

est

docu

men

tatio

n is

a "

mus

t" -

Reg

ress

ion

test

ing

•Q

uest

ions

you

sho

uld

ask

−D

o w

e ha

ve a

tes

t pl

an?

-

Are

tes

ts d

efin

ed?

−H

ave

we

save

d th

e re

sults

?

−C

an w

e re

run

the

test

s an

d co

mpa

re w

ith p

revi

ous

resu

lts?

89


Tes

ting

- 9

(12

/94)

Tec

hnol

ogy

Tra

nsfe

r #

9211

1389

B-T

RG

/ S

ET

EC

94-0

56

Tes

ting

Sum

mar

yA

ctio

n P

lan

•R

evie

w t

he s

ched

ule

for

test

ing

with

you

r so

ftw

are

man

ager

−A

sk f

or a

n op

timal

est

imat

e to

ade

quat

ely

test

−Is

the

diff

eren

ce t

oo u

ncom

fort

able

?

•A

sk f

or a

n ov

ervi

ew o

f th

e te

st c

over

age

and

the

tren

ds

of t

he r

ate

of f

ailu

res

disc

over

ed a

nd d

efec

ts c

orre

cted

−Is

the

rat

e of

fai

lure

s di

scov

ered

gre

ater

or

less

tha

n th

e ra

te o

f de

fect

s

corr

ecte

d?

−D

oes

this

mak

e yo

u fe

el g

ood

abou

t re

leas

ing

the

prod

uct?

•S

erio

usly

con

side

r th

e ap

prop

riat

enes

s of

the

tar

get

rele

ase

date

giv

en t

hese

tw

o se

ts o

f in

puts

90


15 SUPPORT

Sup

port

- 1

(1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

Sup

port

(Rev

iew

and

Dis

cuss

ion)

91


Sup

port

- 2

(1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

Bui

ldin

g R

elia

bilit

y In

to S

oftw

are

•D

efin

e/U

nder

stan

d th

e P

roce

ss

The

fra

mew

ork

in w

hich

dev

elop

men

t oc

curs

•A

dopt

a D

evel

opm

ent

Met

hodo

logy

−N

ot w

heth

er w

e'll

have

a m

etho

dolo

gy,

but

how

rig

orou

s w

ill it

be

−R

equi

rem

ents

Ana

lysi

s an

d D

esig

n T

echn

ique

s -

"O

bjec

t

Ori

ente

d"/"

Str

uctu

red"

, C

ompu

ter

Aid

ed S

oftw

are

Eng

inee

ring

(C

AS

E)

Too

ls,

Tra

inin

g

−In

vest

men

t in

sof

twar

e en

gine

erin

g m

etho

ds (

Too

ls -

Tra

inin

g) w

ill

prov

ide

high

ret

urn

on in

vest

men

t

•D

evel

op C

lear

, U

nam

bigu

ous

Req

uire

men

ts

Com

mon

fau

lts:

Om

ittin

g re

quir

emen

ts n

eede

d to

sat

isfy

cus

tom

er

need

s; u

ntra

ceab

le r

equi

rem

ents

; co

mpl

ex w

ritin

g st

yles

92


Sup

port

- 3

(1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

Bui

ldin

g R

elia

bilit

y In

to S

oftw

are

Con

tinue

d

•U

se S

oftw

are

Insp

ectio

ns (

For

mal

In-

Pro

cess

Rev

iew

s)−

A "

Val

ue A

dded

" ac

tivity

- R

educ

es t

otal

cos

t (d

evel

opm

ent

and

supp

ort)

, te

st t

ime

(with

out

sacr

ifici

ng c

over

age)

, an

d de

fect

s in

shi

pped

soft

war

e -

Par

t of

the

inve

stm

ent

in s

oftw

are

engi

neer

ing

met

hods

−K

ey in

put

to p

roce

ss im

prov

emen

t ac

tiviti

es

−A

llow

s ch

ecki

ng -

On

Tra

ck?

- O

n S

ched

ule?

•A

pply

Con

figur

atio

n M

anag

emen

t−

Cha

nge

cont

rol:

Aut

omat

e

−P

art

of t

he in

vest

men

t in

sof

twar

e en

gine

erin

g m

etho

ds

•P

lan

for

Tes

ting

−A

ll te

stin

g is

impo

rtan

t:

Fun

ctio

nal (

Fea

ture

), S

truc

tura

l, P

erfo

rman

ce

and

Str

ess:

ê

R

elia

bilit

y .

. .

and

Reg

ress

ion

−Is

the

re s

uffic

ient

tim

e in

the

sch

edul

e?

Or

is it

bei

ng s

quee

zed?

93


Sup

port

- 4

(1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

Bui

ldin

g R

elia

bilit

y In

to S

oftw

are

Con

clud

ed

•M

easu

re t

he P

roce

ss

−P

lann

ed v

s A

ctua

l: S

ize,

Cos

t, P

eopl

e, S

ched

ule

−D

efec

ts r

emov

ed b

y ph

ase

(Req

uire

men

ts,

Des

ign,

Cod

ing)

vs

failu

res

duri

ng t

est

(cau

sed

by d

efec

ts)

vs f

ield

rep

orts

(m

ore

defe

cts)

•P

erfo

rm R

oot-

Cau

se A

naly

sis

−W

hat

type

of

faul

t; w

hen,

why

intr

oduc

ed;

whe

n fo

und;

why

not

dete

cted

; w

hat

coul

d ha

ve b

een

done

to

prev

ent

•F

ollo

w-u

p w

ith A

udits

and

Les

sons

Lea

rned

−R

evie

w e

xist

ing

prac

tices

- h

eade

d in

the

rig

ht d

irec

tion?

−A

naly

sis

of c

usto

mer

fee

dbac

k (n

eed

to a

sk f

or it

!)

−W

hat

wor

ked

and

wha

t di

dn't

94


Sup

port

- 5

(1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

RE

SU

LTS

PR

OB

LEM

AR

EA

SA

DD

RE

SS

ED

MA

TU

RIT

YLE

VE

L

5 -

OP

TIM

IZIN

G[F

eedb

ack]

4 -

MA

NA

GE

D[Q

uant

itativ

e]

3 -

DE

FIN

ED

[Qua

litat

ive]

2 -

RE

PE

AT

AB

LE[In

tuiti

ve]

1 -

INIT

IAL

[Ad

Hoc

]

Acq

uire

d C

apab

ilitie

s -

Ada

ptab

le t

o C

hang

e -

Aut

omat

ion

Pro

cess

Con

trol

- S

tatis

tical

Pro

cess

Con

trol

- C

hang

ing

Tec

hnol

ogy

- P

robl

em P

reve

ntio

n

Pro

cess

Mea

sure

men

t -

Fau

lt P

roje

ctio

n -

Tes

t &

Rev

iew

Cov

erag

e -

Pro

cess

Met

rics

Dat

abas

e

Pro

cess

Def

initi

on -

Qua

lity

Poi

nt R

evie

ws

- S

oftw

are

Eng

inee

ring

Tra

inin

g

Bas

ic M

anag

emen

t C

ontr

ol -

Pro

ject

Pla

nnin

g (C

ost,

Sch

edul

e) -

Con

figur

atio

n M

anag

emen

t -

Reg

ress

ion

Tes

ting

- S

oftw

are

Qua

lity

Ass

uran

ce

Pro

duct

ivity

& Q

ualit

y

Ris

k

Whe

re is

YO

UR

Org

aniz

atio

n?S

oftw

are

Eng

inee

ring

Cap

abili

ty

62

%87

%

23%

9%

15%

4%

Sof

twar

e P

roce

ss M

atur

ity D

istr

ibut

ion

SE

I-as

sist

ed:

13 s

ites,

63

Pro

ject

s

Sel

f-as

sess

men

t: 4

6 si

tes,

233

Pro

ject

s

Dav

id K

itson

& S

teve

Mas

ters

, An

Ana

lysi

s of

SE

I Sof

twar

eP

roce

ss A

sses

smen

t R

esul

ts:

1987

-199

1, C

MU

/SE

I-92

-TR

-24

95


Sup

port

- 6

(1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

You

r R

ole

in S

oftw

are

Dev

elop

men

t

•U

nder

stan

d th

e di

scip

line

and

cont

rols

of

soft

war

e

engi

neer

ing

•A

sk t

he R

IGH

T q

uest

ions

The

last

tw

o:

How

do

we

plan

to

mai

ntai

n th

is s

oftw

are?

(Thi

s qu

estio

n sh

ould

be

aske

d ea

rly in

the

proj

ect!)

How

can

I h

elp?

•E

ncou

rage

−M

otiv

atio

n to

do

it ri

ght

com

es f

rom

the

top

!

−Q

ualit

y im

prov

emen

t st

arts

with

you

96


Sup

port

- 7

(1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

Su

mm

ary

•H

igh

qual

ity,

relia

ble

soft

war

e do

esn'

t ju

st h

appe

n

•H

igh

qual

ity s

oftw

are

requ

ires

suf

ficie

nt f

ront

-end

load

ing—

Tra

inin

g &

Too

ls

•T

wo

high

va

lue-

adde

d a

ctiv

ities

—R

equi

rem

ents

&

Insp

ectio

ns

•A

dher

ence

to

proc

ess

redu

ces

risk

of

poor

sof

twar

e

•Im

prov

ing

the

proc

ess

begi

ns w

ith m

anag

emen

t

97


Sup

port

- 8

(1

2/94

)T

echn

olog

y T

rans

fer

# 92

1113

89B

-TR

G /

SE

TE

C94

-056

Su

mm

ary

Act

ion

Pla

n

Inve

stig

ate

the

mat

urity

of

your

com

pany

's s

oftw

are

deve

lopm

ent

proc

ess

capa

bilit

y

−A

sk y

our

soft

war

e m

anag

er (

lead

eng

inee

r) t

o in

form

ally

app

ly t

he S

EI

Leve

l Tw

o S

oftw

are

Mat

urity

Sur

vey

ques

tions

to

your

sof

twar

e

deve

lopm

ent

proc

ess

and

repo

rt o

n th

e re

sults

−S

tres

s be

ing

fran

k an

d op

en

−T

oget

her,

tar

get

one

or t

wo

area

s th

at w

ill im

prov

e yo

ur c

ompa

ny's

abili

ty t

o m

eet

leve

l tw

o at

trib

utes

−R

e-as

sess

thi

s ar

ea in

4 t

o 6

mon

ths

−T

arge

t on

e or

tw

o m

ore

. .

. .

. .

98


16 QUESTIONS TO ASK

1. Have we really defined OUR process?

2. What is our policy for software quality?

3. Do we have standards for software development? - What are they?

4. Do we have a Software Development Plan? - Is it current?

5. Do we have a cost and schedule estimate?

6. How are we tracking to the development schedule?

7. Do we know what the requirements are?

8. What analysis techniques are we using to define/refine requirements?

9. Are the requirements documented?

10. Who is our customer? - Who is the user?

11. Have we drafted the System Test Plan?

12. What reviews are planned? - Which ones should I attend?

13. How have we insured the requirements specification (design) is correct (complete)?

14. How much of the requirements specification (design, code, test plan) has been (will be)inspected?

15. May I see the Inspection Management Report?

16. Do we have a Software Configuration Management Plan?

17. Can we rebuild the current release (baseline)?

18. Can we rebuild the previous release (baseline)?

19. Do we have a test plan? - Are tests defined?

20. Have we saved the results?

21. Can we rerun the tests and compare with previous results?

22. How do we plan to maintain this software?

23. HOW CAN I HELP?

99


17 ACTION PLAN FOR SOFTWARE QUALITY

1. Make an appointment with your software manager to sketch out the structure ofYOUR software development process

Use blank Embedded Life Cycle Model as basis or develop simpler model

Compare to model in this course

Ask: Are there elements of the course model that we want to implement? If so: When canwe begin to implement them? What can we do in the next 6 months? In the next year?

2a. Write out a policy statement defining YOUR company's concept of quality forsoftware

Sample Policy Statement

“Quality software shall be an objective of this company's quality program. This objectiveapplies to the acquisition, development, use, and support of software. The program shalladdress, as a minimum:

Organization, tasks, and responsibilities; software documentation; configurationmanagement; testing, and; reviews and audits.”

2b. Make an appointment with your software manger to outline software developmentgoals for your next new feature (or next generation of equipment)

Integrate Process Model from previous action item - Technical risks - Analysis andDesign methods - Initiate formal schedule and budget estimation, define work packages

Use as basis for a more formal Software Development Plan

3a. Meet with (interview) customer production managers

Ask: What are their top five requirements for your company's equipment.

3b. Meet with your system/project and software managers

Ask: How are we meeting these requirements?

4a. Determine what kind of reviews are currently being conducted to control softwarequality

Discuss with software manager - This can be done this afternoon!

4b. Initiate an evaluation of software inspections

• Have software manager schedule software developers (and manager) for training insoftware inspections

Classes taught regionally (east and west coast) and in-house.

• Apply inspection to a current work product

Two immediate benefits: Improved work product - Insight into necessaryimprovements in development activities leading up to current work product

100


5. Ask for an overview of how problem reporting, and requests for new features, arereceived, reviewed, and software changes incorporated in your company.

6a. Review the schedule for testing with your software manager

Ask for an optimal estimate to adequately test

Is the difference too uncomfortable?

6b. Ask for an overview of the test coverage and the trends of the rate of failuresdiscovered and defects corrected

Is the rate of failures discovered greater or less than the rate of defects corrected?

Does this make you feel good about releasing the product?

6c. Seriously consider the appropriateness of the target release date given these two setsof inputs

7. Investigate the maturity of your company's software development process capability

Ask your software manager (lead engineer) to informally apply the SEI Survey Level Twoquestions to your software development process and report on the results

Stress being frank and open

Target one or two areas to improve your company's ability to meet level 2 attributes

Re-assess this area in 4 to 6 months . . . . . Target one or two more

101


Adapted from "A Spiral Model of Software Development and Enhancement", B. W. Boehm, IEEE Computer, May1988.

Evaluate Alternatives,Identify & Resolve Risks

Develop, VerifyNext-Level Products

Determine Objectives,Alternatives, Constraints

Plan Next Phases

PrototypeBetaSystem

ProductionSystem

18 SEI MATURITY MODEL: LEVEL 2 – REPEATABLE PROCESS

At Maturity Level 2, the organization uses standard methods and practices for managing softwaredevelopment activities such as cost estimating, scheduling, requirements changes, code changes,and status reviews. An organization qualifies at Level 2 if the percentage of affirmative answersis at least 80% and the percentage of affirmative answers to shaded questions is at least 90%.(The answers to these questions should reflect standard organizational practices)

SEI defines mechanism and formal procedure as follows:

Mechanism—a means whereby the performance of a task, procedure, or process is assured.

Formal Procedure—a documented series of steps with guidelines for use.

102


Ques.No. SEI Question – Level 2

Proj1

Proj2

Proj3

Notes(Follow-up Ques)

1.1.1 For each project involving software development, isthere a designated software manager?

1

1.1.2 Does the project software manager report directly to theproject (or project development) manager?

1

1.1.3 Does the Software Qualtiy Assurance (SQA) functionhave a management reporting channel separate from thesoftware development project management?

1

1.1.6 Is there a software configuration control function foreach project that involves software development?

2

1.2.2 Is there a required training program for all newlyappointed development managers designed tofamiliarize them with software project management?

3

1.3.1 Is a mechanism used for maintaining awareness of thestate-of-the-art in software engineering technology?

4

2.1.3 Is a formal procedure used in the management reviewof each software development prior to makingcontractual commitments?

4

2.1.4 Is a formal procedure used to assure periodicmanagement review of the status of each softwaredevelopment project?

4

2.1.5 Is there a mechanism for assuring that softwaresubcontractors, if any, follow a disciplined softwaredevelopment process?

4

2.1.7 For each project, are independent audits conducted foreach step of the software development process?

8

2.1.9 Are coding standards applied to each softwaredevelopment project?

4

2.1.14 Is a formal procedure used to make estimates ofsoftware size?

4

2.1.15 Is a formal procedure used to produce softwaredevelopment schedules?

4

2.1.16 Are formal procedures applied to estimating softwaredevelopment cost?

4

2.1.17 Is a mechanism used for ensuring that the softwaredesign teams understand each software requirement?

4

2.2.1 Are software staffing profiles maintained of actualstaffing versus planned staffing?

5

103


Ques.No. SEI Question – Level 2

Proj1

Proj2

Proj3

Notes(Follow-up Ques)

2.2.2 Are profiles of software size maintained for eachsoftware configuration item, over time?

5

2.2.4 Are statistics on software code and test errorsgathered?

5

2.2.7 Are profiles maintained of actual versus plannedsoftware units designed, over time?

5

2.2.8 Are profiles maintained of actual versus plannedsoftware units completing unit testing, over time?

5

2.2.9 Are profiles maintained of actual versus plannedsoftware units integrated, over time?

5

2.2.10 Are target computer memory utilization estimates andactuals tracked?

5

2.2.11 Are target computer throughput utilization estimatesand actuals tracked?

5

2.2.12 Is target computer I/O channel utilization tracked? 5

2.2.16 Are software trouble reports resulting from testingtracked to closure?

5

2.2.18 Is test progress tracked by deliverable softwarecomponent and compared to the plan?

5

2.2.19 Are profiles maintained of software build/releasecontent versus time?

5

2.4.1 Does senior management have a mechanism for theregular review of the status of software developmentprojects?

4

2.4.5 Is a mechanism used for regular technical interchangeswith the customer?

4

2.4.7 Do software development first-line managers sign offon their schedules and cost estimates?

8

2.4.9 Is a mechanism used for controlling changes to thesoftware requirements?

4

2.4.17 Is a mechanism used for controlling changes to thecode? (Who can make changes and under whichcircumstances?)

4

2.4.20 Is there a mechanism for assuring that regressiontesting is routinely performed?

4

104


19 FOLLOW-UP QUESTIONS

When performing a self assessment, use the requested information inthese follow-up questions as benchmarks to determine if your

organization/project should answer Yes to the original question.

1. Where responsibility assignments are questioned, request the name of a specific individual,tenure in job, job description, and evidence of activity, such as monthly reports, meetingreports, control logs.

2. Where the existence of a group is questioned, request names of members, the organizationrepresented, and recent meeting agendas and minutes.

3. Where the existence of education or training programs is questioned, request the schedule ofrecent courses offered, course outlines, names of attendees, and qualifications of instructorsand students.

4. Where the existence of a mechanism, procedure, standard, criteria, or guideline is questioned,request a copy of the controlling document, its revision history, the name of the individual(s)responsible for tracking, job description(s), and recent issue/activity reports.

5. Where the use of profiles, tracking reports, planned vs. actual comparisons, andmeasurements are questioned, request the three most recent reports, measurement summaries,or comparisons.

6. Where computations or analysis of data is questioned, request copies of the most recentcomputations, analysis reports, or summaries showing results or conclusions reached.

7. Where the initiation of actions are questioned, request copies of recent action tracking and/orsummary reports.

8. Where the conduct of certain actions or use of facilities is questioned, request evidence in theform of procedures, responsibilities, or tracking systems to demonstrate performance.

9. Where the existence of a facility, capability, practice, or method is questioned, requestsupporting evidence in the form of inventory lists, tracking and usage reports, instructionmanuals, education programs, etc.

10. Where the use of an automated tool or facility is questioned, request a demonstration of thattool or facility.

105


20 SOME FURTHER READING

Articles Specific to Sessions

Introduction

Humphrey, Watts S.

"Characterizing the Software Process: A Maturity Framework," IEEE Software,March 1988, pp 73-79.


Basili, Victor R., and Musa, John D.

"The Future Engineering of Software: A Management Perspective," IEEE Computer,September 1991, pp 90-96.


Boehm, Barry W.

"Software Risk Management: Principles and Practices," IEEE Software, January 1991,pp 32-41.

Reviews and Inspections

Fagan, Michael E.

"Advances in Software Inspections," IEEE Transactions on Software Engineering,Vol. SE-12 No. 7, July 1986, pp 744-751.

On Software Project Management

Brooks, Fred

The Mythical Man-Month - Essays on Software Engineering, Addison-Wesley, 1975.

DeMarco, Tom

Peopleware, Yourdon Press, 1987.

Glib, Tom

Principles of Software Engineering Management, Addison-Wesley, 1988.

Utz, Walter J., Jr.

Software Technology Transitions, Prentice-Hall, 1992.

On Software Metrics

Boehm, Barry

Software Engineering Economics, Prentice-Hall, 1981.

DeMarco, Tom

106


Controlling Software Projects: Management, Measurement, and Estimation, YourdonPress, 1982

Fenton, N. E.

Software Metrics: A Rigorous Approach, Van Nostrand Reinhold, 1991.

Grady, R. and D. Caswell

Software Metrics: Establishing a Company-Wide Program, Prentice-Hall, 1987.

Jones, Capers

Programming Productivity, McGraw-Hill, 1986.

Putnam, L. H., and Myers, W.

Measures for Excellence: Reliable Software on Time, within Budget, Yourdon Press,1992.

SEMATECH Technology Transfer2706 Montopolis Drive

Austin, TX 78741

http://www.sematech.org

software quality engineering for managers training · pdf fileon process organization,...

Documents