Cleanroom Method

CS 415, Software Engineering II

Mark Ardis, Rose-Hulman Institute

March 20, 2003

2

Outline

1. Harlan Mills2. Cleanroom method3. Industrial use of cleanroom

3

Harlan Mills

1919 - 1996

4

Mathematics and Programming

Roman accounting"to go from programming as an instinctive, intuitive process to a more systematic, constructive process"

5

Cleanroom Method

Incremental (spiral)

Box structure specification and design

Design verification

No debugging

Statistical testing

6

Box Structures

Black boxes: behavior only

State Boxes: behavior + state

Clear boxes: procedures

7

Black Boxes

S1S2...Sn R

stimulus history response

8

State Boxes

S R

stimulus, old state response, new state

State Data

9

Clear Boxes

S R

stimulus, old state response, new state

State Data

Procedures

10

Box Description Language (BDL)

Invocation: use <type> <name> <args>

Sequence: do B1; B2 od

Alternation: if <cond> then B1 else B2 fi

Iteration: while <cond> do B od

11

Box Structure Hierarchy

BB

SB

CB

BB BB BB

SB SB SB

CB CB CB

12

Cartoon of the Day (1/3)

13

Cartoon of the Day (2/3)

14

Cartoon of the Day (3/3)

15

Design Verification

Procedures in BDL are checked for correctness with their higher-level descriptionsAll boxes (and all procedures) describe functionsFormal proofs of correctness can be performed (but often informal proofs are done, instead)

16

Verification of Sequence

Given a high-level function [f] for statement: do [g]; [h] od

Does [g] followed by [h] compute the same function as [f]?

Example:[f](x) = 2 * x + 7[g](x) = 2 * x[h](x) = x + 7

17

Verification of Selection

Given a high-level function [f] for statement:

if <cond> then[g]else[h]fi

1. Whenever <cond> is true, does [g] compute the same function as [f]?

2. Whenever <cond> is false, does [h] compute the same function as [f]?

18

Verification of Iteration

Given a high-level function [f] for statement:

while <cond> do[g]od

1. Whenever <cond> is true, does [g] followed by [f] compute the same function as [f]?

2. Does the loop always terminate?3. Whenever <cond> is false, does the empty

function compute the same function as [f]?

19

Usage Testing

Develop an operational profile of use

Generate random tests that fit the probabilities

20

Example

FunctionUsage Probability

Distribution Interval

Update 32% 0-31

Delete 14% 32-45

Query 46% 46-91

Print 8% 92-99

21

Test Generation

TestRandom Numbers Test Cases

1 29, 11, 47, 52, 26, 94

U, U, Q,Q, U, P

2 62, 98, 39, 78, 82, 65

Q, P, D,Q, Q, Q

3 83, 32, 58, 41, 36, 17

Q, D, Q,D, D, U

4 36, 49, 96, 82, 20, 77

D, Q, P,Q, U, Q

22

Industrial Use

Used in a few areas of IBM

Used by some military contractors

Tried at NASA

23

Software Engineering Laboratory (SEL)

Joint program of NASA Goddard Space Center, Computer Sciences Corporation, and the University of Maryland

Conduct experiments and case studies on new software technology

24

SEL Experience

First trial at University of Maryland controlled experiment (10 experiment teams,

5 control teams FORTRAN 1.5 KLOC

3 case studies at Goddard flight-dynamics ground support systems FORTRAN 40 KLOC, 22 KLOC, 160 KLOC

25

SEL Results – University Experiment

Cleanroom teams use fewer computer resources satisfy requirements more successfully make higher percentage of scheduled

deliveries

26

SEL Results – Goddard

More effort spent in design

Better reliability of final product

Smaller projects achieve higher productivity, but large project just average

27

Summary

Cleanroom may be an effective method for achieving higher reliability

Requires some culture change (no debugging)

Still being investigated by researchers and practitioners

28

References

Victor Basili and Scott Green, "Software process evolution at the SEL", IEEE Software 11(4), 58-66, July 1994.