myths and realities in software development - the david r

Myths and Realities inSoftware Development

Daniel M. Berry

with material from F.P. Brooks’s Mythical ManMonth

1997 Daniel M. Berry Software Enginering Myths & Realities Pg. 1

I was going to say:

“Myths and Truths in Software Development”

But that would be putting too much strengthto what I say.

For today’s realities may turn out to betomorrow’s myths!


Truths?

“There are no universal truths except, ofcourse, this one”

— David Thewlis


Predictions about Computers

“Computers in the future may weigh no morethan 1.5 tons.”

— Popular Mechanics, 1949

“I think there is a world market for maybe fivecomputers.”

— Thomas Watson, Chair IBM, 1943.


“I have travelled the length and breadth of thiscountry and talked with the best people, and Ican assure you that data processings is a fadthat won’t last out the year.”

— Editor of business booksfor Prentice-Hall, 1957.

“But what ... is it good for?”— Engineer at the AdvancedComputing Systems Division,IBM, 1968, commenting on themicrochip.


“There is no reason anyone would want acomputer in their [sic] home.”

— Ken Olson, president,Chair and founder, DEC.


Outline

Myths concerning a variety of topics:Management IssuesLifecycle ModelsLifecycle Steps

ConceptionRequirementsDesignProgramming


TestingMaintenanceDocumentation

Theory

And then, some truths!


Management Issues

Person MonthDivisibility of TasksHuman CapitalTeam SizesIndividual DifferencesSchedulingTools and Methods

CASE ToolsNo Silver BulletRationality of MethodsFaking It


Project Success/FailureTechnical FactorsPeople FactorsSociological Factors

Project KillersLimitationsCMMProcessesRisk Management


Myth:

Person Month:

a famous unit of measure of work (used to becalled “man month”)

The name of the unit is itself the myth,because it implies the following graph:


Time

Pers

ons


No!

Prime counter-example:

If it takes one couple 9 months to make ababy, then in how many months can 3 couplesmake a baby?

Nu?!?!


Three kinds of tasks

g completely dividableg partially dividableg not dividable


Completely Dividable Tasks

Laying a brick wall is completely dividableafter bottom row has been laid.

You can throw as many brick layers as youwant to the job.

Each works on an independent part.


The shape of the bricks and the laying patternguarantee that the independent sections willinterface properly when they meet.

There is no need for interface discussions.

In essence, the bottom row is a completeinterface specification for all independentparts, no matter how many there are.


For a job requiring P PMs, N people reducesthe elapsed time of the job to close to P/Nmonths.


Not Dividable Tasks

Some tasks take a certain minimum amount oftime, no matter how many people you throw atthe task.

Examples,

Making babyBaking a cakeGrowing a treeLearning a domain


A job requiring T months still requires (atleast) T months even when you have N peopletrying to help.


Partially Dividable Tasks

Most software engineering tasks are onlypartially dividable, because they requirecommunication among the people over whomthe tasks are distributed,

especially when interfaces must be workedout between different people’s work or wheneverybody’s viewpoint must be understoodbefore proceeding with individual work.


In any such situation, the problem is theamount of communication needed.

number of persons, lines of communication

5,104,63,32,11,0


The number of lines of communication growsas the square of the number of people.

“Too many cooks spoil the pie.”


Sobering Reality:

“Adding more people to a late project makes iteven later!”

says Fred Brooks, leader of the IBM project tobuild OS/360.

The time spent catching new people up and innew lines of communication is much greaterthan the time the new people can add to thework.


So, for a job requiring P PMs, N peoplerequire more than P/N months, how muchmore depends on the communication needs

and on N 2.


Human Capital, Unmasked

Tom DeMarco [in NYT 14 Apr. ’96] tells thefollowing story:

Imagine, you’re the manager of crack, highlyeffective, highly motivated, well-knit 5-personteam that is humming smoothly on scheduleon this important, make-or-break project thatsimply must be done on time.


You learn, to your dismay, that one of the 5,Louise will be leaving at the end of the month.

Wotta disaster... you know you’re in deeps__t.

So you ask personnel to send you anotherLouise.

Personnel tells you that Louises are, sadly,out of stock, and offers you a Ralph, who is noslouch, with as much experience and skill asLouise.


So the deal is done; Louise is out on the 31stand Ralph is in on the 1st;

From accounting’s point of view, nothing haschanged; Ralph’s salary, workload, expertise,etc is the same as Louise’s; so you’re tradinglike for like, with no net change! What’s theproblem?


g Louise spends a majority of her remainingtime writing up what Ralph is supposed toknow, contributing less to the project inher last days

or

g Ralph starts earlier over accounting’s loudcomplaints and Louise shows Ralph theropes, contributing even less to the projectwhile Ralph adds more costs to the project.


In either case, on 1st, Ralph comes on the jobalone, spends first day figuring out who iswho, finding the toilets, the coffee room, andsupplies, and reconfiguring the workstationleft by Louise.

His contribution to the project? Zilch, Nada!

The second and subsequent days he startsporing over Louise’s notes.

He still adds nothing to the project.


Every now and then, he has a question andgoes to ask other team members, who findthemselves interrupted from doing their 25%greater workload (since they have to take upslack left by Louise and not yet picked up byRalph). Ralph’s contribution? not even 0, it isnegative; he keeps others from working totheir capacity.

Ralph continues to be a negative for quite awhile and slowly begins to get up to speed.

The graph below shows the situation.


Cost of Getting Up to Speed

LOSTPRODUCTION

writing forRalph

RALPH

TIME-+

PR

OD

UC

TIV

ITY

LOUISE

Ralph starts early


In a typical large project, it can take up to twoyears to get up to speed. In this case, the lostproduction (integral above the curve) is abouta person year of work.

That is, each time a new engineer is hired, afull year’s salary has to be invested in thatengineer before the engineer begins to pay off.

Perhaps people should be recognized as aninvestment and not an expense.


Myth:

Big teams are better!

We have a staff of thousands working on yourprogram!!


Reality:

Big teams are great for 43-person Squamish,symphonies, epic movies, etc., but not formaking pies and software!

“Too many cooks spoil the pie.”

Recall that the number of lines ofcommunication grows as the square of theteam size.


Speaking of Fred Brooks and OS/360, folklorehas it that OS/360 was built in the IBM Army ofAnts approach by a team of consisting ofBrooks and 1000 nameless people, one ofwhom is buried in the tomb of the unknownprogrammer at the IBM corporate cemetery inPoughkeepsie, New York.

The first UNIX system was built by a team ofthree people, Dennis Ritchie, Ken Thompson,and Brian Kernighan, and none was the leaderper se.


g Which operating system is in voluntary useall over the world?

g Which operating system is used as a basisfor many others?

g Which operating system is dissected as anobject of study in text books and coursesabout operating systems?

g Which operating system was started first?Which arrived first at a relatively stablestate in which new releases were forenhancements rather than bug fixes?

Need I say any more?


Various studies indicate that the optimal teamsize is between 2 and 5, with 3 being themode.

Well, certainly a team with fewer than 2members is not a team!

With more than 5 team members, teammanagement begins to dominate the work;i.e., each additional person costs more in teammanagement time than he or she adds topotential work time.


Myth:

All programmers are the same.

All experienced programmers are equallyskilled.

This job requires 5 person years; I’ve got oneyear to do it; so give me 5 programmers.


Reality:

Sackman, Erickson, and Grant’s 1965experiment to show that interactiveprogramming was more effective than batchprogramming failed to produce significantresults because the effect of the independentvariable (use of interactive vs. batchsubmission of the job) was drowned out byindividual differences in programmers ofequal experience.


One experienced programmer was found to be28 times more effective than another equallyexperienced programmer!

g If you have 5 of the first kind, you’ll finish.g If you have 5 of the second kind, you won’t!g In fact, you might even be able to finish in

time with only one of the first kind; if youcan arrange the teams so that he or shewill not be slowed down by having tocommunicate with other team members.


The idea of Chief Programmer Teams is tobuild a small team around one of these superprogrammers

g to allow the super programmer to do his orher stuff with a minimum of distraction bydrudgery, which is done by the other teammembers, and

g to limit communication to a starconfiguration with the chief in the center,for which the growth in number of lines ofcommunication is linear in the growth ofteam size.


Myth:

The program is 95% done!


Reality:

Actual data from Jim Tomayko:

0102030405060708090

100

0 25 50 75 100

Percentage of Total Project Time

Est

imat

ed P

erce

nt

Co

mp

lete

10

30

50

75

9598

95

9899

100

9890


Do you believe “95% done” any more?

Programmers are among the most optimisticpeople in the world.

They continue to believe in their ability tosolve problems instantly even in the face ofcontinued, repeated evidence to the contrary.

Each is even more optimistic abouthim/herself than anyone else.


Myth:

CASE tools will solve all your problems

Just look at all the advertisements in the trademagazines promising 1000% improvement insoftware productivity if you buy the advertisedCASE tools!


Reality:

Fred Brooks says:

“There’s no silver bullet!”

g Essenceg Accidents


“No Silver Bullet” (NSB)

g The essence of building software isdevising the conceptual construct itself.

g This is very hard.

- arbitrary complexity- conformity to given world- changes and changeability- invisibility


g Most productivity gain came from fixingaccidents

- really awkward assembly language- severe time and space constraints- long batch turnaround time- clerical tasks for which tools are helpful


g However, the essence has resisted attack!

We have the same sense of beingoverwhelmed by the immensity of theproblem and the seemingly endlessdetails to take care of,

and we produce the same brain-damaged software that makes the samestupid mistakes

as 30 years ago!


More Reality:

Even the best tool will not make a goodprogrammer out of a bad one.

In fact, a bad programmer will use good toolsto turn out worse programs more quickly thanever before.


The same can be said for softwaredevelopment methods:

Here is an example of a so-called structured,goto-less program.

for i from 1 to 4 docase i in

1: s1,2: s2,3: s3,4: s4

esacod


This is, of course, equivalent to

s1; s2; s3; s4

The so-called structured program is prettydisgusting if you ask me.


A good tool is one that automates clericalportions of tasks that you, a goodprogrammer, do in the course of goodprogramming.

It helps avoid stupid errors.

A stupid error is an algorithmically avoidableerror!

Mainly, you are stupid if you let an error that aprogram can detect go undetected!


Example of good CASE tool: Stu Feldman’s

make

relieves programmers of having to keep trackof which modules depend on what others andof which ones have been updated since lastcompiling and linking the full program so thatno more modules than need to be compiledagain are compiled again.


A bunch of myths:

Programmers would like to think ofthemselves as rational.

Methodologists would like to believe that allprogrammers can be taught to be rational.

All would like to believe that rationalprogrammers write good software!


Methodologists write papers and booksdescribing how to use their methods to writecode rationally.

All of these papers and books have examplesof nice, clear step-by-step rationaldevelopments of code from requirements.


Reality

Funny thing is that these authors probablyrevised their examples as much of the rest ofus!

I know; I have written such a monograph.

The same applies to lecturers on softwareengineering methods.


Myth:

Methodologists would have you believe thatgood programmers actually follow somevariation of the waterfall lifecycle or somesuch.


Closer to Reality:

Realization

Operation

Integration

Design

Specifications

Requirements


Real Reality:

The hurricane model


In both you get wet, but a hurricane is muchwetter and messier.

But in the eye of the hurricane, there is a falsesense of calm.


Not Rational

OK, OK, so process is not rational. Nu?

But, there is value to describing thedevelopment of software as if it were rational,i.e., of faking a rational process.

“I know that I’ve been fakin’ it!”— Paul Simon


Faking It

David Parnas and Paul Clements suggestwriting the documentation as if thedevelopment were rational.

Be prepared to modify it, when thedevelopment changes direction as thedevelopers get too wet in the storm.


Myths:

The most important factors determining thesuccess of a software development project areits

1. programming language and2. tools.


Reality

All wrong, despite what language and tooldesigners would have you believe.

For one thing, the real influence of the listeditems is in reverse order.

1. tools and2. programming language.

A good tool can make even assemblylanguage appear object oriented!


So then what are the important influences?

Certainly, more important than these is thecompetence of the team members.

Recall the discussion on individualdifferences earlier and how they completelywashed out the technological difference in theprogramming environment.


A good programmer can write good code inany language, including assembly or Pascaland can fake the effect of any tool with a fewgood tricks and a flexible programmingenvironment!


However, the most important factordetermining a project’s success or failure issomething else entirely.

Tom DeMarco says,

“The very best technology never has as muchimpact as girlfriend or boyfriend trouble.”

“... the project’s sociology will be moreimportant to eventual success and failure thanthe project’s technology.”


Bill Curtis considers

“techies [themselves to be the key] non-technological factors in softwareengineering,” even more important thantechnological factors.

In other words, the sociology and politics ofthe team will make or break the team.


Curtis, Krasner, and Isco found that:

“Software development tools and practiceshad disappointingly small effects in earlierstudies, probably because they did notimprove the most troublesome processes insoftware development.”

“Processes” refers to the management of thesteps and procedures of the development.


Humphrey, Kitson, and Kasse report that:

“For low maturity organizations, technicalissues almost never appear at the top of keypriority issue lists, ... not because technicalissues are not important but simply becauseso many management problems must behandled first.”


Fred Brooks says it too

“People are Everything”

g The issues are managerial, not technical.g Every study shows the crucial importance

of people.g Projects don’t move; only goals move!


Also, Brooks, in saying that there is no silverbullet, concluded:

“The central question in how to improve thesoftware art centers, as it always has, onpeople.”


Barry Boehm, who has written the book onsoftware cost estimation says:

“Personnel attributes and human relationsactivities provides by far the largest source ofopportunity for improving softwareproductivity.”


Boehm’s determination of relative contributionof various factors:

1.20

1.231.321.341.491.491.511.561.571.661.872.364.18

1.23Language ExperienceSchedule ConstraintDatabase SizeTurnaround Time

Software ToolsVirtual Machine Volatility

Modern Programming PracticesStorage ConstraintApplication Experience

Product ComplexityPersonnel/Team Capability

Virtual Machine Experience

Timing ConstraintRequired Reliability

So

ftw

are

Co

st D

rive

r A

ttri

bu

te

0 1 2 3 4

Relative Effect


Watts Humphrey, who wrote the book onsoftware processes says:

While technology offers considerable potentialfor improvement, in many organizations thesoftware process is sufficiently confused andincoherent that non-technological factorsimpede the effective application oftechnology.


In other words, ...

A team of highly competent programmers whoare also highly territorial, egotisticalpoliticians will fail while a team of equallycompetent programmers, who are alsoegoless, cooperative, team players willsucceed.


Myth:

Technology is the most important factor inproject success (or failure).


Reality:

Joseph Blackburn, Gary Scudder, and Luk VanWassenhove’s survey study of improvingspeed and productivity of softwaredevelopment shows that having talentedpeople is far more important than having goodtechnology.


Given a choice between investing in talented,expensive people and good, expensive tools,go for the talented people even though theyare more expensive than the expensivetechnology.

They say that there is a silver bullet, thecreative, talented, super programmer.

“Faster than a speeding silver bullet! Look upin the sky! Is it a bird? Is it a plane? No, it’ssuper programmer!”


Project Killers

Tom DeMarco, in his ICRE ’96 Keynote, listsbut a few:

g The user hates me.g One of the stakeholders is willing to come

to the table only if a key precondition ismet.

g The 3 principal stakeholders are outragedthat a 4th stakeholder has been identifiedand invited to participate


g One of the stakeholders has publicallydeclared his distrust of one of the others

g One of the key participants stands to besubstantially disenfranchised byinstallation of the new system.

g People don’t feel safe here.


Killer Neutralizing Skills

Tom DeMarco lists only a few:

g interdependent decision makingg conflict resolutiong negotiationg ability to apply Win-Win


Win-Win (Theory W)

Boehm: The Win-Win approach involves thefollowing basic steps performed bystakeholders and an architect-facilitator:

1. Identify stakeholders’ win conditions.2. Identify issues involving win condition

conflicts.3. Formulate and evaluate options addressing

the issues.4. Formulate, vote on, and adopt agreements

on mutually satisfactory options.


Among these skills, there is nothing verytechnical (except knowledge about options)!


Myth:

Next year, the machine will be big enough andfast enough that we won’t have theselimitations.


Reality:

Our ambitions, fueled by a realization ofincreased computing power (speed, space,and bandwidth) always exceeds the limits, notonly of the computing power, but of ourmental capability to deal with them as routine.


Barry Boehm said back in 1984, “There isnever enough time or money to cover all thegood features we would like to put into oursoftware products. And even in these days ofcheap hardware and virtual memory, our moresignificant software products must alwaysoperate within a world of limited computerpower and main memory.”

He could have said it this year too!


Myth:

We have a 4 rating in the CMM, so you knowthat we are a good software developmentcompany.

CMM = Capability Maturity Model [Paulk et al]

A model and a measure of the maturity of asoftware development organization’s softwaredevelopment process.

Scale of 1 through 5 with 5 being best.


iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii5 Technology Innovation

Optimizing Continuous Improvementiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii4 Measurement of Process

Managed Process & Product Qualityiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii3 Continual Reviews

Defined Engineering Processiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii2 Planned Lifecycle

Repeatable Project Managementiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii1 Chaos

Initial Heroesiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiicccccccccccccc

cccccccccccccc

cccccccccccccc


iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii5 Controlled

Optimizing Processiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii4 Measured

Managed Processiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii3 Defined

Defined Processiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii2 Basically Managed

Repeatable Processiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii1 Ad hoc

Initial Processiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiicccccccccccccc

cccccccccccccc

cccccccccccccc


Predictions According to CMM

For 200K-line business data processingproduct:

CMM Durat’n Effort Faults Faults $ CostLevel Months PMs Detect. Deliver. Devel.iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii

1 29.8 593.5 1348 61 5.4M2 18.5 143.0 328 12 1.3M3 15.2 79.2 182 7 .73M4 12.5 42.8 97 5 .39M5 9.0 16.0 37 1 .15M


But ...

That is only a prediction.

To date, to my knowledge, there is noexperimental verification of the prediction.

However, for better or worse, a lot of peoplebelieve in the prediction.


Reality

CMM is a good model in that if an organizationis good then it will score high.

The problem is that the converse, “if anorganization scores high then it is good”,does not necessarily hold!

Unfortunately, the DoD and some MoDs areusing CMM rating as a rating of goodness ofpotential contractors.


Analogy:

Many parents say:

“If you have a fever, you may stay home fromschool.If you don’t have a fever, you may go on thecamping trip.”

While sickness does normally lead to a highfever, fever and sickness are not logicallyequivalent.

Many children learn to manipulate bodytemperature to achieve nefarious goals.


Paradoxes about Processes

Tom DeMarco in his ICSE 18 (’96) Keynotesaid that our efforts to standardize andimprove process have had some positiveeffect, ...

but not much!

In particular:

The organizations that have invested mostheavily in methodology and process havenot been the major beneficiaries.


Paradox 1

Every time you improve process, workbecomes harder.

This is a corollary of Brook’s NSB thesis thatthere is an irreducible core of work, notsubject to improvement and not mechanizable.

So process does not help with these and justadds to the work.


We end up focusing more and more on theirreducible core as more and more of the restgets automated, and what’s left is thus moreand more uniformly hard.


Paradox 2

Focus on process tends to make anorganization risk averse.

And who are the big winners?

The risk takers or the safe ones?


It’s the old

Armor vs. Mobility

argument for the military.

Armor = process and there is no room formobility.


I am told that the process at Microsoft stinks,but they have the money to spend on beingmobile and to hell with the risks! They canafford to lose a million here, a million there onbad projects.


Paradox 3

The problems of software re-use have beenutterly intractable, but also our greatestsuccess.

Just look at the great success of programminglibraries!

But who can predict at planning time, whichlibraries will sell?


Paradox 4

We adapt to fast change but not to slowchange.

So, we adapted well to microcomputers, 4thgeneration languages, WWW, etc., but not wellto the progressive dinosaurization of dataprocessing.


Paradox 5

Riskier projects are safer, in that you end upkeeping your job!

Because the results will have a much highervalue.

The highest risk projects have the biggestpayoff.

And if it fails, so what? You can always find agood job!


Myth:

The new Denver International Airport (the onewith the automated baggage system that wasmore than a year late) is a total failure!

Stay away from DIA!

(But that makes it difficult to get to goodskiing in Colorado)


Reality

DeMarco, in his ICRE ’96 Keynote, observedthat the failure at DIA was a lack of riskmanagement.

They did not defend against the risk that thesoftware might be late.

The software was on the critical path foropening and they simply did not provide anyother way to open without the software.


So they were a year late.

But for all the delay, for all the money lost, twoyears after opening, it has already recoveredthe losses and is making a profit.

And it’s not a bad airport (even though I didbreak my leg in the skiing I did after arriving atDIA).


Lifecycle Models

WaterfallProblems with WaterfallWalkerfallSpiral


Waterfall Model:

Realization

Operation

Integration

Design

Specifications

Requirements

Win Royce’s Waterfall Model


Brooks about Waterfall

In ICSE ’95 Keynote

Brooks says “The Waterfall Model is Wrong!”

g The hardest part of design is deciding whatto design.

g Good design takes upstream jumping atevery cascade, sometimes back more thanone step.

g Even the U.S. DoD finally knows this, to witDefense Science Board Study, KaminskiCommittee, June 1994.


Problems with Waterfall Model

The main overall problem is that it does notwork!

No one writes software that way; no one isable to write software that way!

People make too many mistakes along theway and recognize that they have done so.

So we have to fake having followed thewaterfall with the documentation.


The main problem, from the requirementspoint of view, of the waterfall model is thefeeling it conveys of the sanctity, inviolability,and unchangeability of the requirements.

Barry Boehm produced the following pictureat a 1988 workshop at SEI.


Michael Jackson Says

In the Requirements Engineering ’94 Keynote

Two things are known about requirements:1. They will change!2. They will be misunderstood!


The Walkerfall Model

IterationBeta General

Release ReleaseIteration Iteration Iteration IterationPrototyping

Inception Elaboration Construction TransitionDevelopment Lifecycle

Architecture Useable DeploymentFeasibility

Iterations Iterations Iterations Iterations

The Lifecycle Macroprocess

Walker Royce’s Waterfall Model, Part I


IterationBeta General

Release ReleaseIteration Iteration Iteration IterationPrototyping

Inception Elaboration Construction Transition

Relative Effort by Activity

Management 15%

Reqts. capture 10%

Environment 5%

Analysis, design 15%

Implementation 30%

Testing 20%

Deployment 5%

Total Effort

Walker Royce’s Waterfall Model, Part II


Spiral ModelDetermine objectives, alternatives,

next level product

Develop, verify

Benchmarks

Models,

Simulations,

Risk analysis

identify, resolve risks

Evaluate alternatives;

Plan next phase

constraints

Barry Boehm’s Spiral Model


The spiral model solves the problem of theinappropriate sanctity of the requirementssimply because it is built around everchanging requirements.

The requirements for each sweep of the spiralis derived from the requirements of theprevious sweep and what was learned duringthe previous sweep.

The software is grown incrementally (more onthis later).


Lifecycle Steps

ConceptionRequirementsDesignCodingTestingMaintenance & Legacy SoftwareDocumentation


Conception

Hurrying to CodingCosts to Repair ErrorsCrunch ModeAnnualized DeliveryIncremental BuildMake vs. BuyStartupsClient/Server


Myth:

You people start the coding while I go seewhat the customer wants.

That is, because of impending deadlines, wegotta start working on the code before weknow exactly what the customer wants, and ifsomething the customer wants is a surprise,we can always change the code later!


Reality:

There is never enough time to do it right, butthere is always enough time to fix it or to do itover.

However, it always takes more time to fix itthan to have done it right or to do it over (noteven counting the fact that you have done ittwice).

When you fix it, it is always flaky and neverquite fixed (more on this later!).


What is doing it right?

It is working with client, domain experts, andtechnology experts until requirements areunderstood before designing or coding.

It is getting design worked out so that allmodules are known before coding.


Why?

As requirements change, so do design andcode.

As design changes, so does code (especiallyas a result of interface changes!).

Code is expensive to change, but design ischeaper to change, and requirements are evencheaper to change!


This does not mean not to prototype.

A throw-out prototype is written

g to help understand requirements,g to get client to understand what you

understand about what they said,g to answer questions about user interface,

algorithms, and performance.


Remember: Boehm and others have shownthat the cost to repair an error goes updramatically as project moves towardscompletion and beyond ...

In graph on the next slide, note that cost scaleis logarithmic, and

the graph itself looks exponential even on alogarithmic scale!!! Oy!

Graph is (y) relative cost to repair bug vs.(x) life cycle stage


Phase in which error is detected

1

2

5

10

20

50

100

Rel

ativ

e co

st to

cor

rect

err

or

Preliminarydesign

Detaileddesign

Code anddebug

Integrate Validate Operation


Myth:

Doing it right itself is a myth and doing itwrong is reality.

Evidence shows that skimping onrequirements analysis, specification, anddesign leads to lousy, buggy software that isbrittle and very expensive to repair andenhance.


However, doing it right takes too long in thesedays in which the first to the market gets thewhole market if the software is good enough(but not unless it is good enough!)(and who’sto define what’s good enough except themarket itself!), and later, even better productsare doomed to failure.


John Boddie’s crunch mode does work, butyou need some programmer’s equivalents ofMichael Jordan or Magic Johnson managed bythe programming manager’s equivalent ofPeter Ueberroth to pull it off.

If you have talented people, and the teamclicks just right, you can do it and you caneven learn to manage such teams.

Richard Botting has even produced a theoryshowing why crunch mode works and why ithas to work!


When the problems are compounded by tryingto do maintenance and new development atthe same time, both to keep your currentcustomers and to meet the market demandsfor new features, it can be hopeless.

Steve McConnel suggests AnnualizedSoftware Delivery


You have parallel teams,

g one maintaining the current version untildate D, and

g one aiming to release the new version ondate D.

The current version is retired on date D.


v1 v1.1 v1.2 ...

New Incremental

v2.1

Incremental

v2

New

Time

Information flow for ideas


Old Myth

If we try hard enough, we can get the programright the first time!


Perceived Reality

Fred Brooks says:

“Plan to throw one [the first one] away; youwill anyway!”

In other words, you cannot get it right until thesecond time.

This proved to be a myth too!


Counter-Reality

In keynote of ICSE ’95, Brooks admits that“Plan to throw one away” is wrong.

He says “Incremental Build is Better”

g A crucial function of the designer ishelping the client decide what he or shereally wants

g The best way to decide: rapid prototyping


Counter-Reality, cont’d

g Growing, rather than building, software

- quick to running software (You shouldsee the effect on the morale ofdevelopment team!)

- early user testing- build-to-budget is possible


Myth:

I/We can write a better X than that brain-damaged piece of ____ from ABC Co.

I/We can certainly write X for a lot less than itcosts us to buy it from those jerks at ABC Co.


Reality:

Suppose X has been out y years and it takesyou z years to implement X yourself, thenABC will always have y+z years head start ineliminating bugs and stabilizing their X.


On the assumption that ABC is charging a fairmarket price for X (and if they aren’t, they willnot be in business for long), then they arecharging per copy 3 to 6 orders of magnitudeless than it cost them to build it (on theassumption of selling thousands to millions ofcopies to recover their costs). There is no waythat you are going to build X for anywhereclose to what it costs you to buy it.

Furthermore, your software will always beflakier than theirs.


In general, if the product exists and is selling(and thus, the price is fair and the product isstable enough not to drive customers away),it’s always a better bet to buy it rather thanbuild it.

Furthermore, if your product W requires afunctioning X, buying X, incorporating it intoW, and agreeing to pay ABC a royalty for eachcopy of W(X) sold, allows you to get a stableW out to the market y+z years earlier!


Hopelessness of Startups

This all would seem to say that a startup ishopeless.

Well, it is ! 95% of them fail—poof!

If a startup is to succeed, it needs to makesomething at least an order of magnitudebetter or an order of magnitude cheaper thanwhat is out there; otherwise it cannot grab themarket.


And you gotta make it this order of magnitude{better|cheaper} in less time than anyone elsetrying to make it also, in what John Boddiecalls crunch mode.


Myth:

Distributed, Client/Server Systems are a majorbreakthrough in system architecture that willrevolutionize programming and make it eveneasier.


Reality:

It’s just another useful tool with its goodpoints and its not so good points.

While Burton Swanson in 1996 saw a nearsocial sweep for C/S technology, areplacement wave, Bob Glass in the same yearsaw deployment of C/S technology at only a26% level with mainframes holding at 66%.


But what of the complexity of D, C/S software?

g Components of a distributed system aresimpler.

g Decentralized systems as a whole are morecomplex.

Which dominates?


Scott Schneburger’s 1997 study showed thatthe complexity of decentralization faroutweighs the simplicity of the components.

Given that the total cost of maintenance issome 50-80% of the total cost of a system andthat complexity makes maintenance harder,the D, C/S technology may end up increasingsystem costs in the long run.


Requirements

PrototypingRequirements Difficult for ClientRequirements VolatilityStudy of Requirement ErrorsHow Hard Are They?Formal Methods for RequirementsSafety


More Myths:

A well-written, comprehensive requirementsspecification is all you need!

All of our problems would be solved if we hadwritten a complete requirements specification.


Reality & Myth:

Interview client.

Prepare requirements specification the size ofthe New York City telephone directory.

Give it to the client, saying “Let me know in aweek if it says what you want”.

A week later, the client says, “Yes!”

Do you believe him or her?


Reality:

Of course not!

Nonsense!


Another Way

You build a prototype and give it to the client

You walk him or her through it or you let himor her play with it for a week, saying “Let meknow in a week if it does what you want”.

A week later, the client says, “Yes!”

Do you believe him or her?


Reality:

Far more likely!

Scott Gordon and Jim Bieman observe thatusers are more likely to be comfortable with aprototype than a specification, which can bedull reading and open to many differinginterpretations; sample display output is moredefinitive. Thus, the prototype makes it easierfor users to make well-informed decisions andsuggestions.


Myths:

Several related myths (& more about aprevious one):

You people start the coding while I go find outwhat the customer wants.

Requirements are easy to obtain.

The client/user knows what he/she wants.


Reality:

According to Ruth Dameron (by e-mail):

The programmer who says these is sufferingfrom the myth that the customer would be ableto know what he or she wants and to say itjust because the programmer asked.


Most people (especially non-technicallyoriented) learn while doing; they’ve got to seesome kind of prototype (even if it’s only yellowstickies on a board) to discover what theywant.


Myth:

After the requirements are frozen, ...

When the customer is satisfied, ...

When the customer stops asking for changes,...


Reality:

Poppycock!!

The only customers that are satisfied andhave stopped asking for changes arethemselves frozen!


E-Type Systems

Meir Lehman identifies concept of E-typesystem.

It is a system that solves a problem orimplements an application in some real worlddomain.


Once installed, an E-type system becomesinextricably part of the application domain, sothat it ends up altering its own requirements.

g Consider a bank that exercises an optionto automate its process and then discoversthat it can handle more customers.

g It promotes and gets new customers, easilyhandled by the new system but beyond thecapacity of the manual way.

g It cannot back out of automation.g The requirements of the system have

changed!


Daily use of a system causes an irresistibleambition to improve it as users begin tosuggest improvements.

Who is not familiar with that, from either end?


In fact, data show that most maintenance isnot corrective, but for dealing with E-typepressures!

Perfective

Adaptive

Corrective

Oth

er


More Realities:

Martin & Tsai’s study of requirement errors:

They conducted an experiment to identifylifecycle stages in which requirement errorsare found.

An experienced user produced a polished 10-page requirements document for a centralizedrailroad traffic controller.


Ten 4-person teams of software engineerswere given the requirements document inorder to find errors in it.

The user believed that the teams would findonly 1 or 2 errors.


92 errors, some very serious, were found!

The average team found only 35.5 errors, i.e.,it left 56.5 to be found downstream!

Many errors were found by only one team!

The errors of greatest severity were found bythe fewest teams!

CONCLUSIONS: Requirements are hard to getright!


How hard are they?

Most errors are introduced duringrequirements specification!

Boehm: at TRW, 54% of all errors weredetected after coding and unit test; and, 65-85% of these errors were allocatable to therequirements, design, and documentationstages rather than the coding stage, whichaccounted for only 25% of the errors.


So most errors either are required or are theunplanned result of situations that are noteven mentioned in the requirementsspecifications.


So how do you find the requirements?

g Interviewg Observeg Become a userg Use imaginationg Prototypeg Validate all that you think you learng Accept that you will not find everything!


Myth:

If only you had written a formal specificationof the system, you wouldn’t be having theseproblems

Mathematical precision in the derivation ofsoftware eliminates imprecision


Reality

Yes, formal specifications are extremelyuseful in identifying inconsistencies inrequirements specifications, especially if onecarries out some minimal proofs ofconsistency and constraint or invariantpreservation,

just as writing a program for the specification!

Formal methods do not find all gaps inunderstanding!


As Eugene Strand and Warren Jones observe,“"Omissions of function are often difficult forthe user to recognize in formalspecifications”....

just as they are in programs!


von Neumann and Morgenstern (Theory ofGames ) say,

“There’s no point to using exact methodswhere there’s no clarity in the concepts andissues to which they are to be applied.”


Preservation of Difficulty

Indeed, Oded Sudarsky has pointed out thephenomenon of preservation of difficulty.Specifically, difficulties caused by lack ofunderstanding of the real world situation arenot eliminated by use of formal methods;instead the misunderstanding gets formalizedinto the specifications, and may even beharder to recognize simply because formaldefinitions are harder to read by the clients.


Bubbles in Wall Paper

Sudarsky adds that formal specificationmethods just shift the difficulty from theimplementation phase to the specificationphase. The “air-bubble-under-wallpaper”metaphor applies here; you press on thebubble in one place, and it pops upsomewhere else.


One Saving Grace

Lest, you think I am totally against formalmethods, they do have one positive effect,and it’s a BIG one:

Use of them increases the correctness of thespecifications.

Therefore, you find more bugs at specificationtime than without them, saving considerablemoney for each bug found earlier rather thanlater.


Myth:

Replace that ancient, slow, creaky electro-mechanical system that can wear out withsleek, fast, whiz-bang software software thatcan never wear out.


Reality:

Sure, it won’t wear out, but the questions are,“Will it even be correct?” and “If it is correct atall, will it be correct at all times?”

“If it ain’t broke, don’t fix it!”


Therac 25 Disaster

Between June 1985 and January 1987, thecomputer-controlled radiation therapymachine Therac-25 massively overdosed 6people, all of whom developed severedradiation sickness and all but 1 of whom hasdied (as of 1994). It was the worst accident inthe history of radiation therapy machines.


A study by Nancy Leveson showed that earliermachines, the Therac-6 and Therac-20, werecontrolled by computer, but the computer wasadded after the machines had been availablewith electromechanical (EM) controls. Inparticular, the safety controls were still EMeven after the addition of the computer.


In the Therac-25, designed from the start withcomputer control, more of the control,including the maintenance of safety, wasgiven to the computer.

Software checks were substituted for many ofthe traditional hardware interlocks.

Nominally, this was a good plan; they reusedcode that appeared to be reliable.


The problem was that the Therac-20 was areliable system !

The original Therac-20 software had a bug thatjust never showed up because theindependent hardware interlocks preventedoverdoses.

When they programmed the new checks intothis buggy code, and they happened to neverduplicate the error causing situation in thetests, the old bug was never discovered andreared its ugly head later with fatal results.


After much denial and protestation that theoverdose was impossible, the manufacturerwas forced to put the independent hardwareinterlocks back into the machine, just to besure, even after they had found and fixed thebugs.


Design

Extending PrototypeReuse


Myth:

The prototype can always be extended.

Reality:

Extending a prototype is a good way topreserve lousy design decisions.


Advice:

Build the prototype in a language like LISP orProlog so that temptation to turn it intoproduction version is reduced!


Myth:

Reuse will save money and increase softwarereliability

After all, a reused module does not have to bedesigned, developed, inspected, and testedagain. Moreover the fact that the reusedmodule has been out there under daily usemeans that it is far more tested, debugged,stable, and reliable than any newly writtenmodule for the same functionality.


Yes, the module designed for reuse costsabout twice what the same module designedfor only one-time use. However, you amortizethis extra cost by reusing the module dozens,hundreds, or thousands of times.


Note that there are many different kinds ofreuse:

g specificationg designg source codeg object codeg macrog procedureg class/abstract data type


and there are many means of customization tospecific requirements.

g parameter passingg parameter passingg white box modificationsg black box encapsulation


Reality:

Neil Maiden and Alistair Sutcliffe point out thatthe human issues in reuse confound theadvantages.

Finding opportunities for reuse, identifyingcandidate reusable components, adaptingthem to the current requirements, etc. maycost more than just developing from scratch.


A slight mismatch between the newrequirements and the reused module’sspecification may make more reliabilityproblems than exists in immature softwaredesigned for the specific occasion.

Identifying the right reusable components,comprehending them, customizing them, allnecessary for successful reuse, is a lot harderthan thought.


A 1991 IBM study by Kruzela on softwaremaintenance found that 50% of allmaintenance time is spent in justunderstanding the code to be changed; this iswhat makes maintenance so much moreexpensive per line than writing new code.

According to Thompson and Huff,understanding unfamiliar software is complexand error prone.

How well it is done by a given programmer is afunction of both skill and luck, and individualdifferences will come to play here.


However

However, as Ivar Jacobson, Marty Griss, andPatrik Jonsson point out, there arecircumstances in which reuse works and payswell.


In an organization producing a family ofrelated products, a carefully planned internalreuse business helps to tame thepandemonium that usually results as theorganization tries

g to maintain and enhance a variety ofexisting related products and

g to introduce new products in this familyrapidly, to stay ahead of the competition

all running on a variety of platforms.


In such a situation, large portions of the codein all versions, both revisions and variations,of the products in the family are very similar,with differences that are minor in size butmajor in importance.


When fully functioning, the reuse businessencourages developers

g to build adaptable modules for all functionsand

g to use and adapt already developedmodules when producingf new versions of existing products andf new products.

The reuse maturity model below summarizesthe way such a reuse business works.


None

Informalcode reuse

Black-boxcode reuse

Managedworkproduct

reuse

Architectedreuse

Domain-specificreuse-drivenorganization

Reducedmaintenance

costs

Broadercoverage

Interoperability,high reuse levels

Reduceddevelopment

time

Improved time to market, costs, quality

Bene

fit

Rapid custom productdevelopment

Investment, experience, time


Such reuse businesses avoid the problemsmentioned above because

g the modules subject to reuse are all forproducts in a single family of similarproducts

g the modules are designed from thebeginning with reuse in mind

Much of the mystery about which modules toreuse and how to adapt them is gone.


Coding

Coding CostsOptimizationObject Orientation


Myth:

Coding is expensive.

Coding is a major part of the lifecycle.

Coding is the resource eater.


Reality:

As it can be if you are careful

EXPENDITURE DISTRIBUTION OVER LIFE-CYLE

C T MDR

MTCDR

As it often is

R: Requirements Definition

C: Coding

T: Testing

M: Maintenance

D: Design


Even if you’re not doing it right (i.e., are doingthe top), coding is bubkes compared torequirement specification and design ortesting.

What is the implication of all this?

For one thing, methods and tools directed atonly the coding phase cannot have a majorimpact on the lifecycle.


Myth:

We gotta optimize it!

We gotta squeeze every last nanosecond outof it!

We gotta squeeze every last byte out of it!


Reality:

Today’s computers are sitting there twiddlingtheir thumbs > 95% of the time, even if thehuman users think they are using them all thetime.

For most software, efficiency just does notmatter.

The response time bottle neck is time to movecursor’s image on screen to the next line or todraw the characters being displayed.


Optimization?

First Rule (Don Knuth):

DON’T!With today’s machine speeds and costs, thesavings in machine time and its cost cannever equal the cost of the programmer’s timeto optimize.

With today’s underutilization of machines andfor most software, there’s no real need.


The lost in clarity of program and increase indifficulty to maintain are not worth it!

However, sometimes you must optimize(sigh!),

g to fit in machine,g to meet hard real-time constraints,g to beat performance of competitor in

commercial software.


Second Rule (Knuth):

NOT YET!In any case, it’s useless to optimize beforeyou have a running program.

Only then can you know what parts needoptimization and, thus, where it will pay off.

80% of the execution time is spent in 20% ofthe code.


If you knock a microsecond off something thatis executed only once, you’ve gained nothing,and maybe you’ve introduced an error!

If you knock a microsecond off something thatis executed 2,000,000 times, you’re reallysaving time!

You need to have a running, instrumentedprogram to determine where to optimize.


Third Rule (Berry):

Very Carefully ...

from a known correct program

Hide the optimizations in as small a module aspossible.

It is often better not even to optimize; find abetter algorithm-data-structure combination.


Last Rule (Berry):

DON’T!


C++ is like teenage sex

It is on everyone’s mind all the time.Everyone talks about it all the time.Everyone thinks everyone else is doing it.Almost no one is really doing it.The few that are doing it are:- doing it poorly,- sure it will be better next time, andnot practicing it safely.

— Graffiti found in a toilet stall in the Faculty ofComputer Science, Technion, November, 1993


C++ and teenage sex

It should say, “Programming in C++ is liketeenage sex”.

And even that’s wrong!

It should really say, “Object-orientedprogramming in C++ is like teenage sex”.

It is really inappropriate to equate OOP andC++; you can do OOP without C++, and youdon’t need to do OOP when you use C++.


Testing

BugsBug FrequencyRegression TestingTesting and Reviews

WalkthroughsInspection

Time for InspectionsGrowth of BugsBugs & ReliabilityN-Version Programming


Myth:

Bug

The word itself is a myth

It implies that a bug is something that anotherwise healthy program gets

Maybe as a result of contagion from sitting inthe same memory with other buggyprograms?!


Reality:

The bug that shows up after delivery to theclient was probably required into thesoftware, with probability of about 60%,according to data by Boehm and others.


Myth:

“Whew! that was the last bug!”

“We found the last bug! now we’ll fix it!”


Reality:

Mills says:

“The best way to know that you have foundthe last bug is never to find the first bug.”

Any bug, even a tiny one, is a sign ofsloppiness or lack of understanding indevelopment, and sloppy development or lackof understanding leads to lots of bugs, neverjust one.


Studies by Myers and others have shown thefollowing bug arrival graph over the testing ofone release.

Time

Bug

s fo

und

per

unit

time

That is, they tend to arrive in bunches if theyarrive at all.


faults

0

1

Number of faults already found

Probabilityof existenceof additional


More Reality:

Dijkstra says:

“Testing can be used to show the presence oferrors, never their absence!”

This quote suggests a certain mind set fortesters.

They should be trying to find errors ratherthan trying to show that there are none.

Goals have a bad habit of turning into reality!


Myth:

After an error is found in an unexpected place:

But, I tested that part before and didn’t touch itfor this new change!


Reality:

All parts of a program are connected to allother parts, even if you don’t think so,especially if you have pointers (and who doesnot?).

This is why experienced testers re-run allpreviously run test cases whenever a newversion of a program is produced.


There is even a name for this kind of testing,Regression Testing.

Having a program automatically running testcases and comparing actual to expectedoutput helps a lot.


Myth:

Let’s test it thoroughly.

Testing will find the problems.

We’ll find the bugs later when we test it.


Testing is running a program againstpredetermined data for the purpose ofdetecting a difference between the program’soutput and the expected output for thepurpose of finding errors in the program.

Test cases and expected outputs aredeterminedg according to the specifications, exercising

every feature, option, etc.g according to the program structure,

exercising every statement, path, etc.


Thoroughly testing a program is impossible(requires unbounded number of test cases);so try to choose test cases that will expose allerrors.

That’s very difficult, especially since we donot know what all the errors are, and if we did,we would not need the test cases!


Reality:

A number of studies have shown testing notvery effective at finding bugs.

They have compared errors located by variousmethods to all errors ever reported for aproduct over its lifetime:

g Inspections found 67 – 82% of them.

g Walkthroughs found around 38% fewer ofthem than inspections.

g Traditional testing found 15 – 50% of them.


The studies are described and summarized intwo good books on software inspection, oneby Gilb and Graham and the other by Ebenauand Strauss.


Walkthrough:

Author of a module describes the innerworkings of the module to a group of people,generally from the same project, for them tospot internal errors and inconsistencies aswell as interface problems with their ownmodules.


Inspection:

Walkthrough +

g Reviewers are given the module at least aday before meeting and are told to reviewthe module privately ahead of time.

g Meeting moderated by facilitator.g Minutes of meeting captured by recorder.g Meet for exactly two hours.


g Goal of meeting is to find as manyproblems as possible.

g No time is wasted at meeting to considersolutions.

g Author finds solutions later and submitsmodule again for another inspection.


Why are walkthroughs and inspections moreeffective?

Probably because the walkers through and theinspectors are humans who are capable ofusing their keppeles (noggins) to think.

When was the last time you saw a test casethink?


Ah, but why cannot the human running thetest cases think? He or she can, but generallythere are too many test cases, very purposelyto get maximum coverage, to allow time forcareful thinking about the implications ofeach.

Also many times, the test cases are run by aprogram that runs each case and comparesthe output with the expected output, reportingonly deviations; this driver does not think, andhumans have fewer opportunities to thinkabout test cases in general.


Note that the difference in the effectiveness ofwalkthroughs and inspections is probably dueto the formality difference.


Still Pretty Bad

As good as inspections are, the situationleaves a bit to be desired.

Even with inspections, the data show that atleast 18% of the bugs found in a program overits lifetime will be found by the customers andusers

And of course, you know what kind ofwonders this does for your company’sreputation!


Myth:

We don’t have time to do inspections now onevery step!

We gotta finish the code sooner so we can testit sooner.


Reality:

If you don’t have time to do the inspectionsthen you don’t have time to finish the projectsatisfactorily, period.

g In any case, an error that is in the softwarenow is not going to disappear by itself; youhave to find it first.

g If you don’t find it now, then you may ormay not find it later before shipping.


g If you don’t find it before shipping, then thecustomer will find it.

g In any case, finding it later means that itcosts (time, money, whatever) 10 to 100times to fix as finding it now.

g Finding it now means the least cost (time,money, whatever) to fix it.

g So if you don’t have time to inspect, find it,and fix it now, then you certainly will nothave time later, when it will take longer tofix it.


Myth:

The next release will be better!

The bug will be fixed in the next release!

When the product is finally bug-free ...


Ha!


Reality:

The famous Belady-Lehman graph of bugsfound over all releases of a product:

Time

Bug

s fo

und

per

unit

time


This curve is the result of a theory thatattempts to explain a well-known observedphenomenon of eventual unbounded growthof errors in software that was beingcontinually fixed, i.e., a general decay inmodified software.

The theory assumes an ε > 0 probability ofintroducing more than one error whencorrecting one error.


This suggests that at a certain point, either:

g declare all current and remaining bugs tobe features (Knuth has decreed that uponhis death, all remaining bugs in TEX andMETAFONT are features!)

g start all over with a new programdevelopment or maybe reengineer thesoftware.


More Reality:

The real-life graph is not as smooth as thetheoretically derived graph.

Time

Bug

s fo

und

per

unit

time


We will not be able to identify the real minpoint until we are well past it.

So this means we must keep the sources of allreleases, with a configuration managementsystem, so we can roll back to to the bestrelease, some number of releases ago.


Myth:

The program has to be perfect.

We have to squeeze out all bugs.


Reality:

Reliable computations are obtainable frombuggy programs, which after all, are the onlykind of programs there are!

David Parnas observed:

I can build a reliable system with thousands ofbugs, if you let me choose my bugs carefully.


Working around bugs is possible and is oftenrequired.

Both users and programs evolve to allowusers to get trustable computations fromprograms despite bugs.

Simply, users learn not to use the bug that is,in effect, a feature (unless of course they wantthat feature!).


Myth:

The number of defects is a good predictor ofreliability:

The fewer the defects the greater thereliability

Hence some measure defects duringdevelopment as evidence of code reliabilityand to calculate the likely reliability.


Reality:

It’s mostly right, as we observed before, but..

Shari Lawrence Pfleeger, Ross Jeffrey, BillCurtis, and Barbara Kitchenham caution not tocount all defects the same way.

They report that Ed Adams of IBM found that80% of the reliability problems are caused byonly 2% of the defects.


We need to examine the defects carefully tosee which cause reliability problems.

Remember what David Parnas says:

“I can build a reliable system with thousandsof bugs, if you let me choose my bugscarefully.”


Myth:

A standard approach to hardware fault-tolerance, to have multiple versions of a unit,is a great approach to software reliability.

N-version programming will increase yoursoftware’s reliability.


Reality:

N-version programming is a good way tospend N times the original costs while gettingno more reliability and possibly even less.


Hardware

In hardware, the problem is decayingcomponents.

Executing multiple copies of the unitconcurrently, voting, and using the majority(say 2 out of 3) result is a good way to guardagainst unit failure.


But note what is being protected against:

physical component failurenot

design errors

If none of the units have failed physically, thenif there is a design error that causes anerroneous result on one of them, then all ofthem will have the same result.

The approach works because relative to thecomputational speed, the time until unit failureis independent of that of all other units.


Software

Assuming no failure of the underlyinghardware, all copies of the same program willalways produce the same result, even whenthey are committing an error.

Therefore, N voting copies of a program willalways choose unanimously and obviouslyhas the same reliability as the program itself.


For software, the reliability problem is that ofdesign errors; programs do not wear out; thatis a program presented with the same inputwill always produce the same result no matterhow many times the program has been runand how long it has been sitting in thememory unused.

In other words, multiple identical softwareunits are no more a protection from designerrors than are multiple identical hardwareunits.


If N identical software units are useless, thenperhaps N independently developed programswith the same intended functionality will help.

The theory shows, but it should be clear too,that this approach to protecting againstdesign errors depends on the errors that eachprogram can make being independent of thoseof every other program.


Experiments by Nancy Leveson and JohnKnight show that this independenceassumption does not hold, that in fact,different programmers working from the samespecification tend to make the same errors.

In fact every experiment with the N-versionapproach to software fault tolerance has foundthat independently written software routinesdo not fail in statistically independent ways.


People tend to make the same mistakes in thesame harder parts of the problem, with theessentially the same well-known bestalgorithm and in the same boundary cases ofthe input space.

Any shared specification can lead to commonfailures and testers omitting the same nominaland unusual cases that the developersoverlooked.

Consequently a majority of so-calledindependent programs might indeed vote forthe same erroneous output.


Ah! so have the programmers program fromindependently written specifications.

But then, it is not even clear that theyprograms will be implementing the samerequirements! We all know how hard it is toget requirements right.


Moreover, even when programming from thesame requirements, there are situations inwhich N so-called independently developedprograms will be less reliable than any one.

The first experiments in N-versionprogramming were conducted in the early1980s in my SE class at UCLA with a simple5-command formatting program.


Different programs had exactly the same listof words on each line, but with differentdistributions of extra white space between thewords on any given line.

A human would not regard these lines asdifferent, but the original voting program didregard them as different and in fact producedfewer correct results than any one program.

Many times no majority could be found at all!


Ah! So change the definition of agreement!

But then the voting program begins to bemore complex and its own reliability begins tobe an issue.

The conclusion: There are far better andcheaper approaches to improving reliabilitythan writing the same program N times, e.g.,inspection which typically costs 15% more.


Maintenance & Legacy Software

PervasivenessDominance


Myth:

Design/coding/development is where theaction/excitement/money is!

Most programmers develop new code.

Most of a programmer’s work is developingnew code.


Reality:

In 1980s,

Maintenance

Specification

Requirements

Planning

Inte

grat

ion

Coding

Design

Testing


Per David Parnas, 1994

100

80

60

40

1960 1970 1980 1990 2000

Spen

t on M

ainte

nan

ceP

erce

nta

ge

of

Soft

war

e R

eseo

urc

es


We are heading to the day in which nearly allof our work will be modifying existingsoftware.

Even today, a new field is springing up, that ofLegacy Software, existing software that is

g too valuable to scrap,g too difficult to modify or extend without

error,g too expensive to rebuild, butg inadequate in its current form.


Jim Horning observed:

Hardware is the part you can replace.

Software is the part you have to keep,

because it’s so expensive and nobodyunderstands it any more!


Relative Hardware-Software Costs

1950 19900%

100%

Hardware Software


The last two realities suggest that the key tokeeping software costs down is to write codethat is easily modified.

Choose programming methods whose guidingconcern is making inevitable modificationseasier.


For example, Parnas’s information hidingmethod tries to decompose a system intomodules such that whenever theimplementation of one concept is changed,only the one module implementing thatconcept is affected!


The Y2K Problem

You’ve all heard of the famous Y2K problem,the Year 2000 Problem?


Myth:

The problem with the Y2K is that all of the datastructures are big enough for only the last twodigits of the year. Therefore, come the year2000, we’re going to need more digits or else2000 will look smaller than 1999.


Reality:

Given the cheap cost of memory these days,the space problem is minor, compared to theproblem of having to suddenly work withdifferent algorithms that take into account thechanges, i.e.,

g 00 is really more than 99,org some dates are 4 digits and others are 2,org ...


More Reality:

The different algorithms problem is minorcompared to the fact that everything will haveto be recompiled with new data structuredefinitions and new date calculationalgorithms.


Even More Sobering Reality:

All of the space problems and all of the funnynew algorithms and all of the recompilationrequired are bubkes (zilch, nada, rien, nichts,klum) compared to the REAL problem with theY2K...

What is it??????


The REAL problem:The REAL problem is that all of the code todeal with dates is scattered all over all of thelegacy programs, very often not identified bycomment and very often disguised asarithmetic or shifting or other obscurities.

Compared to finding all the code to change,and you have to find it all or else the programbombs on 1 January 2000, all the otherproblems are a piece of cake, a puzzledesigned for 2 year olds, putting a round peginto a round hole, etc. etc.


Solutions

All solutions that attack only the datastructure and algorithm problems are doomedto repeat history.

No matter what size you set the data to be,there will be a maximum representable date,although maybe very far into the future, and atthat date, X you will have a YX problem.

The only REAL solution is one that attacks theREAL problem.


g Build an encapsulated Date abstraction.g Hide the data structure and algorithm

inside of it.g Rewrite all the code, replacing code that

uses the date data structure directly bycalls to proceudres of the Date abstraction.

You still pay for the data structure andalgorithm changes, the recompilation, and thefinding of all date accessing code...


BUT you never again face the REAL problem.

At YX , you change the inside of the Dateabstraction and recompile; you do not have tofind the using code again.

That is, we were pretty stupid about the datesonce, but we will have learned from ourmistake not to repeat history.


Documentation


Myth:

We’ll document that later when we have time.Now, we gotta finish up the coding to meet thedeadline.

We’ll update the documentation next Tuesdayafter we get this module out of the way.


Reality:

Documentation that is put off never seems toget written!

Either,

g there never is enough time to write it, org when there is enough time to write it, we’ve

forgotten what we wanted to say


Guindon, Krasner, and Curtis have studieddesigners and have concluded:

Many design breakdowns occur because ofcognitive limitations:

g Designers forget to return to design goalsthey have postponed.

g While working on one part they cannotrecord opportunistic design ideas thataffect another part.


Sound familiar?

Likewise with documentation!

Also, if one is documenting duringprogramming, these breakdowns are lesslikely.


“The job is not done until the paper work isdone”

The problem with documentation is that it isperceived as a necessary evil which is doneonly after the fact

So it must be done during the programming!

Remember, there never is enough time to do itright!


But it is not going to be done duringprogramming unless there is an incentive todo so.

Also any technological or managerial schemeto force documentation can be subverted byunwilling programmers.


Theory

Mathematics vs. ProgrammingKnuth DiscoversMushinessNP-Completeness


Myth:

Mathematics is harder than programming.

Proving theorems is harder thanprogramming.

The true sign of intellectual achievement isbeing able to prove theorems, notprogramming; programming is trivial.


Reality:

Each new program is a new formal system,modeling a real-world system, and this formalsystem is built from the ground (or library) up!

Each program is a formal system in the sensethat one can reason logically about itsbehavior.


The data of a program form a model of therelevant real-world domain, and the operationsof the program transform this modelaccording to a related model of real-worldtransformations.


Correspondences:Basic Statements AxiomsConstructors RulesData DefinitionsInvariantsStatements

MNO

TheoremsFunctions

So, programming is at least as difficult asdeveloping a mathematical theory.


But, not all programs build a new theory!

True, but not all mathematics is new either!


Actually, programming is harder in severalvery important senses!

First, consider the audience of the work:

Theorems PeopleUKWIM worksCan accept imprecision

Programs ComputersUKWIM does not workDWIM does not workCannot accept imprecision


Mathematicians will make simplifyingassumptions to keep the math tractable; thegoal is usually good math, not alwaysmodeling reality, e.g., Euclidean geometry.

Software cannot make simplifyingassumptions that can cause deviations fromreality that invalidate functionality, e.g., inprocess control of fast and dangeroussituations.

Therefore, program models tend to be morecomplex than models that mathematiciansstudy.


The notions of correctness in mathematicsand programs are different.

A mathematical model must be consistent; itneed not match reality (be correct), and it neednot be complete (in the formal sense).

A program model must be consistent; it mustmatch reality; and it must be complete (in thesense that it reacts gracefully to all inputs).

For example,g √ is not defined for input < 0g sqrt must deal with all input


Social processes for mathematics andprograms are different.

Theorems written by mathematicians forpublication

g undergo scrutiny of other interestedmathematicians,

g are interesting; otherwise, why bother?

As a result, errors are found and corrected.


Programs

g are not looked at by other programmers,g are boring.

Therefore, there are no error-finding socialprocesses.


Program proofs verify only consistency withspecification and not correctness, and aremeaningless if the specification is not what isintended.

Testing shows only the presence of errors andnot their absence.

Therefore, it is much harder to ensure thecorrectness of programs than theorems.


Note that a key point of inspections is to try tointroduce to programming the socialprocesses that are so successful at findingerrors in theorems.


Knuth and Software

Donald E. Knuth, one of the premier computerscientists in the world, who has done suchmathematically respectable work as:

g Member of Algol 60 Committeeg Attribute Grammarsg 3-Volume Encyclopedia on Algorithmsg Knuth-Bendix Algorithm

has spent ten years of his life developing twomajor programs for document typesetting, TEXand METAFONT.


These projects started out as a brief attemptto make sure that all of his subsequent books,to be printed with computer-driventypesetters, would look as good as his earlierhand-typeset books.

They mushroomed into 10-year effortsyielding, to date, two releases of each.


I recall Knuth’s giving a lecture at UCLA,about one year after starting, in which he saidthat he had hoped to have TEX fully running bytonight—as if another day or so would havecracked the problem that was preventing itfrom running!

It took him another 10 years to finish, and hestill is not finished.

Knuth published a paper in 1989, The Errors ofTEX, describing this effort and listing the 867errors found by users in the 14,000 line Pascalprogram.


Knuth gave a keynote address at IFIP ’89“Theory and Practice”.

He explained that one of the lessons learnedfrom the development of his typesettingsoftware is that “software is hard.”


What were the lessons I learned from somany years of intensive work on thepractical problem of setting type bycomputer? One of the most importantlessons, perhaps, is the fact thatSOFTWARE IS HARD.... From now on I shallhave significantly greater respect for everysuccessful software tool that I encounter.


During the past decade I was surprised tolearn that the writing of programs for TEXand for METAFONT proved to be much moredifficult than all the other things I had done(like proving theorems or writing books).The creation of good software demands asignificantly higher standard of accuracythan those other things do, and it requires alonger attention span than other intellectualtasks.


The remark that writing software is moredifficult and requires greater accuracy thanproving theorems merits close examination.

The difference is the audience.

The programmer is writing for an incrediblystupid and literal audience, the computer, thatcannot tolerate minor incompleteness.

The mathematician is writing for a highlyintelligent audience, the professionalmathematicians, that can be counted on to fillin on missing or wrong details.


But note that for Knuth, proving theorems ishard too.

He, above all, knows how easily publishedtheorems contain, plainly and simply,mistakes.

Knuth has had to publish two consecutivecorrections to a published proof of a theoremabout attribute grammars.


Myth:

Software engineering (as an academicdiscipline) is so soft and mushy. How can youwork in it?


Reality:

Bill Curtis says

“Whenever I discuss human issues insoftware engineering someone always says,‘Right, that’s the soft side of computerscience.’ ... Nevertheless, with all theallusions to soft, mushy material, hardprograms continue to be written by humansand not by machines, expert systems,automatic program generators, and otherobjects worthy of federal funding.”


It was computer scientists that observed thatthere is a serious problem in softwaredevelopment. To my mind, if we have aproblem that we have to solve, we gotta gowhere the problem takes us and explorewhatever solutions may work. If thosesolutions are non-technical and not based ontheory, so be it.


The fact is that for nearly 30 years now, wehave attempted to solve the problem withtechnology, theory, etc. These have madesome advances, but the problems remain andseem to have gotten worse, in some sense(perhaps due to ambition fueled bysuccesses). Recently, we have begun torecognize that the problems will not becracked unless we also consider non-technical issues.

Note word “also”.


Myth:

The problem is NP-complete! Therefore,

g we can give only small inputs to theprogram (for the problem),

g we have to use heuristic methods,g we have to accept only an approximate

solution,g we have to accept the possibility of no

solution being found,g we may have to wait an unacceptably long

time.


Reality:

Actually, usually the above is not a myth, butoccasionally we have TechnologicalTrivialization of the problem.

Ultimately any problem can be thought of afinite problem, i.e., for any problem, we canbuild a finite state machine that will solve allthe problem for all input no larger than a givenmaximum.


However, usually, the number of states of themachine is so large as to be impractical; thememory of real machines is too small or realmachines are not fast enough to complete thecomputation in a reasonable time even thoughthe complexity of the solution is linear (with aBIG multiplicative constant).


So we have gotten into the habit ofconsidering the problem unbounded andfinding algorithms that handle all possibleinputs (the Turing Machine game).

Many times these algorithms have exponentialor higher growth.

So we work on heuristic, probabilistic, andapproximate solutions.


However, surprise, surprise, surprise!!

For some of these problems, machine sizesand speeds have grown to the point that for allpossible inputs that happen in real life,exploring all possible states is feasible.

We can suddenly write programs that givecomplete, exact solutions in a reasonableamount of time for all possible inputs.


For example, the problem of optimal floor planlayout according to specified criteria isnormally NP-complete.

However, recently a representation for a floorplan was discovered such that:

For all floor plans that are possible for anormal house (e.g., < 20 rooms),

it is possible to generate all unique-up-to-symmetries floor plans and to evaluate allaccording to the criteria in less than 1hour.


Suddenly, all the heuristic, approximate, andprobabilistic solutions are unnecessary.

Same thing happened with relationaldatabases, which were only a nice theory untilmid 80s.


Conclusions


Truth:

A good software engineer is a lazy one.

What kind of laziness?

g planning ahead to avoid work

g reuse


Truth:

We software engineers need humility, humilityto recognize our limitations,

g the need for help, andg the limitations of the help, both

methodological and technical.


myths and realities in software development - the david r

Documents