they write the right stuff _ fast company

8/12/2019 They Write the Right Stuff _ Fast Company

1/19

07/08/12 hey Write the Right Stuff | Fast Company

Page 1ttp://www.fastcompany.com/28121/they-write-right-stuff

Skip navigation

Subscribe

Newsletter

Search

Follow @fastcompany 536K followers

Like 15k

Co.DESIGN

Co.CREATE

Co.EXIST

Co.LEAD

TECHNOLOGY

MOST CREATIVE PEOPLE

MAGAZINE

Features

They Write the RightStuff

y Charles Fishman

ecember 31, 1996

As the 120-ton space shuttle sits surrounded by almost 4 million pounds of rocket fuel, exhaling noxious fumes,

isibly impatient to defy gravity, its on-board computers take command.

The right stuff kicks in at T-minus 31 seconds.

As the 120-ton space shuttle sits surrounded by almost 4 million pounds of rocket fuel, exhaling noxious fumes,

isibly impatient to defy gravity, its on-board computers take command. Four identical machines, runningdentical software, pull information from thousands of sensors, make hundreds of milli-second decisions, vote on

very decision, check with each other 250 times a second. A fifth computer, with different software, stands by to

ake control should the other four malfunction.

At T-minus 6.6 seconds, if the pressures, pumps, and temperatures are nominal, the computers give the order to

ght the shuttle main engines -- each of the three engines firing off precisely 160 milliseconds apart, tons of

uper-cooled liquid fuel pouring into combustion chambers, the ship rocking on its launch pad, held to the ground
http://www.fastcompany.com/most-creative-people/2012http://www.fastcompany.com/technologyhttp://www.fastcompany.com/leadershiphttp://www.fastcodesign.com/http://ad.doubleclick.net/clk;254486090;45085438;o?http://itunes.apple.com/us/app/fast-company-magazine/id500813317?mt=8http://ad.doubleclick.net/clk;254486090;45085438;o?http://itunes.apple.com/us/app/fast-company-magazine/id500813317?mt=8http://ad.doubleclick.net/click%3Bh%3Dv8/3cca/2/0/%2a/d%3B257850051%3B0-0%3B0%3B43924277%3B3454-728/90%3B48567120/48565548/1%3B%3B~aopt%3D2/1/ff/0%3B~sscs%3D%3fhttp://googleads.g.doubleclick.net/aclk?sa=L&ai=BDi9XlochUKqeI5KwsQfX7oDQDqeZpbkCx4uPnS7PibiXFvD21gUQARgBINTsmhU4AFDjkIGJ______8BYOXj84SAFrIBE3d3dy5mYXN0Y29tcGFueS5jb226AQk3Mjh4OTBfYXPIAQLaATdodHRwOi8vd3d3LmZhc3Rjb21wYW55LmNvbS8yODEyMS90aGV5LXdyaXRlLXJpZ2h0LXN0dWZmqQIREOj2lVh0PsACBMgCt4-XHKgDAcgDHegDnAXoAwX1AwIEAEDIBAGgBgI&num=1&sig=AOD64_1YvM5cZ0jK2MJhuimDu8zQo0zB9Q&client=ca-pub-9569526099065935&adurl=http://www.telcel.com/portal/roaming/begin.do%3Fmid%3D1500http://ad.doubleclick.net/click%3Bh%3Dv8/3cca/2/0/%2a/d%3B257850051%3B0-0%3B0%3B43924277%3B3454-728/90%3B48567120/48565548/1%3B%3B~aopt%3D2/1/ff/0%3B~sscs%3D%3fhttp://googleads.g.doubleclick.net/aclk?sa=L&ai=BDi9XlochUKqeI5KwsQfX7oDQDqeZpbkCx4uPnS7PibiXFvD21gUQARgBINTsmhU4AFDjkIGJ______8BYOXj84SAFrIBE3d3dy5mYXN0Y29tcGFueS5jb226AQk3Mjh4OTBfYXPIAQLaATdodHRwOi8vd3d3LmZhc3Rjb21wYW55LmNvbS8yODEyMS90aGV5LXdyaXRlLXJpZ2h0LXN0dWZmqQIREOj2lVh0PsACBMgCt4-XHKgDAcgDHegDnAXoAwX1AwIEAEDIBAGgBgI&num=1&sig=AOD64_1YvM5cZ0jK2MJhuimDu8zQo0zB9Q&client=ca-pub-9569526099065935&adurl=http://www.telcel.com/portal/roaming/begin.do%3Fmid%3D1500http://ad.doubleclick.net/click%3Bh%3Dv8/3cca/2/0/%2a/d%3B257850051%3B0-0%3B0%3B43924277%3B3454-728/90%3B48567120/48565548/1%3B%3B~aopt%3D2/1/ff/0%3B~sscs%3D%3fhttp://googleads.g.doubleclick.net/aclk?sa=L&ai=BDi9XlochUKqeI5KwsQfX7oDQDqeZpbkCx4uPnS7PibiXFvD21gUQARgBINTsmhU4AFDjkIGJ______8BYOXj84SAFrIBE3d3dy5mYXN0Y29tcGFueS5jb226AQk3Mjh4OTBfYXPIAQLaATdodHRwOi8vd3d3LmZhc3Rjb21wYW55LmNvbS8yODEyMS90aGV5LXdyaXRlLXJpZ2h0LXN0dWZmqQIREOj2lVh0PsACBMgCt4-XHKgDAcgDHegDnAXoAwX1AwIEAEDIBAGgBgI&num=1&sig=AOD64_1YvM5cZ0jK2MJhuimDu8zQo0zB9Q&client=ca-pub-9569526099065935&adurl=http://www.telcel.com/portal/roaming/begin.do%3Fmid%3D1500http://www.fastcompany.com/28121/they-write-right-stuff#searchhttp://www.fastcompany.com/#contenthttp://www.fastcompany.com/user/charles-fishmanhttp://www.fastcompany.com/magazine/6http://www.fastcompany.com/magazinehttp://www.fastcompany.com/most-creative-people/2012http://www.fastcompany.com/technologyhttp://www.fastcompany.com/leadershiphttp://www.fastcoexist.com/http://www.fastcocreate.com/http://www.fastcodesign.com/http://ad.doubleclick.net/clk;254486090;45085438;o?http://itunes.apple.com/us/app/fast-company-magazine/id500813317?mt=8http://www.fastcompany.com/28121/they-write-right-stuff#searchhttp://www.fastcompany.com/newslettershttps://magazine.fastcompany.com/loc/FST/topnavhttp://www.fastcompany.com/#contenthttp://ad.doubleclick.net/click%3Bh%3Dv8/3cca/2/0/%2a/d%3B257850051%3B0-0%3B0%3B43924277%3B3454-728/90%3B48567120/48565548/1%3B%3B~aopt%3D2/1/ff/0%3B~sscs%3D%3fhttp://googleads.g.doubleclick.net/aclk?sa=L&ai=BDi9XlochUKqeI5KwsQfX7oDQDqeZpbkCx4uPnS7PibiXFvD21gUQARgBINTsmhU4AFDjkIGJ______8BYOXj84SAFrIBE3d3dy5mYXN0Y29tcGFueS5jb226AQk3Mjh4OTBfYXPIAQLaATdodHRwOi8vd3d3LmZhc3Rjb21wYW55LmNvbS8yODEyMS90aGV5LXdyaXRlLXJpZ2h0LXN0dWZmqQIREOj2lVh0PsACBMgCt4-XHKgDAcgDHegDnAXoAwX1AwIEAEDIBAGgBgI&num=1&sig=AOD64_1YvM5cZ0jK2MJhuimDu8zQo0zB9Q&client=ca-pub-9569526099065935&adurl=http://www.telcel.com/portal/roaming/begin.do%3Fmid%3D1500https://twitter.com/intent/user?original_referer=http%3A%2F%2Fwww.fastcompany.com%2F28121%2Fthey-write-right-stuff&region=count_link&screen_name=fastcompany&source=followbutton&variant=2.0https://twitter.com/intent/follow?original_referer=http%3A%2F%2Fwww.fastcompany.com%2F28121%2Fthey-write-right-stuff&region=follow_link&screen_name=fastcompany&source=followbutton&variant=2.0


2/19



nly by bolts. As the main engines come to one million pounds of thrust, their exhausts tighten into blue

iamonds of flame.

Then and only then at T-minus zero seconds, if the computers are satisfied that the engines are running true,

hey give the order to light the solid rocket boosters. In less than one second, they achieve 6.6 million pounds of

hrust. And at that exact same moment, the computers give the order for the explosive bolts to blow, and 4.5

million pounds of spacecraft lifts majestically off its launch pad.

's an awesome display of hardware prowess. But no human pushes a button to make it happen, no astronaut

ockeys a joy stick to settle the shuttle into orbit.

The right stuff is the software. The software gives the orders to gimbal the main engines, executing the dramatic

elly roll the shuttle does soon after it clears the tower. The software throttles the engines to make sure the craft

oesn't accelerate too fast. It keeps track of where the shuttle is, orders the solid rocket boosters to fall away,

makes minor course corrections, and after about 10 minutes, directs the shuttle into orbit more than 100 miles

p. When the software is satisfied with the shuttle's position in space, it orders the main engines to shut down --

weightlessness begins and everything starts to float.

But how much work the software does is not what makes it remarkable. What makes it remarkable is how well

he software works. This software never crashes. It never needs to be re-booted. This software is bug-free. It is

erfect, as perfect as human beings have achieved. Consider these stats : the last three versions of the program

- each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors.

Commercial programs of equivalent complexity would have 5,000 errors.

This software is the work of 260 women and men based in an anonymous office building across the street from

he Johnson Space Center in Clear Lake, Texas, southeast of Houston. They work for the "on-board shuttle

roup," a branch of Lockheed Martin Corps space mission systems division, and their prowess is world

enowned: the shuttle software group is one of just four outfits in the world to win the coveted Level 5 ranking of

he federal governments Software Engineering Institute (SEI) a measure of the sophistication and reliability of the

way they do their work. In fact, the SEI based it standards in part from watching the on-board shuttle group do its

work.

The group writes software this good because that's how good it has to be. Every time it fires up the shuttle, their

oftware is controlling a $4 billion piece of equipment, the lives of a half-dozen astronauts, and the dreams of the

ation. Even the smallest error in space can have enormous consequences: the orbiting space shuttle travels at

7,500 miles per hour; a bug that causes a timing problem of just two-thirds of a second puts the space shuttle

hree miles off course.

NASA knows how good the software has to be. Before every flight, Ted Keller, the senior technical manager of

he on-board shuttle group, flies to Florida where he signs a document certifying that the software will not

ndanger the shuttle. If Keller can't go, a formal line of succession dictates who can sign in his place.

Bill Pate, who's worked on the space flight software over the last 22 years, says the group understands the


3/19



takes: "If the software isn't perfect, some of the people we go to meetings with might die.

Software is everything. (It also sucks.)

n the history of human technology, nothing has become as essential as fast as software.

Virtually everything -- from the international monetary system and major power plants to blenders and microwavevens -- runs on software. In office buildings, the elevators, the lights, the water, the air conditioning are all

ontrolled by software. In cars, the transmission, the ignition timing, the air bag, even the door locks are

ontrolled by software. In most cities so are the traffic lights. Almost every written communication that's more

omplicated than a postcard depends on software; every phone conversation and every overnight package

elivery requires it.

Software is everything. It also sucks.

t's like pre-Sumerian civilization," says Brad Cox, who wrote the software for Steve Jobs NeXT computer and is

professor at George Mason University. "The way we build software is in the hunter-gatherer stage."

ohn Munson, a software engineer and professor of computer science at the University of Idaho, is not quite so

enerous. "Cave art," he says. "It's primitive. We supposedly teach computer science. There's no science here at

ll."

Software may power the post-industrial world, but the creation of software remains a pre-industrial trade.

According to SEI's studies, nearly 70% of software organizations are stuck in the first two levels of SEI's scale of

ophistication: chaos, and slightly better than chaos. The situation is so severe, a few software pioneers from

ompanies such as Microsoft have broken away to teach the art of software creation (see "Drop and Code me

Twenty!" )

Mark Paulk, a senior member of the SEI technical, says the success of software makes its weaknesses all the

more dramatic. "We've developed software products that are enormously complex and enormously powerful.

We're critically dependent on it," says Paulk. "Yet everyone complains how bad software is, with all the defects. If

ou bought a car with 5,000 defects, you'd be very upset."

n this software morass, the on-board shuttle group stands out as an exception. Ten years ago the shuttle group

was considered world-class. Since then, it has cut its own error rate by 90%.

To be this good, the on-board shuttle group has to be very different -- the antithesis of the up-all-night, pizza-and-

oller-hockey software coders who have captured the public imagination. To be this good, the on-board shuttle

roup has to be very ordinary -- indistinguishable from any focused, disciplined, and methodically managed

reative enterprise.

n fact, the group offers a set of textbook lessons that applies equally to programmers, in particular, and

roducers, in general. A look at the culture they have built and the process they have perfected shows what
http://www.fastcompany.com/magazine/06/dropcode.html


4/19



oftware-writing must become if software is to realize its promise, and illustrates what almost any team-based

peration can do to boost its performance to achieve near-perfect results.

Software for Grown-Ups

Shipping hell continued today. Grind, grind, grind. We'll never make it. Have I said that already? Why do we

lways underestimate our shipping schedules? I just don't understand. In at 9:30 AM; out at 11:30 PM Dominosor dinner. And three diet Cokes."

No, it's not the on-board shuttle group. It's Douglas Coupland's "Microserf's," a true-to-life fictional account of life

n the software-fast-lane. And it's the dominant image of the software development world: Gen-Xers sporting T-

hirts and distracted looks, squeezing too much heroic code writing into too little time; rollerblades and mountain

ikes tucked in corners; pizza boxes and Starbucks cups discarded in conference rooms; dueling tunes from

Smashing Pumpkins, Alanis Morrisette and the Fugees. Its the world made famous, romantic, even inevitable by

tories out of Sun Microsystems, Microsoft, and Netscape.

's not the story of the on-board shuttle group. Their quarters are a study in white-collar pedestrian. The most

triking thing is how ordinary they look. Other than the occasional bit of shuttle memorabilia, you could be in the

ffices of any small company or government agency. Everyone has his or her own small office, and the offices

ave desks, PCs, and sparse personal artifacts. People wear moderately dressy clothes to work, neat but

othing flashy, certainly nothing grungy.

's strictly an 8-to-5 kind of place -- there are late nights, but they're the exception. The programmers are

ntense, but low-key. Many of them have put in years of work either for IBM (which owned the shuttle group until

994), or directly on the shuttle software. They're adults, with spouses and kids and lives beyond their

emarkable software program.

That's the culture: the on-board shuttle group produces grown-up software, and the way they do it is by being

rown-ups. It may not be sexy, it may not be a coding ego-trip -- but it is the future of software. When you're

eady to take the next step -- when you have to write perfect software instead of software that's just good enough

- then it's time to grow up.

Ted Keller, 48, the group's senior technical manager, looks and sounds like the headmaster of a small private

igh school. It's Keller's job to make sure the software gets delivered on time, with all its capabilities, without

egard to turf battles. He's a compact, bald man, a little officious and persnickety, qualities any astronaut would

nd reassuring. He has a gentle, geeky sense of humor, not so much with outsiders, but with his crowd.

comes out in a meeting between members of the software group and their NASA counterparts. It's held in a

mall conference room overstuffed with 22 people and an overhead projector. Several times, from the back of

he room, Keller issues a wry remark about the speed of code delivery, or the detail of some specifications, and

he room lightens with laughter.

Otherwise, the hour-long meeting is sober and revealing, a brief window on the culture. For one thing, 12 of the


5/19



2 people in the room are women, many of them senior managers or senior technical staff. The on-board shuttle

roup, with its stability and professionalism, seems particularly appealing to women programmers.

For another, it's an exercise in order, detail, and methodical reiteration. The meeting is a classic NASA

erformance -- a rehearsal for an almost identical meeting several weeks away. It consists of walking through an

normous packet of data and view -- graphs which describe the progress and status of the software line by line.

Except for Keller's occasional asides, the tone is businesslike, almost formal, the view - graphs flashing past as

uickly as they can be read, a blur of acronyms, graphs, and charts.

What's going on here is the kind of nuts-and-bolts work that defines the drive for group perfection -- a drive that is

ggressively intolerant of ego-driven hotshots. In the shuttle group's culture, there are no superstar

rogrammers. The whole approach to developing software is intentionally designed not to rely on any particular

erson.

And the culture is equally intolerant of creativity, the individual coding flourishes and styles that are the signature

f the all-night software world. "People ask, doesn't this process stifle creativity? You have to do exactly what the

manual says, and you've got someone looking over your shoulder," says Keller. "The answer is, yes, the processoes stifle creativity."

And that is precisely the point -- you can't have people freelancing their way through software code that flies a

paceship, and then, with peoples lives depending on it, try to patch it once its in orbit. "Houston, we have a

roblem," may make for a good movie; it's no way to write software. "People have to channel their creativity into

hanging the process," says Keller, "not changing the software."

The tight strictures the group practices can make the siren song of the rock n roll software world hard to resist.

Quinn Larson, 34, had worked on shuttle software for seven years when he left last January to go to work for

Micron Technology in Boise, Idaho, automating the manufacture of Microns memory chips.

At Micron, Larson was given the task to automate the saws that cut finished chip wafers to the right size. Screw

p the program, you destroy the valuable wafers.

t was up to me to decide what to do," says Larson. "There were no meetings, there was no record-keeping." He

ad freedom; it was a real kick. But to Larson's way of thinking, the culture didn't focus on, well, the right stuff.

Speed there was the biggest thing," he says. "The engineers would say, these are our top priorities, and we

eed to get'em as fast as we can." But the impression Larson got was that engineers didn't seem too concerned

bout how well the finished software actually worked. "Basically, they wanted quick software -- just put it out the

oor."

arson started back at the shuttle group in mid-August. "The people here are just of the highest caliber," he said

n his first day back in Clear Lake.

t's the Process


6/19



How do they write the right stuff?

The answer is, it's the process. The group's most important creation is not the perfect software they write -- it's

he process they invented that writes the perfect software.

's the process that allows them to live normal lives, to set deadlines they actually meet, to stay on budget, to

eliver software that does exactly what it promises. It's the process that defines what these coders in the flat

lains of southeast suburban Houston know that everyone else in the software world is still groping for. It's the

rocess that offers a template for any creative enterprise that's looking for a method to produce consistent - and

onsistently improving -- quality.

The process can be reduced to four simple propositions:

. The product is only as good as the plan for the product.

At the on-board shuttle group, about one-third of the process of writing software happens before anyone writes a

ne of code. NASA and the Lockheed Martin group agree in the most minute detail about everything the newode is supposed to do -- and they commit that understanding to paper, with the kind of specificity and precision

sually found in blueprints. Nothing in the specs is changed without agreement and understanding from both

ides. And no coder changes a single line of code without specs carefully outlining the change. Take the upgrade

f the software to permit the shuttle to navigate with Global Positioning Satellites, a change that involves just

.5% of the program, or 6,366 lines of code. The specs for that one change run 2,500 pages, a volume thicker

han a phone book. The specs for the current program fill 30 volumes and run 40,000 pages.

Our requirements are almost pseudo-code," says William R. Pruett, who manages the software project for

NASA. "They say, you must do exactly this, do it exactly this way, given this condition and this circumstance."

This careful design process alone is enough to put the shuttle organization in a class by itself, says John Munson

f the University of Idaho. Most organizations launch into even big projects without planning what the software

must do in blueprint-like detail. So after coders have already started writing a program, the customer is busily

hanging its design. The result is chaotic, costly programming where code is constantly being changed and

nfected with errors, even as it is being designed.

Most people choose to spend their money at the wrong end of the process," says Munson. "In the modern

oftware environment, 80% of the cost of the software is spent after the software is written the first time -- they

on't get it right the first time, so they spend time flogging it. In shuttle, they do it right the first time. And they

on't change the software without changing the blueprint. That's why their software is so perfect."

. The best teamwork is a healthy rivalry.

Within the software group, there are subgroups and subcultures. But what could be divisive office politics in other

rganizations is actually a critical part of the process.


7/19



The central group breaks down into two key teams: the coders - the people who sit and write code -- and the

erifiers -- the people who try to find flaws in the code. The two outfits report to separate bosses and function

nder opposing marching orders. The development group is supposed to deliver completely error-free code, so

erfect that the testers find no flaws at all. The testing group is supposed to pummel away at the code with flight

cenarios and simulations that reveal as many flaws as possible. The result is what Tom Peterson calls "a

riendly adversarial relationship."

They're in competition for who's going to find the errors," says Keller. "Sometimes they fight like cats and dogs.

The developers want to catch all their own errors. The verifiers get mad, 'Hey, give it up! You're taking away from

ur time to test the software!'"

The developers have even begun their own formal inspections of the code in carefully moderated sessions, a

gorous proof-reading they hope will confound the testers. The verifiers, in turn, argue that they deserve credit

or some errors found before they even start testing. "From the verification group's perspective," says Pat

McLellan, a senior manager, "we're aware that if there was no independent verification group, the developers

would tend to be more lax. Just the presence of our group makes them more careful."

The results of this friendly rivalry: the shuttle group now finds 85% of its errors before formal testing begins, and

9.9% before the program is delivered to NASA.

. The database is the software base.

There is the software. And then there are the databases beneath the software, two enormous databases,

ncyclopedic in their comprehensiveness.

One is the history of the code itself -- with every line annotated, showing every time it was changed, why it was

hanged, when it was changed, what the purpose of the change was, what specifications documents detail the

hange. Everything that happens to the program is recorded in its master history. The genealogy of every line of

ode -- the reason it is the way it is -- is instantly available to everyone.

The other database -- the error database -- stands as a kind of monument to the way the on-board shuttle group

oes about its work. Here is recorded every single error ever made while writing or working on the software,

oing back almost 20 years. For every one of those errors, the database records when the error was discovered;

what set of commands revealed the error; who discovered it; what activity was going on when it was discovered -

testing, training, or flight. It tracks how the error was introduced into the program; how the error managed to slip

ast the filters set up at every stage to catch errors -- why wasn't it caught during design? during development

nspections? during verification? Finally, the database records how the error was corrected, and whether similar

rrors might have slipped through the same holes.

The group has so much data accumulated about how it does its work that it has written software programs that

model the code-writing process. Like computer models predicting the weather, the coding models predict how

many errors the group should make in writing each new version of the software. True to form, if the coders and

esters find too few errors, everyone works the process until reality and the predictions match.


8/19



We never let anything go," says Patti Thornton, a senior manager. "We do just the opposite: we let everything

other us."

. Don't just fix the mistakes -- fix whatever permitted the mistake in the first place.

The process is so pervasive, it gets the blame for any error -- if there is a flaw in the software, there must be

omething wrong with the way its being written, something that can be corrected. Any error not found at the

lanning stage has slipped through at least some checks. Why? Is there something wrong with the inspection

rocess? Does a question need to be added to a checklist?

mportantly, the group avoids blaming people for errors. The process assumes blame - and it's the process that

s analyzed to discover why and how an error got through. At the same time, accountability is a team concept: no

ne person is ever solely responsible for writing or inspecting code. "You don't get punished for making errors,"

ays Marjorie Seiter, a senior member of the technical staff. "If I make a mistake, and others reviewed my work,

hen I'm not alone. I'm not being blamed for this."

Ted Keller offers an example of the payoff of the approach, involving the shuttles remote manipulator arm. "Weelivered software for crew training," says Keller, "that allows the astronauts to manipulate the arm, and handle

he payload. When the arm got to a certain point, it simply stopped moving."

The software was confused because of a programming error. As the wrist of the remote arm approached a

omplete 360-degree rotation, flawed calculations caused the software to think the arm had gone past a

omplete rotation -- which the software knew was incorrect. The problem had to do with rounding off the answer

o an ordinary math problem, but it revealed a cascade of other problems.

Even though this was not critical," says Keller, "we went back and asked what other lines of code might have

xactly the same kind of problem." They found eight such situations in the code, and in seven of them, the

ounding off function was not a problem. "One of them involved the high-gain antenna pointing routine," says

Keller. "That's the main antenna. If it had developed this problem, it could have interrupted communications with

he ground at a critical time. That's a lot more serious."

The way the process works, it not only finds errors in the software. The process finds errors in the process.

ust a Software Problem

The B-2 bomber wouldn't fly on its maiden flight -- but it was just a software problem. The new Denver airportwas months late opening and millions of dollars over budget because its baggage handling system didn't work

ght -- but it was just a software problem. This spring, the European Space Agency's new Ariane 5 rocket blew

p on its maiden launch because of a little software problem. The federal government's major agencies - from

he IRS to the National Weather Service -- are beset with projects that are years late and hundreds of millions of

ollars over budget, often because of simple software problems. Software is getting more and more common and

more and more important, but it doesn't seem to be getting more and more reliable.


9/19



As the rest of the world struggles with the basics, the on-board shuttle group edges ever closer to perfect

oftware. Admittedly they have a lot of advantages over the rest of the software world. They have a single

roduct: one program that flies one spaceship. They understand their software intimately, and they get more

amiliar with it all the time. The group has one customer, a smart one. And money is not the critical constraint: the

roups $35 million per year budget is a trivial slice of the NASA pie, but on a dollars-per-line basis, it makes the

roup among the nation's most expensive software organizations.

And that's the point: the shuttle process is so extreme, the drive for perfection is so focused, that it reveals what's

equired to achieve relentless execution. The most important things the shuttle group does -- carefully planning

he software in advance, writing no code until the design is complete, making no changes without supporting

lueprints, keeping a completely accurate record of the code -- are not expensive. The process isn't even rocket

cience. Its standard practice in almost every engineering discipline except software engineering.

Plastered on a conference room wall, an informal slogan of the on-board shuttle group captures the essence of

eeping focused on the process: "The sooner you fall behind, the more time you will have to catch up."

Charles Fishman ([email protected]) is a writer based in Raleigh, North Carolina.

A version of this article appears in the Dec 1996/Jan 1997issue of Fast Company.

You Might Also Like

[?]

Share:

Tweet

Print

Watch This ManBuild An AmazingCardboard BicycleThat You CanActually Ride

CO.EXIST

The 4 DumbestRules That Will KillYour Company'sCulture | FastCompany

FAST COMPANY

Top 5 FranchisesWorth Looking At

THE BUSINESS

FINANCE STORE

ShareShare

45Like

FROM THE WEB
http://www.facebook.com/plugins/like.php?action=like&api_key=178479832213933&channel_url=http%3A%2F%2Fstatic.ak.facebook.com%2Fconnect%2Fxd_arbiter.php%3Fversion%3D9%23cb%3Df19dc5c88%26origin%3Dhttp%253A%252F%252Fwww.fastcompany.com%252Ff356e1e4c8%26domain%3Dwww.fastcompany.com%26relation%3Dparent.parent&extended_social_context=false&href=http%3A%2F%2Fwww.fastcompany.com%2Fnode%2F28121&layout=button_count&locale=en_US&node_type=link&sdk=joey&send=false&show_faces=false&width=90#http://www.fastcompany.com/print/28121http://www.fastcompany.com/28121/they-write-right-stuff#http://www.fastcompany.com/magazine/6https://twitter.com/intent/tweet?original_referer=http%3A%2F%2Fwww.fastcompany.com%2F28121%2Fthey-write-right-stuff&related=null&source=tweetbutton&text=They%20Write%20the%20Right%20Stuff&url=http%3A%2F%2Fwww.fastcompany.com%2Fnode%2F28121&via=FastCompanyhttp://void%280%29/


10/19



Like

Sort by best rating

Email

Newsletter

Subscribe

Browse Current Issue !

Real-time updating is enabled. (Pause)

Showing 11 comments

LoginAdd New Comment

Type your comment here.

Hi man I and hundreds of others was writing flawless code back in the dark ages, apart from raw

machine code, assembler and Fortran, there was C not "C++" or "C#" or C~!@#$%^ just plane C the

brain child of two American heros Brian Kernigan and Dennis M Ritchie and straight out of Bell Labs.

using C everything from 4-bit micro controllers to the UNIX operating system has been crafted; and here

is the rub. Something like the sequence control you describe for the Space Shuttle is fairly basic

compared to an Oil Refinery or a Nuclear Power Plants safety critical systems which if it fails would be a

lot more lethal, if less spectacular than a ST brewing up on the pad. You can also bet these are multe-

redundant systems with every system duplicated not once but a number of times. The Apollo 11 moon

lander Eagle had 7 systems - theyt all failed just above the surface of the moon so it was left to NeilArmstrong to land by the seat of his pant - proof that the best redundant system is a human, you cannot

beat the skills of trained U.S military pilots Navy, Marine or Airforce, nor the observational capacity of

the mark-one eyeballs attached to such gentalmen.

Spirals

cortney valentina
http://disqus.com/fastcompany-249515/http://www.fastcompany.com/28121/they-write-right-stuff#http://www.fastcompany.com/28121/they-write-right-stuff#http://www.fastcompany.com/28121/they-write-right-stuff#http://www.fastcompany.com/magazine/currenthttp://www.fastcompany.com/magazine/currenthttps://magazine.fastcompany.com/loc/FST/articlehttp://www.fastcompany.com/newslettershttp://www.fastcompany.com/forward/?path=node/28121http://www.fastcompany.com/28121/they-write-right-stuff#http://www.fastcompany.com/28121/they-write-right-stuff#http://www.fastcompany.com/28121/they-write-right-stuff#


11/19



This is a pretty interesting article, with some strong points. Thanks for the post.

so good paper. thanks gor your share good information,plz keep up good work.

...hot videos

james lena

Outside, in the real world, this isnt possible.

Costs way too much, has external clients to please...

Done that way, a photoshop licence would cost millions...

Bob

Bleh, I think if I had to work in an environment like that I would shoot myself within a month.

Programming is as much art as science, and the best programmers I know are also the most creative.

While I admit that when programs are controlling life and death, you obviously don't want mistakes in

your code, but most programs are not about life and death.

Making a hugely complex program that has 5000+ bugs is normal because we don't want to take 20

years to write 420,000 lines of code. We want to do 1,000,000+ lines in less than a year, get things to a

state that they work in almost all cases, then move on to bigger and better things. Those 5000 bugs thearticle talks about are usually edge cases that just don't MATTER (i.e. If my character walks too close to

a specific model of a standing lamp when I have the hairstyle blonde_long_31, the ends of the hair clip

slightly into the lamp. So what. We leave those 5000 bugs in because no one gives a shit about them).

Obviously, much worse bugs DO slip through the cracks sometimes, and no one is proud of that, but

bugs like that are the relatively rare exception. Evolution is about making mistakes and all good

programmers make mistakes. But the industry as a whole has made incredible technology possible

because we've not been afraid to try big things and fall flat on our faces sometimes. The Computer

Science profs the article quotes are idiots. Our industry is not in a cave-man stage at all. We've

propelled many other industries OUT of the cave-man stage because of the HUGE leaps and bounds of

progress in software engineering.And contrary to what the article implies, the mountain-dew-fueled,

bleary-eyed programmer culture is just as responsible for the success of that endeavor as the suit-and-

tie programmers in that company. I'll bet they use IDEs made by those imperfect programmers. I'll bet

they didn't write the OS of the computers they use to write their code. I'm pretty damn sure the science

they rely on to launch that shuttle was created by lots of engineers, physicists, chemists, etc. that would

never have been able to make the discoveries, manufacturing processes, or or economies of scale that

made things possible without using software created by imperfect programmers.The mindset of anyone

that would want all programmers to be like the ones in the article boggles me. The corporate suits want

programmers to be predictable and boring. In a limited number of cases (specifically ones like the article

Arthur Lee Wilson
http://www.facebook.com/people/Arthur-Lee-Wilson/578098929http://disqus.com/facebook-578098929/http://disqus.com/guest/84ea925f6472174984dd0819c893c10c/http://disqus.com/guest/82e1733fc7fdfafd6698593824208887/http://www.porn007.net/http://disqus.com/guest/e020003025524beaae569097d06f0494/


12/19



describes - where the software HAS to be right or people DIE), that kind of programming is necessary

and I applaud the programmers who can do it. If we ALL worked like that, though, the world would be a

VERY different (and far less advanced!) place!

I could probably write this perfection of space shuttle software in about 200 lines of PHP... Just sayin'.

Greg Bowler

"Four identical machines, running identical software, pull information from thousands of sensors, make

hundreds of milli-second decisions, vote on every decision, check with each other 250 times a second"

This is where the Amaizing happens..pretty impressive computations..

Albert Mutangiri

That's where the Amazing happens => Four identical machines, running identical software, pull

information from thousands of sensors, make hundreds of milli-second decisions, vote on every

decision, check with each other 250 times a second. Pretty impressive computations!!

Albert Mutangiri

Ive never been interested in all that nasa stuff, but this article made me think. im of to wikipedia to read

some more :)

andy k

that is stupid to pay 35 millions just for maintenance

Video Girl

With all due respect to the author, $35 million per year to maintain a 420,000 line program is expensive.

Tony BenBrahim
http://disqus.com/guest/5492e2550d35190f483a5d3ed9a56cbf/http://disqus.com/guest/c016ca2c042566a87133982520c8857f/http://disqus.com/guest/759093e5aeedf1d175981ca2447a795c/http://disqus.com/guest/7af215eebd2abd3ef8ee6d3f0cc7298f/http://disqus.com/guest/7af215eebd2abd3ef8ee6d3f0cc7298f/http://disqus.com/fastcompany-d4ea190043f3b44b7b91229d7a23b44c/


13/19



M Subscribe by email S RSS

Reactions

Who's That Woman in the Twitter Bot Profile?

Most organizations and consumers are not willing to shoulder than kind of expense. And no, this kind of

process is not standard in every engineering discipline. There are tradeoffs for example in automotive

design, it is possible to design and build a car that never breaks down, but would anyone be able to

afford it? Yes, there are more rigourous engineering processes when the stakes are high (airplanes,

chemical plants, etc..), but they come with commensurate costs. For a consumer operating system or a

word processor, or a corporate line of business application, faster and cheaper wins out very time over

flawless and unaffordable.
http://twitter.com/rhetonik/status/232354139571630080http://ad.doubleclick.net/click%3Bh%3Dv8/3cca/3/0/%2a/h%3B253829833%3B0-0%3B0%3B43924277%3B18754-336/150%3B47738169/47753592/1%3B%3B~aopt%3D2/1/ff/0%3B~sscs%3D%3fhttp://ad.doubleclick.net/clk;256069769;76982327;hhttp://www.fastcompany.com/3000064/whos-woman-twitter-bot-profilehttp://www.fastcompany.com/3000064/whos-woman-twitter-bot-profilehttp://fastcompany.disqus.com/they_write_the_right_stuff/latest.rsshttp://www.fastcompany.com/28121/they-write-right-stuff#


14/19



Inside Axe's Highly Scientific, Totally Irresistible

Marketing Machine Built On Lust

It Takes Guts To Start A Company--So How Do YouGet 'Em?

The Surprisingly Smart Strategy Behind LondonsInfamous Olympic Branding
http://www.fastcodesign.com/1670429/the-surprisingly-smart-strategy-behind-london-s-infamous-olympic-brandinghttp://www.fastcompany.com/3000132/it-takes-guts-start-company-so-how-do-you-get-emhttp://www.fastcompany.com/3000132/it-takes-guts-start-company-so-how-do-you-get-emhttp://www.fastcompany.com/3000210/inside-axes-highly-scientific-totally-irresistible-marketing-machine-built-lusthttp://www.fastcocreate.com/1681417/inside-axes-highly-scientific-totally-irresistible-marketing-machine-built-on-lusthttp://ad.doubleclick.net/click;h=v8/3cca/0/0/%2a/v;253330114;0-0;0;43924277;4307-300/250;46615438/46632081/1;;~aopt=2/1/ff/0;~sscs=%3fhttp://www.fastcodesign.com/


15/19



Office 15: Microsoft's Best Bet For Beating Dropbox, iCloud, Google Drive

With one billion Office users worldwide, the newest upgrade to Microsoft's productivity suite ...

Infographic: A Map Of Every Big Idea Ever Printed
http://www.fastcodesign.com/1670395/infographic-a-map-of-every-big-idea-ever-printedhttp://ad.doubleclick.net/click;h=v8/3cca/0/0/%2a/p;257075505;0-0;0;43924277;18754-336/150;48159853/48174101/1;;~aopt=2/1/ff/0;~sscs=%3fhttp://fastcompanyconnection.com/http://www.fastcompany.com/embed/90049c4f0b4bbhttp://www.fastcompany.com/embed/6bad939b18b32http://www.fastcompany.com/embed/06a56413c5fc0http://www.fastcompany.com/1842877/office-15-microsofts-best-bet-beating-dropbox-icloud-google-drivehttp://www.fastcompany.com/3000154/surprisingly-smart-strategy-behind-london%E2%80%99s-infamous-olympic-branding


16/19



5 Years In The Making, Lufthansas Ingenious NewBusiness Class Seats

The 4 Dumbest Rules That Will Kill Your Company'sCulture

Hate Small Talk? These 5 Questions Will Help YouWork Any Room
http://www.fastcompany.com/1843752/hate-small-talk-these-5-questions-will-help-you-work-any-roomhttp://www.fastcompany.com/1844194/4-dumbest-rules-will-kill-your-companys-culturehttp://www.fastcompany.com/1844194/4-dumbest-rules-will-kill-your-companys-culturehttp://www.fastcompany.com/1844346/5-years-making-lufthansa%E2%80%99s-ingenious-new-business-class-seatshttp://www.fastcodesign.com/1670423/5-years-in-the-making-lufthansa-s-ingenious-new-business-class-seatshttp://www.fastcompany.com/1844470/infographic-map-every-big-idea-ever-printed


17/19


18/19



Minnesota Courts Cyclists With A New MobilePlatform

masthead

Executive Editor:Noah Robischon

Editorial Director:Tyler Gray

News Editor:Mac Montandon

Editor, Co.Design:Cliff KuangEditor, Co.Exist:Morgan Clendaniel

Editor, Co.Create:Teressa Iezzi

Editor, Co.Lead:Erin Schulte

more
http://www.fastcompany.com/staff-mastheadhttp://www.fastcompany.com/user/erin-schultehttp://www.fastcocreate.com/users/teressa-iezzihttp://www.fastcoexist.com/users/morgan-clendanielhttp://www.fastcodesign.com/users/cliff-kuanghttp://www.fastcompany.com/user/maccabee-montandonhttp://www.fastcompany.com/user/tyler-grayhttp://www.fastcompany.com/user/noah-robischonhttp://www.fastcompany.com/1844218/minnesota-courts-cyclists-new-mobile-platformhttp://www.fastcodesign.com/1670392/minnesota-courts-cyclists-with-a-new-mobile-platformhttp://www.fastcompany.com/1844101/why-apples-next-iphone-important-first-one


19/19


Page 19ttp://www fastcompany com/28121/they-write-right-stuff

dvertise

bout Us

vents

ubscribe

Manage Subscriptions

ewsletters

eprints

rivacy

erms of Service

opyright 2012 Mansueto Ventures LLC. All rights reserved. Fast Company, 7 World Trade Center, New York, NY

0007-2195

Close
http://www.fastcompany.com/28121/they-write-right-stuff#http://www.aboutads.info/http://www.fastcompany.com/terms-and-conditionshttp://www.fastcompany.com/privacy-policyhttp://www.fastcompanyreprints.com/http://www.fastcompany.com/newslettershttp://www.fastcompany.com/renewhttps://magazine.fastcompany.com/loc/FST/Footerhttp://www.fastcompany.com/eventshttp://www.fastcompany.com/about-ushttp://mediakit.fastcompany.com/http://ad.doubleclick.net/click;h=v8/3cca/0/0/%2a/l;255400019;0-0;0;43924277;4307-300/250;46933357/46949660/1;;~aopt=2/1/ff/0;~sscs=%3fhttp://www.fastcocreate.com/

they write the right stuff _ fast company

Documents