0018-9162/03/$17.00 © 2003 IEEE June 2003 39

G U E S T E D I T O R S ’ I N T R O D U C T I O N

Agile SoftwareDevelopment: It’s aboutFeedback and Change

ilarities, these practitioners authored their method-ologies independently.

Underlying valuesIn February 2001, 17 of these practitioners and

authors met in Snowbird, Utah, to discuss the fun-damental similarities of their experiences and theirthen-called lightweight methodologies.2 Finding thattheir work habits had much in common, they rec-ognized that the lightness in their work process pro-vided a means to achieve the more relevant end:customer satisfaction and high quality. They cate-gorized their methodologies as “agile”—a term witha decade of use in flexible manufacturing practices.

The participants wrote the “Manifesto for AgileSoftware Development” (www.agilemanifesto.org),which describes the four comparative values under-lying the agile position:

• individuals and interactions over processes andtools,

• working software over comprehensive docu-mentation,

• customer collaboration over contract negotia-tion, and

• responding to change over following a plan.

These four comparative values recognize the impor-tance of the items on the right of each comparison,

Currently, the focus is on determining how to blend agile methodologies with plan-drivenapproaches to software development.

LaurieWilliamsNorth Carolina StateUniversity

Alistair CockburnHumans and Technology

A gile software development has hit a nervein the software development community.Some people argue vociferously for it, oth-ers argue equally against it, and others areworking to mix agile and plan-driven

approaches. Many more people wonder just whatagility is.

WHAT IS AGILITY?In many development methods, particularly plan-

driven ones, work begins with soliciting and docu-menting a complete set of requirements. Beginningin the mid-1990s, many found this initial require-ments documentation step frustrating and, perhaps,impossible.1 Both technology and the business envi-ronment kept shifting during the project, and boththe requirements and project plans got out of datewithin even relatively short projects. Customersbecame increasingly unable to definitively state theirneeds up front.

As a result, practitioners developed methodolo-gies and practices to embrace, rather than reject,higher rates of change. These methodologies weredeveloped on three different continents: theDynamic Systems Development Method in Europe;Feature-Driven Development in Australia; andExtreme Programming, Crystal, Adaptive SoftwareDevelopment, and Scrum in the US. Although theirpractices and philosophies shared fundamental sim-

P u b l i s h e d b y t h e I E E E C o m p u t e r S o c i e t y

40 Computer

but assert the dominance of the ones on the left.

Empirical versus defined processesAnother point of commonality for all

agile methods is the recognition of softwaredevelopment as an empirical (or nonlinear)process. In engineering, processes are classi-fied as defined or empirical.3 A defined processis one that can be started and allowed to runto completion, producing the same resultsevery time.4 Assembling an automobile is sucha process. Engineers can design a process toassemble the car and specify an assembly

order and actions on the part of the assembly-lineworkers, machines, and robots. Generally speak-ing, if the manufacturing process follows these pre-defined steps, it will produce a high-quality car.

Software development cannot be considered adefined process because too much change occursduring the time that the team is developing theproduct. It is highly unlikely that any set of prede-fined steps will lead to a desirable, predictable out-come because requirements change, technologychanges, people are added and taken off the team,and so on.

In an engineering context, empirical processesnecessitate short “inspect-and-adapt” cycles andfrequent, short feedback loops.4 These shortinspect-and-adapt cycles can help agile method-ologies better handle the software industry’s con-flicting and unpredictable demands.

Other empirical methods that are in synch withthe Agile Manifesto can also be considered agile.Agility, would-be agility, and various competitivecounterparts have become hot topics in conferencesand magazines—including IEEE Computer Societypublications.5-8 There are now both books andannual international conferences on the topic.

CURRENT CONVERSATIONSAs practitioners have gained experience with the

use of agile methods, the focus of their conversa-tions about this approach to software developmenthas changed.

Suitable circumstancesIn 2001, the conversation around agile was: What

is it really? Is it old, new, a fad, crazy, or what?We found that agile development is not particu-

larly new. Software developers have been sporadi-cally using the techniques since at least the 1960s.What is new, if anything, is the bundling of the tech-niques into a theoretical and practical framework

and the strong, sometimes vehement, declarationof their importance.

In 2001, enough sample projects had been col-lected to make the case that agile developmentworks under “suitable circumstances.” These cir-cumstances generally can be categorized as non-safety-critical projects with volatile requirements,built by relatively small and skilled colocated teams.

ScalabilityIn 2002, the agile conversation focused on two

questions: How do we make it scale beyond these“suitable circumstances?” Does agility contradictthe Software Engineering Institute’s CapabilityMaturity Model and ISO 9000 rules?

A few projects of several hundred people haveused the agile framework succesfully.9 However, dur-ing 2002, agile practitioners and other researchersconcluded that the agile value set and practices bestsuit colocated teams of about 50 people or fewerwho have easy access to user and business expertsand are developing projects that are not life-critical.Although the agile value set might be adopted (andmay be adapted) under other circumstances, theboundary conditions for truly agile behavior seemto be fairly set.

A close but liberal reading of the CMM revealslittle that would prevent CMM-concerned teamsfrom using agile methods.10 However, some com-mon CMM practices are at odds with the agilevalue set because of their reliance on processes,plans, and delivering to original contract.Generally, people willing to spend the money onCMM certification are less interested in the agilevalue proposition, while those needing agility forbusiness reasons are less interested in getting CMMor ISO 9000 certification.

AdaptabilityIn 2003, the conversation has changed again:

Can we blend selected agile practices with our “reg-ular” practices? How much change is necessarywhen transitioning to and using agile methods?How can agile practices improve the quality of ourproducts?

Development groups are trying pair program-ming without any other agile practices, test-firstdevelopment without pair programming, and shortdelivery cycles without pair programming or test-first development, and trying to determine how toget any of these practices to work with interna-tionally distributed teams. Researchers worldwideare gathering these agile subset experiences anddesigning and staging additional experiments to

A point of commonality for all agile methods is the recognition

of software development

as an empiricalprocess.

empirically assess the efficacy of isolated practicesor agile subsets.

We see agile philosophies and practices dispersedover a wider range of people, projects, and organi-zations. For example, many CMM and ISO 9000organizations now think that partial adoption ofagile practices, when handled with care, mightincrease their efficiencies without damaging theircertifications.

Making changesAs agile development spreads from the early

adopters to a larger population of developers anddevelopment managers, the question of change iscoming to the fore.

Agile development carries with it the implicationof changing work habits. For those who have notused agile techniques, the change involves tryingout and selecting appropriate ones. For thosealready using some of the techniques, the changeinvolves adjusting and improving their use overtime. Some proportion of the development com-munity has no interest in change, however, whichmay impose an inevitable limit to the penetrationof agile techniques.

Agile development can affect the power structurewithin an organization because it spreads out thedecision-making authority. Executives are expectedto make and take accountability for business-leveldecisions, deferring to the developers on technicalissues such as development techniques and timeestimates. The developers are then accountable forthese issues.

This decision-making approach differs fromwhat we see in many organizations. In some, theprogrammers make many key decisions, includingthose pertaining to business, process, and systemsrequirements, while their managers either happilyor reluctantly go along for the ride. In other orga-nizations, programmers have limited decision-mak-ing authority, and they are treated as little morethan clerical workers.

In either case, the move to agile development,with its reallocation of decision-making authorityand accountability, can cause consternation. Weencounter managers who report that they consideragile development to be a license for developers tohack, but we also encounter developers who con-tend that agility is merely a ploy to let managersmicromanage their work. Just how to handle thispower shift remains an open question.

Finally, some question the ability of agile prac-tices to improve product quality. Agile practicesstress early involvement of test groups and the need

for rapid feedback, but they don’t say muchabout the relationship between the develop-ment and quality assurance groups. Integrat-ing quality assurance into projects to enhancetheir agility requires making changes in therelationship between these groups and evenin hiring practices.

IN THIS ISSUEThe articles in this issue capture some

results from earlier conversations and set thestage for new ones.

In their short and colorful “Agility throughDiscipline: A Debate,” Kent Beck and Barry Boehmsuggest that agile and plan-driven developmentexist in different ecological niches and that eachwill thrive or dominate depending on the “currentweather conditions.” Somewhere between the ele-phant, the monkey, the mouse, and the dinosaur,their debate conveys the message that bothapproaches require discipline to work—althoughwith some different shadings of what disciplinemeans—and that each works well in specific circumstances.

Craig Larman and Vic Basili’s article, “Iterativeand Incremental Development: A Brief History,”traces this cornerstone agile practice back to the1950s and even before. They cite NASA’s 1957Mercury project, which used half-day increments,and a 1969 report to IBM management that recog-nized “the futility of separating design, evaluation,and documentation processes in software-systemdesign.” Their article includes a quote from GeraldWeinberg, who writes of an experience at about thistime “where the technique used was, as far as I cantell, indistinguishable from XP.”

Barry Boehm and Rich Turner examine the agileversus plan-driven debate and provide recommen-dations for how and when to mix the twoapproaches. In “Using Risk to Balance Agile andPlan-Driven Methods,” they highlight both thehome grounds and risks associated with eachapproach. They describe how they would choosewhich approach to use and under what circum-stances they would mix them, highlighting theirideas with three sample projects that should usedifferent combinations of agile and plan-drivenpractices. They characterize the projects using afive-point radial diagram that isolates importantproject features driving this decision. The diagramalso captures some part of the organization’s abil-ity to undertake change.

Boehm and Turner list three categories of riskthat can affect the choice of how heavily to bias a

June 2003 41

Integrating quality assurance

into projects to enhance their agility

requires makingchanges.

42 Computer

project toward the agile or plan-driven side:

• risks stemming from an agile approach:scalability, criticality, design simplicity,staff churn, and staff skills;

• risks stemming from a plan-drivenapproach: emerging requirements, con-stant change, need for rapid results, andstaff skills; and

• general environmental risks: technologyuncertainties, diverse stakeholders, andcomplex systems.

In listing these risks, they establish a frame-work for examining the project experiences andcharacteristics of various development approaches,a framework echoed in the other three articles.

In “Developing Complex Projects Using XP withExtensions,” Martin Lippert and his coauthorsdescribe how they have dealt with circumstancesencountered on projects using (mostly) ExtremeProgramming. There is an echo of the Boehm-Turner list in this work, even though the authorsarrived at their cyclical approach to software devel-opment independently, based on specific projectexperiences.

The authors discuss how they handled diverse andconflicting interest groups driving a project, lack oftesting on the customer’s side of the equation, com-plex systems and imprecise requirements, long-termplanning, dependencies among nonprogrammingtasks, and the need for requirements definition atleast for funding purposes. It is refreshing that theyidentify the special skills the people in key rolesneed, including the product manager, the author-critic, and even the customer. Their experiences pro-vide additional practices for agile developers toconsider using and demonstrate the benefits of mix-ing selected traditional approaches with standardagile and situationally derived practices.

In “Introducing an Agile Process to an Organi-zation,” Mike Cohn and Doris Ford describe theirexperiences with introducing Scrum into seven orga-nizations over a four-year period and reflect on whatthey learned along the way. They describe watch-ing some developers add formal documents backinto their work practices—“we invariably foundprogrammers who enjoy producing noncode arti-facts far more than they are willing to admit”—andrecall other overzealous developers who view agiledevelopment as meaning they must not think ahead.

In light of the Beck-Boehm debate around disci-pline, Cohn and Ford provide an interesting descrip-tion: “They did not have the discipline XP requires

and, while paying lip service to XP, they were actu-ally doing nothing more than hacking.” Theseauthors discuss their experiences with some Boehm-Turner issues such as distributed development anddescribe other issues that popped up on their pro-jects, such as encountering developer resistance tolightening the process, working with the humanresources department, using testers, and managingthe project and its interfaces to other groups.

Finally, in “Migrating Agile Methods toStandardized Development Practice,” Mark Lycettand his coauthors address a new wrinkle to intro-ducing agile processes into an organization: work-ing in an ISO 9000 or CMMI environment or onein which a large corporation has mandated that allprojects must use a common process. They makeclear the need for a company to establish quality ofproduct, process, and service while still letting dif-ferent projects work in the various ways that suitthem best.

Reporting on their experiences with a large com-pany, these authors describe their particularapproach to project-specific process tailoring: a setof process “patterns” that can be mapped to theCMMI framework. For ISO 9000, they adopted apattern-based framework “to supplant repeatabil-ity with consistency, while providing the audit trailnecessary for assessment.” For good project-specific results, they revisited their use of the pat-terns after each iteration and found that “makingthe decision process visible is extremely valuablebecause it forces management to actively determinewhether an activity or artifact is both necessary andsufficient.” Echoing and extending the first AgileManifesto value, they describe their framework as“suggesting suitable techniques for collaboration,interaction, and communication.”

Lycett and his coauthors offer their own resolu-tion of the agile versus engineering debate: “Agileprocess does not fly in the face of engineering prac-tice. If used thoughtfully, it provides a clear man-date for making engineering practice lean and wellfocused.”

I t is a pleasure to introduce these articles in thisspecial issue. They capture the state of the cur-rent conversation, and we hope they serve as the

catalyst for future ones. �

References1. J. Highsmith, Agile Software Development Ecosys-

tems, Addison-Wesley, 2002.

Working in an environment in

which all projectsmust use a

common processadds a new wrinkle

to introducing agile processes.

2. M. Fowler and J. Highsmith, “The Agile Manifesto,”Software Development, Aug. 2001, pp. 28-32.

3. B.A. Ogunnaike and W.H. Ray, Process Dynamics,Modeling, and Control, Oxford Univ. Press, 1994.

4. K. Schwaber and M. Beedle, Agile Software Devel-opment with SCRUM, Prentice Hall, 2002.

5. B. Boehm, “Get Ready for Agile Methods, withCare,” Computer, Jan. 2002, pp. 64-69.

6. A. Cockburn and J. Highsmith, “Agile SoftwareDevelopment: The People Factor,” Computer, Nov.2001, pp. 131-133.

7. J. Highsmith and A. Cockburn, “Agile SoftwareDevelopment: The Business of Innovation,” Com-puter, Sept. 2001, pp. 120-122.

8. L. Williams, “The XP Programmer: The Few-MinutesProgrammer” (guest editor’s introduction), IEEESoftware, May/June 2003, pp. 16-20.

9. R. Crocker, Large-Scale Agile Software Development,Addison-Wesley, 2003.

10. M. Paulk, “Extreme Programming from a CMM Per-spective,” IEEE Software, Nov./Dec., 2001, pp. 19-26.

Laurie Williams is an assistant professor of com-puter science at North Carolina State University.

Her research interests include agile software devel-opment methodologies and practices, software reli-ability, software testing, and software engineeringfor secure application development. She received aPhD in computer science from the University ofUtah. Williams is the coauthor of Pair Program-ming Illuminated and Extreme Programming Per-spectives (both Addison-Wesley, 2003). She is amember of the IEEE and the ACM. Contact her atwilliams@csc.ncsu.edu.

Alistair Cockburn, a coauthor of the “Manifestofor Agile Software Development,” received a PhDfrom the University of Oslo on the subject of peo-ple and methodologies in software development.He has published widely on formal specification,object-oriented design, and development processes,but is most widely recognized as the author of Writ-ing Effective Use Cases and Agile Software Devel-opment. Many of his publications are availableonline at alistair.cockburn.us. Contact him atacockburn@aol.com.

IEEE Pervasive Computing delivers the latest peer-revieweddevelopments in pervasive, mobile, and ubiquitous computing and acts as a catalyst for realizing the vision of pervasive (orubiquitous) computing, described by Mark Weiser nearly adecade ago. In 2003, look for articles on

• Security & Privacy

• The Human Experience

• Building Systems That Deal with

Uncertainty

• Interfacing with the Physical World

SUBSCRIBE NOW!

http://computer.org/pervasive

M. SatyanarayananCarnegie Mellon Univ. and Intel Research Pittsburgh

Associate EICs

IEEE Pervasive Computing

Editor in Chief

Roy Want, Intel Research; Tim Kindberg, HP Labs; Deborah Estrin, UCLA; Gregory Abowd, GeorgiaTech.;

Nigel Davies, Lancaster University and Arizona University