An Analysis of the History of Classical An Analysis of the History of Classical Software Development and Agile Software Development and Agile

DevelopmentDevelopment

Li JiangLi Jiang

School of Computer Science, The School of Computer Science, The University of Adelaide, Australia University of Adelaide, Australia

Armin Eberlein Armin Eberlein

Computer Engineering Department, Computer Engineering Department, American University of Sharjah, UAE American University of Sharjah, UAE

OutlineOutline

Introduction Historical Analysis of the Practices Used in Classical SE and

Agile Development Historical Analysis of the Principles used in Classical SE and

Agile Development Historical Analysis of the Technological Context of Classical

SE and Agile methods Historical Analysis of the Technological Context of Waterfall model Historical Analysis of the Technological Context of SASD Historical Analysis of the Technological Context of Agile Development

Conclusion and Future Work

Introduction (1)Introduction (1) Significant effort has been invested over the last decades in Significant effort has been invested over the last decades in

identifying good practices, models and methods for efficient identifying good practices, models and methods for efficient software development. software development. A large number of methodologies has been developed :A large number of methodologies has been developed :

1) Classical SE methodologies: often referred to as heavy-weight or plan-1) Classical SE methodologies: often referred to as heavy-weight or plan-driven which require upfront requirements definition e .g. waterfall driven which require upfront requirements definition e .g. waterfall model and the spiral model, Larger frameworks, such as the CMM is model and the spiral model, Larger frameworks, such as the CMM is generally considered in this category, generally considered in this category,

2) Agile methodologies: They are often called light-weight or agile. This 2) Agile methodologies: They are often called light-weight or agile. This category includes e.g., (Extreme Programming) XP and Scrum, which category includes e.g., (Extreme Programming) XP and Scrum, which follow the 12 agile principles follow the 12 agile principles

These two categories appear to have conflicting ideas and are These two categories appear to have conflicting ideas and are supported by two groups of proponents that have rarely supported by two groups of proponents that have rarely engaged in constructive discussions. engaged in constructive discussions.

Introduction (2)Introduction (2) With the continuous growth of size and complexity of software With the continuous growth of size and complexity of software

applications, it is very necessary to systematically examine the applications, it is very necessary to systematically examine the differences and relationships between the two categories and differences and relationships between the two categories and their associated development philosophies. their associated development philosophies.

Our initial research shows that there are multi-dimensional Our initial research shows that there are multi-dimensional relationships between SE methodologies. relationships between SE methodologies. We have proposed a five dimensional framework to help analyse the We have proposed a five dimensional framework to help analyse the

relationshipsrelationships Historical, Historical, Contextual, Contextual, Analysis by Analogy, Analysis by Analogy, Phenomenological, and Phenomenological, and Linguistic Linguistic

Focus of the paper

Historical Analysis of the Historical Analysis of the PracticesPractices Used in Used in Classical SE and Agile Development (1)Classical SE and Agile Development (1)

Agile methodologies has its root in traditional engineering Agile methodologies has its root in traditional engineering disciplines disciplines The terms “agile” and “agility” can be traced back to the manufacturing The terms “agile” and “agility” can be traced back to the manufacturing

industryindustry ““lean development” emerged in manufacturing in 1991 with the aim of lean development” emerged in manufacturing in 1991 with the aim of

eliminating waste, amplifying learning, delivering as fast as possible and eliminating waste, amplifying learning, delivering as fast as possible and empowering teams empowering teams

the term “agile manufacturing” was even coined around that time. It the term “agile manufacturing” was even coined around that time. It therefore appears that the. therefore appears that the.

The idea of iterative and incremental development used in most agile The idea of iterative and incremental development used in most agile process models can be found as early as the 1930s process models can be found as early as the 1930s a quality expert at Bell Labs used this practice to improve product quality. a quality expert at Bell Labs used this practice to improve product quality.

Historical Analysis of the Historical Analysis of the PracticesPractices Used in Used in Classical SE and Agile Development (2)Classical SE and Agile Development (2)

A lot of Agile practices can be mapped to traditional software A lot of Agile practices can be mapped to traditional software engineering practices or other engineering practices. engineering practices or other engineering practices. Gladden and Gilb have proposed the practice of “delivering working Gladden and Gilb have proposed the practice of “delivering working

software early” in the early 1980s to address the issue of late delivery of software early” in the early 1980s to address the issue of late delivery of software products resulting in customer dissatisfaction. software products resulting in customer dissatisfaction. These practices can be directly mapped to the agile practice “frequent These practices can be directly mapped to the agile practice “frequent

delivery” and “continuous integration” delivery” and “continuous integration”

As one of the principles in agile modelling, the idea of using a prototype As one of the principles in agile modelling, the idea of using a prototype of working software as the primary measure of progress can be found in of working software as the primary measure of progress can be found in manufacturing as early as 1982 manufacturing as early as 1982 Gladden stated that a physical object conveys more information than a Gladden stated that a physical object conveys more information than a

written specification. written specification. Prototyping has been extensively used in classical SE for feasibility Prototyping has been extensively used in classical SE for feasibility

studies and elicitation of requirements. For instance, Rapid Application studies and elicitation of requirements. For instance, Rapid Application Development (RAD) is a development approach that is based on this Development (RAD) is a development approach that is based on this practice.practice.

Historical Analysis of the Historical Analysis of the PracticesPractices Used in Used in Classical SE and Agile Development (2)Classical SE and Agile Development (2)

A lot of Agile practices can be mapped to traditional software A lot of Agile practices can be mapped to traditional software engineering practices or other engineering practices. engineering practices or other engineering practices. Joint Application Design (JAD) is a practice used in IBM Research Labs Joint Application Design (JAD) is a practice used in IBM Research Labs

in Toronto in the late 1970’s. The fundamental objective of this practice in Toronto in the late 1970’s. The fundamental objective of this practice is to get quality business requirements through active participation of is to get quality business requirements through active participation of stakeholders. Some ideas used in JAD are similar to stakeholder stakeholders. Some ideas used in JAD are similar to stakeholder collaboration used in Dynamic Systems Development Method (DSDM) collaboration used in Dynamic Systems Development Method (DSDM) [20], another representative of agile methodologies. [20], another representative of agile methodologies.

The importance of having a work environment that helps improve The importance of having a work environment that helps improve communication among team members has been discussed by Weinberg in communication among team members has been discussed by Weinberg in 1971. 1971. He argued that face-to-face communication can help exchange information He argued that face-to-face communication can help exchange information

efficiently. The practice in XP of having an “open workspace” also efficiently. The practice in XP of having an “open workspace” also emphasizes the importance of face-to-face communication.emphasizes the importance of face-to-face communication.    

Historical Analysis of the Principles used in Historical Analysis of the Principles used in Classical SE and Agile Development (1)Classical SE and Agile Development (1) A lot of principles proposed in the traditional software process A lot of principles proposed in the traditional software process

model share the same or similar values with the principles of model share the same or similar values with the principles of Agile, e.g.Agile, e.g.

Waterfall model: Waterfall model:

A designer “must communicate with A designer “must communicate with interfacing designers, with his interfacing designers, with his management and the customers” management and the customers” and needs to “maintain customer and needs to “maintain customer involvement in specification and involvement in specification and certification”. certification”.

Agile methodsAgile methods

““Active user involvement” and Active user involvement” and “stakeholder collaboration” in “stakeholder collaboration” in DSDM , and “On-site customer” DSDM , and “On-site customer” in XP. in XP.

““using better and competent using better and competent people in software people in software development” is one of the development” is one of the prominent principles of agileprominent principles of agile

Shares the same values with the practices of

Many principles in a lot of classical Many principles in a lot of classical SE literature, SE literature, e.g. Prof. Dijkstra addressed the e.g. Prof. Dijkstra addressed the exact issue in his Turing award exact issue in his Turing award presentation in 1972: The Humble presentation in 1972: The Humble Programmer. Programmer.

Historical Analysis of the Principles used in Historical Analysis of the Principles used in Classical SE and Agile Development (2)Classical SE and Agile Development (2)

Pair programming [XP]Pair programming [XP]Inspections [FDD]Inspections [FDD]

Regular build schedule [FDD]Planning game [XP]

Inspect codeInspect code

Examples of Classical Software Examples of Classical Software Engineering PracticesEngineering Practices

Software Engineering Software Engineering PrinciplesPrinciples

Examples of Agile Examples of Agile PracticesPractices

Peer review of individual Peer review of individual work [C]work [C]

Code should be subjected Code should be subjected to a simple visual scan by to a simple visual scan by a second party who was a second party who was not involved in writing not involved in writing the original code [Wthe original code [W]

Determination of plans [S]Determination of plans [S]

Identification and resolution of Identification and resolution of risks during the entire software risks during the entire software development process [S]development process [S]

Revise incremental plans for Revise incremental plans for new requirements and respond new requirements and respond to schedule and budget changes to schedule and budget changes [C][C]

Risk Risk managementmanagement

Identify and Identify and manage manage uncertaintyuncertainty

Historical Analysis of the Technological Historical Analysis of the Technological Context of Classical SE Context of Classical SE

Technological context of the Waterfall ModelTechnological context of the Waterfall Model The programming languages were inefficient and no document management systems were The programming languages were inefficient and no document management systems were

availableavailable Hardware consisted of mainframe computers with slow CPUs, very limited memory, and Hardware consisted of mainframe computers with slow CPUs, very limited memory, and

electronic storage space was expensiveelectronic storage space was expensive

Project type and knowledge of SE:Project type and knowledge of SE: large, long-term projects , big team, and limited understanding of the nature of software large, long-term projects , big team, and limited understanding of the nature of software

engineeringengineering

Project management strategies to deal with the issue Project management strategies to deal with the issue well-defined development phases and some basic SE principles well-defined development phases and some basic SE principles Requirements are contracted, and are not supposed to change very oftenRequirements are contracted, and are not supposed to change very often

Technical writers would be hired to handle the large amount of documentation required Technical writers would be hired to handle the large amount of documentation required for the project. for the project.

Having well-defined requirements documents was an important means for effective Having well-defined requirements documents was an important means for effective communication among developers in big teams.communication among developers in big teams.

A librarian would keep track of code libraries and other paper documents It isA librarian would keep track of code libraries and other paper documents It is

Thus, Waterfall is considered suitable within the technological Thus, Waterfall is considered suitable within the technological context context

Historical Analysis of the Technological Historical Analysis of the Technological Context of SASDContext of SASD

Thus, SASD is considered suitable within the technological context Thus, SASD is considered suitable within the technological context

SE methodologies Technological Context Type of projects• Structured

programming

• Modularization & information hiding

• Languages: third generation language, e.g., C, FORTRAN, COBOL

• Tools:

oCompilers run on microcomputer

oUnix system, DOS

o Some text editors for program languages

o Some tools provided isolated support for single activity, like editing programs, debugging

• Microcomputers and workstations are available

• Storage space is still very limited

• Other hardware facilities are available such as various monitor, printers, interfaces, etc.

• Software projects increase size and complexity.

• Information systems increase size and complexity.

• Computer games

• Large number of software projects of medium size and complexity

1970-1979

Historical Analysis of the Technological Historical Analysis of the Technological Context of Agile DevelopmentContext of Agile Development

Technological context of the Agile MethodsTechnological context of the Agile Methods Very powerful computer and large amount of storage space are easily available Very powerful computer and large amount of storage space are easily available Network facilities offer tremendous capabilities of solving both technical problems and Network facilities offer tremendous capabilities of solving both technical problems and

communication problems. communication problems. Very powerful OO languages, such as C++, Java and J2SE techniques Very powerful OO languages, such as C++, Java and J2SE techniques Internet and Web technologies are widely availableInternet and Web technologies are widely available Visual programming technologies and highly interactive GUI and IDEs help design Visual programming technologies and highly interactive GUI and IDEs help design

interactive user interfaces and allow the fast implementation of prototypesinteractive user interfaces and allow the fast implementation of prototypes .. Project type and knowledge of SE:Project type and knowledge of SE:

Large portion of projects are small, short-term projects, and the size of most of software Large portion of projects are small, short-term projects, and the size of most of software companies are small or with small teams dealing with different small to medium projectscompanies are small or with small teams dealing with different small to medium projects

Unclear or unavailable requirements and high constraints of budget and pressure of time Unclear or unavailable requirements and high constraints of budget and pressure of time to market are common characteristics of large portion of projects to market are common characteristics of large portion of projects

Overall, the team has better knowledge of software engineering and the developer usually Overall, the team has better knowledge of software engineering and the developer usually get better training in software development.get better training in software development.

Thus, Agile is considered suitable within the technological context of many projects Thus, Agile is considered suitable within the technological context of many projects

Conclusion and Future Work (1)Conclusion and Future Work (1)

ConclusionConclusion All SE methodologies still have significant limitations All SE methodologies still have significant limitations It is worth to investigate the origin of these methodologies, the philosophy behind the It is worth to investigate the origin of these methodologies, the philosophy behind the

methodologies, and the technologies and social environment which supported the methodologies, and the technologies and social environment which supported the generation of these methodologies. generation of these methodologies.

Research into the historical links between classical SE methodologies and agile Research into the historical links between classical SE methodologies and agile methodologies can help us understand the relationships between the practices in the two methodologies can help us understand the relationships between the practices in the two development philosophies. development philosophies.

There is significant evidence that practices used in SE methodologies and in There is significant evidence that practices used in SE methodologies and in agile methodologies have historical links. agile methodologies have historical links.

Moreover, many practices in both approaches have roots in other Moreover, many practices in both approaches have roots in other  disciplines as well as in traditional engineering disciplines  disciplines as well as in traditional engineering disciplines

An analysis of the fundamental principles behind some practices used in agile An analysis of the fundamental principles behind some practices used in agile methodologies shows that the same principles are used in the practices of classical SE methodologies shows that the same principles are used in the practices of classical SE methodologies, methodologies,

The technological context has a significant impact on the emergence, evolution, use and The technological context has a significant impact on the emergence, evolution, use and change of methodologies. change of methodologies.

Conclusion and Future Work (2)Conclusion and Future Work (2)

Future workFuture work Analysis of Analysis of Classical SE and Agile methods from other dimensions to Classical SE and Agile methods from other dimensions to

explore inherent complementary relationship between the methodologies in explore inherent complementary relationship between the methodologies in the two streams:the two streams: Contextual, Contextual, Analysis by Analogy, Analysis by Analogy, Phenomenological, and Phenomenological, and LinguisticLinguistic

Provide general guidelines for using methodologies in the two streams Provide general guidelines for using methodologies in the two streams based on the results of the research discussed above. based on the results of the research discussed above.

Thank you!Thank you!