agile system development methods

Degree Course Business

Information Systems

Agile system development methods

Henning Müller


Information Systems

Overview

• Introduction• Motivations• Some vocabulary

– Agile methods in general

• Response to heavyweight waterfall model– Many small concepts

• Stories of Agile projects


Information Systems

Henning Müller

• Diploma in Medical Informatics– University of Heidelberg (1992-1997)

• Daimler Benz research and technology– Portland, OR, USA (1997-1998)

• PhD in image analysis– University of Geneva (1998-2002)

• Medical image analysis and information systems– University and Hospitals of Geneva (2002-now)

• HES SO (2007-now)


Information Systems

You

• Background– Practical experiences (also concerning agile

methods)

• Expectations from this course

• Brainstorming: What is Agile for you?


Information Systems

Goals

• Obtain a solid knowledge on modern information system development methods– Agile vs. iterative methods vs. waterfall

– Why Agile can help

• Understand how to choose the right methodology for each project– Or components of it

– There is no one size that fits all


Information Systems

Textbooks

• Ken SCHWABER, Mike BEEDLE,Agile Software development with SCRUM, Prentice Hall, 2002.

• Craig LARMAN,Agile and iterative development, Addison-Wesley, 2006.


Information Systems

Definition Agile methods (wikipedia)• Agile software development is a conceptual framework for software

engineering that promotes development iterations throughout the life-cycle of the project.

• There are many agile development methods; most minimize risk by developing software in short amounts of time. Software developed during one unit of time is referred to as an iteration, which may last from one to four weeks. Each iteration is an entire software project: including planning, requirements analysis, design, coding, testing, and documentation. An iteration may not add enough functionality to warrant releasing the product to market but the goal is to have an available release (without bugs) at the end of each iteration. At the end of each iteration, the team re-evaluates project priorities.

• Agile methods emphasize face-to-face communication over written documents. Most agile teams are located in a single open office sometimes referred to as a bullpen. At a minimum, this includes programmers and their "customers" (customers define the product; they may be product managers, business analysts, or the clients). The office may include testers, interaction designers, technical writers, and managers.


Information Systems

Definition iterative/incremental methods• Incremental development is a scheduling and staging strategy in which

the various parts of the system are developed at different times or rates, and integrated as they are completed. It does not imply, require nor preclude iterative development or waterfall development - both of those are rework strategies. The alternative to incremental development is to develop the entire system with a "big bang" integration.

• Iterative development is a rework scheduling strategy in which time is set aside to revise and improve parts of the system. It does not presuppose incremental development, but works very well with it. A typical difference is that the output from an increment is not necessarily subject to further refinement, and its' testing or user feedback is not used as input for revising the plans or specifications of the successive increments. On the contrary, the output from an iteration is examined for modification, and especially for revising the targets of the successive iterations.


Information Systems

Definition waterfall model

• Strict following of phases– No way back

• Was often mandatory– DARPA, CIA,

governments

• Static• No changes foreseen

– Not realistic for most software projects


Information Systems

Critics towards the waterfall model

• Software projects are not as static as many other product developments (scope creep)– Up front specifications may not be realistic

– The final product is often not known at the start

• Risk in the waterfall model is pushed towards the end– Plus “big bang” is risky per se

• A single contact with the customer is not enough


Information Systems

General problems with software development: Exercise

• Read the text “Standish Group Report 1995”, 1-4• What are the most surprising results?

• Read pages 5-7• Which factors are linked with software success?

• Why are so many software projects failing?


Information Systems

Standish Report 2000/2003

• Read the one page abstract

• What has improved? What has not?

• Any propositions or explications?


Information Systems

Exercise:

• Read the text on agile methods and the lack of their use.

• Why is the waterfall model still used so broadly?

• What needs to be done to increase the number of agile development projects?


Information Systems

Product development

• Software vs. other products– Cell phones– Plastic dinosaurs– Houses – architecture

• Predictable vs. managing change• Estimate efforts and costs

– Possible or not?• Plan a schedule, order all activities, details


Information Systems

Predictable/new product manufacturing

• Most software is not a predictable or mass manufacturing problem. Software development is new product development

• Often new, buggy technologies are used, increasing the novelty and unpredictability of processes– How long will this version of Java be supported?


Information Systems

Prevention of detailed up front specifications

• Clients are not sure what exactly they want• Clients have difficulty stating everything• Details will only be revealed during the

development• Details are very complex for people• External forces (actions of competitors,

economic crisis) lead to change– Requirements to act quickly in Internet economy


Information Systems

Iterative development

• Several iterations in sequences constitute the full development cycle– Every iteration is an independent sub (mini)

project– Each iteration has analysis, design,

development, testing– Goal is an iteration release

• Incomplete but working prototype– Software across teams is integrated every

release (subset of full system, no throw away)


Information Systems

Overview of iterative development


Information Systems

Overview of the project cycle


Information Systems

Length of iteration

• Depends on methods– Sometimes fixed (SCRUM), sometimes variable

– Most often recommended is 1-6 weeks

– Depends on total length of the project

• Cutting a project into smaller blocks helps to improve the success rate– Reduce complexity

– Satisfaction in finishing off something


Information Systems

Project success vs. length of project


Information Systems

Risk-driven development

• Reduce overall project risk early in the project• Start each iteration with the riskiest parts

– If there is a real problem it is discovered quickly and can be addressed

• Risk is not always easy to define or estimate– Probability and impact

can be estimated


Information Systems

Risk in the waterfall model


Information Systems

Risk in iterative models


Information Systems

Client-driven development

• Choice of features for next iteration comes from the client directly

• Client steers the development iteration by iteration

• Client has ongoing control of the project and makes most important decisions based on latest data, and not speculatively – Client is working directly with the developers


Information Systems

Timeboxing

• Fix end date for each iteration without change• If too many features for completion, reduce the

scope (improve estimates over time)– Remove features with lower priority

• Always a running and tested system at the end of each iteration

• Entire project can be timeboxed, or only iterations• Iterations can have varying length


Information Systems

Evolutionary and adaptive development

• Requirements, plans and solutions evolve and are refined over time– No fully frozen specification at the start

– Unpredictable discoveries can occur

• System can be adapted based on feedback from users, clients, or developers

• Internet economy is based on quick reaction times and quick changes based on the market


Information Systems

Evolutionary requirements

• This does not mean that everything changes all the time!– No sloppy specifications

• First iterations have often a larger percentage of requirements analysis

• Most requirements are defined early on and iterations have increasingly short analyses

• Requirements workshops in some early iterations for larger projects– Details on risky developments


Information Systems

Cone of uncertainty


Information Systems

Adaptive planning

• Concentrate on high level influential parts– Architecturally important

– High risk (load, usability, …)

• Initial phase of high uncertainty• Make a more detailed planning 10%-20% into

the project– After first developments and experiences

• Initial exploratory phase


Information Systems

Cowboy coding

• Absence of a defined method• Members of the team do what they think is right• No details on communication

• This is not agile development!• Agile development follows methods and

protocols, sometimes very strictly


Information Systems

Prizing

• Short fixed-prize exploratory iterations are possible– Then larger-scale parts with possibility to stop at

any moment

• Software should be produced not documents• Waterfall method is easier to pay for

– But is this realistic (promises)

– Risk is mainly with the customer


Information Systems

Incremental/iterative delivery

• Running product is delivered in several iterations

• Delivery cycle is usually different from the development cycle– For example 6-month delivery cycle having 6-10

short iterations per delivery• Feedback can be integrated directly into each

release• Web environment has many projects like this


Information Systems


Information Systems

Common mistakes

• Project leader X:• Sure, we do not apply the waterfall, everyone

knows that it does not work. We use Method X and are in our first project.

• We are now into it for two months and the use case analysis is nearly finished. Then, we will plan and schedule what will be done in each iteration before the programming will start.


Information Systems

Text on success of agile methods

• Read the text (4 pages)

• What are the characteristics of the described projects?

• What are the main characteristics of Agile development mentioned in the text?

• Why are these methods successful?


Information Systems

The Agile alliance

• http://www.agilealliance.com/• Grouping of people active on Agile development

(founded in 2001)• 4’200 members, many events, articles,

publications available from the web side• Agile manifesto, and definition of main

principles of Agile software development• Definitions described on their web pages


Information Systems

A little bit of history

• Evo – evolutionary project management– 1960s: Evo and first elements of iterations etc.

– 1986: SCRUM

– 1996: eXtreme Programming

– Mid 1990s: tendencies to more lightweight development as result of many failures

– 2001: Agile alliance


Information Systems

The Agile manifesto

• Individuals and interactions over processes and tools

• Working software over comprehensive documentation

• Customer collaboration over contract negotiation

• Responding to change over following a plan


Information Systems

The Agile principles (1)

• 1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

• 2. Welcome changing requirements, even late in the development. Agile processes harness change for the customer’s competitive advantage.

• 3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to a shorter time scale.


Information Systems


• 4. Business people and developers must work together daily throughout the project.

• 5. Build projects around motivated individuals. Give them the environment and support their need, and trust them to get the job done.

• 6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.


Information Systems


• 7. Working software is the primary measure of progress.

• 8. Agile processes promote sustainable development.

• 9. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

• 10. Continuous attention to technical excellence and good design enhances agility.


Information Systems


• 11. Simplicity – the art of maximizing the amount of work not done – is essential.

• 12. The best architectures, requirements, and designs emerge from self-organising teams.

• 13. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.


Information Systems

Suitability of Agile methods• The culture of an organisation must be supportive

of negotiation• People must be trusted• Fewer staff with higher level of competency• Organisation must live with decisions the

developers make– Low level of hierarchies is better

• Organisation needs to have a development that facilitates rapid communication between team members


Information Systems

A little break in the presentation …

• Read the text exploring agile development

• What are the two most important advantages of the waterfall approach and of Agile methods? What are the disadvantages?


Information Systems

So what exactly is Agile?

• Not a single definition• Timeboxed, iterative, incremental development• Change is embraced• Promote evolutionary delivery• Simplicity, lightness, communication• Some methods common to a single practice

– Common project room, standup meetings, pair programming, …


Information Systems

Classification of Agile methods

• Length and number of cycles/iterations– From a few days to months

• Ceremony or strict guidelines to follow– Formal steps, …

• Suitability for particular sorts of projects– Projects with many/few developers

– Projects with a long/short duration


Information Systems

Classification of four examples


Information Systems

Function points to measure project size

• http://www.ifpug.org/• FPU – Function Point analysis

– Estimate size of a software project

– Budget application development

– Determine project productivity

• Complex analysis in closed documents


Information Systems

Agile project management (Augustine et al.)

• Guiding Vision: Establish a guiding vision for the project and continuously reinforce it through words and actions

• Teamwork and collaboration: Facilitate teamwork and collaboration through relationships and community

• Simple rules: Establish and support the team’s set of guiding practices such as SCRUM or XP


Information Systems

Agile project management (2) (Augustine et al.)

• Open Information: Provide visible and open access to project management and other information

• Light touch: Apply just enough control to foster emergent behaviour in a self-directed team

• Agile vigilance: Reinforce the vision, follow or adapt the rules, listen to the people


Information Systems

People matter!

• Programming is human activity– Happy developers work better

• Overwork is not good, reduces concentration and increases errors– Sustainable development is the goal

• Productivity of programmers varies by factor 10– Education and mentoring by others

– Right knowledge is important for the right task


Information Systems

Simplicity

• Low tech – high touch– Using paper cards, white boards, …

• Stand up meetings• Simple web tools

– Wikis etc. are changing the tool world

• Developers should quickly see benefit of simple tools


Information Systems

Empirical vs. defined processes

• Defined processes have well-defined steps or ordered activities– Predictable manufacturing

• Empirical processes in high-change and instable domains– Frequent measurements and response

mechanisms


Information Systems

Test-driven development

• Part of many Agile methods• Exact definition of function tests before

programming often with the customer• Automated tests of almost everything

– Pass or fail, no human intervention

• Tests are re-executed after each build cycle• Test suits grow with the project


Information Systems

Cockburn scale to compare projects

• People– Small

– Large

• Criticality– High

– Low


Information Systems

Agile modelling


Information Systems

Several agile methods

• SCRUM• XP• Adaptive Software Development (ASD)• Dynamic Solutions Delivery Model (DSDM)• Crystal Clear• Feature Driven Development (FDD)• Lean Development• Pragmatic Programming


Information Systems

Story

• Read page 41-47

• Mention all agile principles you encounter as soon as they appear


Information Systems

Motivations for agile projects

• If the waterfall has good results there is no need to change it in an organisation– Do not simply follow fashions but needs!!

• Change is usually motivated by partial failure– Failure has been high in software projects

• Failure can have many faces– Time, budget, wrong product


Information Systems

Facts of change in software projects

• Uncertainty is inherent and inevitable in software projects– Scope

creep


Information Systems

Risk management

• Iterative development has lower risk– Tackle the hardest problems first

• Project can be stopped in case a problem is insurmountable

– Keep a running system with a subset of the functionality and avoid the big bang in the end

• Permanent testing

• IKIWISI – I’ll Know It When I See It


Information Systems

Timeboxing benefits

• Focus increases productivity– Psychological focus (day before vacation)

– 3-week milestone different than 3-month milestone

• People remember slipped dates but not necessarily slipped features

• Need to be realistic with goals– … as presented regularly to customers


Information Systems

Factors of challenged projects


Information Systems

Actual use of requested features


Information Systems

Waterfall

• Made for:– Low-change projects

– Low-novelty projects

– Low-complexity projects

• Complexity overload• DoD in the US changed the waterfall dogma in

1994


Information Systems

Estimation of complexity - Wideband Delphi

• Method to obtain estimations for projects• Have several developers make estimations independently

(sometimes anonymously)– Kickoff together

– Estimation separately

– Meeting to discuss results with a facilitator

• Estimate=(Optimistic+Pessimistic+4*MostLikely)/6• LikelyDeviation= (Pessimistic-Optimistic)/6


Information Systems

Studies on software practices

• Many studies exist:– Less than 5% of source code is actually used

• Software pollution

– Productivity increases when releasing products/prototypes early

– Very successful projects have fewer roles than average


Information Systems

Productivity research

• Small projects are often more productive• Iterative projects are more productive by cutting

a big project into pieces• Timeboxing increases productivity by focussing

the scope• Complexity of a project decreases productivity


Information Systems

Productivity vs. size of projects


Information Systems

Software quality

• Number of defects per function point becomes often larger in large projects

• Tests and evaluations increase software quality– Test early, test often

– Short time between coding and testing can decrease errors

• Early contact with users can help finding errors


Information Systems

Abbreviations

• ASD - Adaptive Software Development• CIA - Central Intelligence Agency• DARPA - Defence Advanced Research Project Agency• DSDM - Dynamic Solutions Delivery Model• FDD - Feature Driven Development • FPU - Function Point Units• IS - Information System• SCRUM - no abbreviation, term from Rugby• SOA - Service Oriented Architectures• SURF - Standish User Research Forum• UML - Unified Modeling Language• XP - eXtreme Programming


Information Systems

Links and references

• http://www.wikipedia.org/• http://www.agilealliance.com/

agile system development methods

Documents