a study in agile development - university of birmingham...the best architectures, requirements, and...

Slide 1

A Study in Agile Development

Birmingham University

Commercial Programming Lectures

Tuesday, 29 November 2011

Slide 2

Agenda

1. Introductions

2. Agile fundamentals

3. A large scale commercial Agile development project

4. Getting the Agile basics done well

5. More advanced Agile adoption

6. Agile in the marketplace

1. Introductions

Slide 3

Richard Hilsley...

Slide 4

• Senior Technology Manager at Bank of America Merrill Lynch (BAML)

• In charge of a large development programme building a new global banking platform: 3 year programme, $multi-million, 60 person global team

Background:

• MEng Electronic Engineering, Southampton University, 1996

• I was a techy …C, C++, Mainframe, VB, Java & web (9 yrs)

• Since then, roles in team leading, Project & Program management, Architecture, Consulting, Sales…

• Broad cross-sector experience: retail, healthcare, insurance, new media & entertainment, insurance, banking

• Keen interest in Methodologies, especially Agile for the last 8 years

• richard.hilsley@baml.com

All comments made in the presentation are the opinion of the presenters and do not

represent any opinion from the Bank of America Merrill Lynch

Bank of America Merrill Lynch...

Slide 5

• One of the world's largest financial institutions

• Serving individual consumers, small- and middle-market businesses and

large corporations

• Biggest Consumer Bank in US (1 in 2 US Households)

• Major supplier of Home Loan and Insurance products in US

• Biggest Credit Card Provider in Europe and 2nd biggest in US

• Has a presence in over 40 countries

• Approx 300,000 employees

• Technology & Operations ~100,000 people!

IBM, Oracle, Microsoft, Google. . . . . BAML

2. Agile Fundamentals

Slide 6

Agile Manifesto – The Four Guiding Value Statements

Slide 7

We are uncovering better ways of developing

software by doing it and helping others do it.

Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on

the right, we value the items on the left more.

Kent Beck

Mike Beedle

Arie van Bennekum

Alistair Cockburn

Ward Cunningham

Martin Fowler

James Grenning

Jim Highsmith

Andrew Hunt

Ron Jeffries

Jon Kern

Brian Marick

Robert C. Martin

Steve Mellor

Ken Schwaber

Jeff Sutherland

Dave Thomas

© 2001, the above authors

this declaration may be freely copied in any form,

but only in its entirety through this notice.

The 12 Agile Development Principles

Slide 8

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Welcome changing requirements, even late in development. Agile processes harness change for

the customer's competitive advantage.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the

shorter timescale.

Business people and developers must work together daily throughout the project.

Build projects around motivated individuals. Give them the environment and support they need,

and trust them to get the job done.

The most efficient and effective method of conveying information to and within a development

team is face-to-face conversation.

Working software is the primary measure of progress.

Agile processes promote sustainable development. The sponsors, developers, and users should be able to

maintain a constant pace indefinitely.

Continuous attention to technical excellence and good design enhances agility.

Simplicity--the art of maximizing the amount of work not done--is essential.

The best architectures, requirements, and designs emerge from self-organizing teams.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts

its behavior accordingly.

Agile is...

• Agile is a set of software development best practices that allow for rapid

delivery of high-quality software

•Test-driven development; test automation

•Continuous code improvement

• Agile is a project management process that encourages frequent

inspection and adaptation

•Iterative development

•Release early, release often (e.g. every 2 weeks)

•„Just-in-time‟ requirements definition - teams respond quickly to changing

business requirements.

• Agile fosters strong team work

•“People” over “Process”

•…but this is also a challenge as it is a shift in the way people have traditionally

worked in software engineering

Slide 9

Agile lifecycle

• Incrementally deliver versions of the solution.

• At each iteration, design modifications are made and new functional capabilities

(stories) added, informed from learning of previous development and usage

• Constant feedback cycle; highly collaborative

• Focussed on delivering Business Value early – “release early, release often”

Slide 10

Improvements over Waterfall

Slide 11

• Waterfall is linear & process driven. Work cascades down the stages, handled by

separate teams

• BUT Waterfall does not cope with uncertainty. It requires complete closure of a

stage before moving to next

• YET software development is inherently uncertain:

– Many of the [system's] details only become known to us as we progress in the

[system's] implementation. Some of the things that we learn invalidate our design and

we must backtrack1

Agile development seeks to remedy this

1 David Parnas, early pioneer of Software Engineering

Stories, Story points, Velocity, Burn-downs...??

... Are all Agile terms & metrics for requirements

management & progress reporting

Slide 12

Stories and Story Points

Slide 13

34

1 2 3 5 8 13 21 34 55 89 144 233 377 ....

13

55

5

89

Story „Backlog‟

Fibonacci series

• Story points provide a means for estimating the

relative size and complexity of stories without

creating the implied precision that estimates

given in hours would give

• The Project Team and Project Stakeholders

should understand the lack of precision that is

possible when estimating requirements,

regardless of the format

• Normalizing the estimation of requirements is

critical to tracking team metrics during the

project, and to estimating and prioritizing

features

Story (requirement)

-100

0

100

200

300

400

500

600

700

800

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Sto

ry P

oin

ts

Iteration

Burn-Up and Burn-Down

Remaining Story Points

Completed Story Points Cumulative

Total Story Points (Scope)

Linear (Remaining Story Points)

Burn Charts (Progress)

14

Green Line representsthe estimated story backlog which has been established during iteration

Red Line representsthe cumulative delivery of working functionality (stories)

Blue Line represents

the remaining

estimated story

backlog

3. A large scale commercial Agile

development project

Slide 15

Global Liquidity Platform

“A strategic platform to enhance the delivery of its global

liquidity product offering. The new Global Liquidity

Platform is a centralized technology hub that enables the

company to provide consistent, seamless and integrated

liquidity solutions to clients around the world.”

http://www.finextra.com/news/announcement.aspx?pressreleasei

d=34704

16

Project background & why Agile

• A strategic new platform

– multi-year

– substantial $ investment

– Market differentiating

• Being a new platform, solution is not crystal clear, problems are not

predictable

– We can embrace change

– Deliver market-aligned products

• Business see results faster in the form of working versions of

software

– Frequent releases, faster products to market

– Future investment more assured

17

Technology Challenges

• New platform, „new‟ technology in legacy dept with prevailing „slow‟

processes

• Business want everything yesterday! AND they want high quality

• How many people? >> Double it!!

• Lack of domain knowledge

• Lack of Agile know-how

Slide 18

4. Getting the Agile basics done well

Slide 19

lessons learned

1. Write good stories, especially good quality acceptance

criteria

Stories are the currency of engagement with the business so if we get it

wrong, we can lose our audience or build the wrong product.

Story acceptance criteria drive the testing activity so a poorly written

story can create a lot of defect rework later on

20

lessons learned

2. Done must mean DONE

This is about confidence that each new story is properly delivered

(coded and fully tested), which all too often had not been happening.

It is vital as it drives the burn-up charts which is the only true measure

of progress (working software)

21

lessons learned

3. You need some basic Engineering Practices in place at

the start

Continuous Integration

Pair Programming

Domain Driven Design

Test Driven Development

Refactoring

You need these in order to become Agile (flexible) in the first place -

retrofitting is hard.

22

lessons learned

4. Effective communication

Agile is all about frequent communication (people over process) but is a

culture that needs fostering.

Despite the apparent simplicity of agile principles, continuous re-

enforcement & coaching to teams is necessary

23

5. More advanced Agile adoption

Slide 24

Team room

Slide 25

In a recent survey of agile projects, success rates for teams located in the same

room were over 20% higher than those for geographically distributed teams

Scott Ambler, Has Agile Peaked?, Dr. Dobbs Journal, June 2008

Kanban story wall

Slide 26

Nice reports

Slide 27

Delivery pipeline reports

Slide 28

Blockage charts

Slide 29

Continuous Delivery Pipeline

Slide 30

Automate

Everything!!

Continuous Delivery Tools (e.g. Hudson)

Slide 31

Quality Analysis tools (e.g. Sonar)

Slide 32

6. Agile in the Marketplace

Slide 33

Industry research says

Slide 34

Forrester Research‟s “Forrester Wave Q2 2010” says, “Agile

rapidly becoming the norm” for software development:

Lots of noise & rhetoric in the Agile community!

Slide 35

Extreme Programming (XP)

scrum Iterative

Agile Unified Process (AUP)

Kanban

Cleanroom

lean

RAD Rational Unified Process (RUP)

Amazon book search!

Slide 36

# books

time1980 1990 2000 2010

?3,979

1,224

Oct 1985

C++ first

commercial

release

Early 2001

Agile manifesto

published

“C++” “Agile”

Project Topics

1. Large scale software projects can have many technical Enterprise dependencies. What patterns & practices are needed in introducing a continuous delivery pipeline in this type of environment?– Critique the assumption that people processes are less important than

technical rigour/automation

– Describe the types of tools required to facilitate a Continuous Delivery pipeline (assuming a software project written in the language of your choice, e.g. Java, C#)

2. Compare & contrast iterative methodologies RUP, Scrum & Kanbanand comment on their relative strengths, weaknesses and suitability for commercial application.

References:http://agilemanifesto.org (the underlying principles of Agile by its Inventors)

http://www-01.ibm.com/software/awdtools/rup/

http://www.scrum.org/

http://www.kanbanblog.com/explained/index.html

http://continuousdelivery.com

Book: Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation

Slide 37

Slide 38

Further reading

Agile Software Development

• Head First Software Development, Dan Pilone & Russ Miles, 2008 (Overview Book)

• The Pragmatic Programmer: From Journeyman to Master, Andy Hunt & Dave Thomas, 1999

• Extreme Programming Explained: Embrace Change 2nd Edition, Kent Beck & Cynthia Andres, 2004

User Stories

• User Stories Applied: For Agile Software Development, Mike Cohn, 2004

Agile Estimation & Planning

• Agile Estimating & Planning, Mike Cohn, 2005

Test Driven Development

• Test Driven Development: By Example, Kent Beck, 2002

• Pragmatic Unit Testing (NUnit and JUnit editions), Andy Hunt et al, 2004-2007

• xUnit Test Patterns: Refactoring Test Code, Gerard Meszaros, 2007

Refactoring

• Refactoring: Improving the Design of Existing Code, Martin Fowler et al, 1999