my.fragility · •a must for new product development ... “business involvement in the...

17/11/2009

1

My.FragilityA Vision and Framework on Agile and Scrum

Author:

Date:

Gunther Verheyen

20/10/2009

Presentation Goal

Present my Vision and Experience

on Scrum and Agile

in Software Development Projects

17/11/2009

2

1. Implementation Methods

2. Maturity & Adoption

3. Scrum

4. My.Fragility

5. Q&A

17/11/2009

My.Fragility

3Agenda


17/11/2009

3

1. Ad hoc (a.k.a. Cowboy Coding)

– The absence of a defined method

…but still widely spread

17/11/2009

My.Fragility


• „Traditional‟ methods

– Key characteristics

• Plan-Ceremony driven

• Single pass waterfall (sequential model) Analysis paralysis?

• Success = compliancy with predictive plan

– End = original destination

• Progress = „deliverables‟ (specs/design/code/reviews)

• Resisting/blocking „change‟

17/11/2009

My.Fragility


17/11/2009

4

– Origin

• US-DOD*-2167(A) *DEPARTMENT OF DEFENSE

• A 80‟s DICTATE of contracting constraints

– Area of application

• Defined, predictable, repeatable activities

• Static requirements

• Industrial-like manufacturing Ford-Taylor

17/11/2009

My.Fragility


– Research has shown

• Standish Group („CHAOS reports‟ on >23.000 projects)

– Update 2009: highest failure rate in over a decade

» 44% slipped: late, over budget, less features

» 24% failed: never delivered

– Up to 64% of predicted specs are never or rarely used

– 25-35% specs change during development

• US-DOD (1999 review on application of imposed standards)

– 75% projects failed or never used

– Only 2% concluded without extensive modification

17/11/2009

My.Fragility


17/11/2009

5

• Agile („Lean‟) Software Development

– Key characteristics

• Change-People driven

• Defined process (iterative-incremental)

• Success = delivered Business Value

• Progress = frequent delivery of working software

– Theory of constraints

• „Embrace change‟ (encourage!)

17/11/2009

My.Fragility


– Origin Refer to appendix

• 60‟s – 70‟s: „IID‟ (Iterative-Incremental Development)

• mid 90‟s – now: The definite rise of „Agile‟

– From Scrum and XP to the Agile Manifesto

– Area of application

• Painkiller for „traditional‟ problems

• High change (Complex projects)

– Mind the „people‟

• Immediate ROI tROI

17/11/2009

My.Fragility


17/11/2009

6


1. Technology Adoption Life Cycle Geoffrey Moore

– Early Market

• Technology Enthusiasts & Visionaries

• (very) Early competitive advantage

– CHASM

• Unpredictable length

• For eternity?

17/11/2009

My.Fragility


Technology AdoptionLife Cycle

Visibility

17/11/2009

7

– Bowling Alley

• Pragmatists (early majority)

• Niche markets-products

– Tornado

• Pragmatists (early majority)

• (re-)Generalized product

– Main Street

• Pragmatists (late majority)

• Established market

17/11/2009

My.Fragility


The Bowling Alley

Discontinuity andThe Life Cycle

Low

Low

High

High

– Agile is

• A high-tech service a new paradigm

– Has crossed the Chasm / Entered the Bowling Alley

– Therefore evidence is anecdotal (by definition)

• A must for new product development

– Products in „Bowling Alley‟ & „Tornado‟

– Create business value in chaordic circumstances tROI(changing markets, competitors, technologies)

But… works equally good in more orderly

circumstances

17/11/2009

My.Fragility


17/11/2009

8

2. Gartner Hype Cycle

17/11/2009

My.Fragility


Gartner’s Hype Cycle forApplication Development

(2007)

– Matching the TALC:

• Sliding into the Trough?

– 5-10 years to cross the chasm?

• Agile is more than:

– development only

– (1) method

17/11/2009

My.Fragility


TALC vsHype Cycle

Visibility

17/11/2009

9

– Gartner concludes:• “Through 2008, 90% of agile development efforts will be confined to

departmental or workgroup product development”

• “agile approaches actually have, in many ways, closer control (daily control) of development and offer clearer practices”

• “traditional formalism (e.g. big documents) that have traditionally provided comfort, … have not resulted in improved execution”

• “Only true adoption of the spirit of agility, and consistent use of its practices, will lead to broader and more successful use”

• “Agile projects need smart, disciplined developers”; “sophisticated teams that are not averse to process discipline”

• “already delivering the benefits of fast, accurate delivery of priority application requirements”

• “on-site customer (note: Product Owner) is a key success factor”; “business involvement in the development process”

17/11/2009

My.Fragility


3. CMMI compatibility

• Level 2: “Repeatable”

17/11/2009

My.Fragility


CMMI Scrum My.Fragility

Requirements Management ν ν

Project Planning ν

Project Monitoring and Control ν

Measurement and Analysis ν

Supplier Agreement Management ν

Process and Product Quality Assurance ν

Configuration Management

17/11/2009

10

• level 3: “Defined”

17/11/2009

My.Fragility


CMMI Scrum My.Fragility

Requirements Development ν

Technical Solution

Product Integration (ν)*

Verification ν

Validation ν

Organizational Process Focus (ν)*

Organizational Process Definition (ν)*

Organizational Training

Integrated Project Management ν

Risk Management ν

Decision Analysis and Resolution ν

* Scrum type-C

3. Scrum

17/11/2009

11

• Scrum is…

– A process

• to deliver high business value in the shortest time

– Technology independent

• Demands the definition of „engineering standards‟

– Worldwide the most adopted Agile method

17/11/2009

My.Fragility

213. Scrum

VersionOne Survey Q3 2008Scrum Job Postings Google Searches on “Scrum”

• A diamond tour

– Roles

– Meetings

– Artefacts

– Principles

17/11/2009

My.Fragility

223. Scrum

17/11/2009

12

– Roles Scrum Team

• ScrumMaster

– Facilitator and protector of the team

– Tracks progress and removes impediments

– Owner of practices, meetings and communication

• Product Owner

– (proxy) Business and functional representative

– Tracks value and ROI

– Owner of the Product Backlog and the priorities

• Development Team 5-9 people

– Empowered and self-organizing

– Incrementally develops within engineering standards

– Collective Owner of the code

17/11/2009

My.Fragility

233. Scrum

– Meetings

• Sprint Planning

– Part 1: Product Owner explains User Stories

– Part 2: The Team discusses technicalities, estimates and

commits

• Sprint Review

– Part 1 (Sprint Demo): demo, acceptance and feedback

– Part 2 (Sprint Retrospective): look back to self-improve

• Daily Scrum:

– 15 min daily meeting to share information

» What did I do since the previous Daily Scrum?

» What do I plan to do until the next Daily Scrum?

» What are impediments in my way?

17/11/2009

My.Fragility

243. Scrum

17/11/2009

13

– Artefacts

• Product Backlog

– List of prioritized product features (Epic/User Stories)

– Delta Backlog to register all Product changes

• Sprint Backlog

– Subset of Product Backlog Velocity

– Should be User Stories INVEST

• Burndown Charts

– Cumulative remaining effort at Product and Sprint level

– Observe trend vs target

17/11/2009

My.Fragility

253. Scrum

– Principles

• Open workspace

– Remove all physical barriers

– Use Information Radiators

– Face-to-face communication

» The cost of communication

» The quality of information

• Cross-functional

– All disciplines are part of the Development Team

– All disciplines are available on-site

17/11/2009

My.Fragility

263. Scrum

17/11/2009

14

• Self-organizing

– Empowered, self-adaptive teams

– A continuous exercise of „inspect and adapt‟

– Trust and support

– Incremental design and architecture over large, upfront designs

– Self-inspection via Daily Scrum and Sprint Retrospective

17/11/2009

My.Fragility

273. Scrum

• A defined process

– Vision > Product Backlog• Objectives / ROI Value-Driven Development

– Sprint Planning

– Sprint• Daily Scrum

• Implement User Stories within „engineering standards‟

• Perform all traditional IT tasks daily

– Slack Quality Time

– Sprint Review / Retrospective

17/11/2009

My.Fragility

283. Scrum

17/11/2009

15

4. My.Fragility

• A framework of experience in [3R4Q]

– [R1] Releases

– [R2] Roles

– [R3] Requirements

– [Q] for Quality

17/11/2009

My.Fragility

304. My.Fragility

How 3 R’s lead to Q!

17/11/2009

16

– [R1] Releases

• A Project Life cycle

• Multi-Stage Scalability

– (1+ Stages)

– Scrum Type-C

17/11/2009

My.Fragility

314. My.Fragility

Jeff Sutherland

SprintPlanning Review

Sprint

ScrumDaily

• 1 Stage consists of:

– Pre-game staging Sprint (1)

» Story writing & estimating

» (Agile) Design just enough

» Contracts fixed price-negotiable scope

– Development Sprints (3-4)

» Design=Development; Development=Design

» Incremental architecture

» Engineering standards / Quality practices

– Release Sprint (1)

» Installation and release

» Training

» Documentation (mind the cost)

17/11/2009

My.Fragility

324. My.Fragility

Mike Cohn (“Agile Estimating and Planning”)

17/11/2009

17

– [R2] Roles

• Scrum roles

• Coach introduced by using XP as engineering standards

– Technological director

– Participates in (pair) programming

– Owner of the design philosophy and technical standards

• Guide

– Functional tester

– Writes and executes test scenarios

– Owner of the Kanban issue board

17/11/2009

My.Fragility

334. My.Fragility

– [R3] Requirements

• Initial Product Backlog @ pre-game staging

– User or Epic Stories

– Estimatable

• Sprint Backlog @ development time

– User Stories (essential Use Cases) “INVEST”

• Change registrations in Delta Backlog

• Value Driven Development

17/11/2009

My.Fragility

344. My.Fragility

17/11/2009

18

– [Q] for Quality

• 4 (daily) Quality Loops

– Implementation of „engineering standards‟

• Pair programming roles and rotation

– #Lines Of Code: -20%

– #Defects: -60%

– Elapse time: -45%

17/11/2009

My.Fragility

354. My.Fragility

• Tools

– Product Backlog Estimation model

• Create and estimate Product Backlog

• Set Velocity (at Story level)

– Results in total Story Points and total Planning Time

17/11/2009

My.Fragility

364. My.Fragility

Re

leas

e

Spri

nt

ID Val

ue

Name Description ti tp v (1) Cost Incl

ud

e?

Budget/Story Days/Story

0 X Pre-game staging 15,5 15,5 0,00 10.850,00 € X 18.631,00 € 25,9

1 10 Epic / User Story 1 12,0 15,6 0,30 10.920,00 € X 18.751,20 € 26,1

2 10 Epic / User Story 2 12,0 15,6 0,30 10.920,00 € X 18.751,20 € 26,1

3 10 Epic / User Story 3 12,0 15,6 0,30 10.920,00 € X 18.751,20 € 26,1

4 10 Epic / User Story 4 12,0 15,6 0,30 10.920,00 € X 18.751,20 € 26,1

5 10 Epic / User Story 5 12,0 15,6 0,30 10.920,00 € X 18.751,20 € 26,1

6 10 Epic / User Story 6 12,0 15,6 0,30 10.920,00 € X 18.751,20 € 26,1

7 10 Epic / User Story 7 12,0 15,6 0,30 10.920,00 € X 18.751,20 € 26,1

8 10 Epic / User Story 8 12,0 15,6 0,30 10.920,00 € X 18.751,20 € 26,1

9 10 Epic / User Story 9 12,0 15,6 0,30 10.920,00 € X 18.751,20 € 26,1

10 10 Epic / User Story 10 12,0 15,6 0,30 10.920,00 € X 18.751,20 € 26,1

11 X Release 8,0 8,0 0,00 5.600,00 € X 9.616,00 € 13,4

100 Total development: 143,5 179,5 0,25 75% ! !

Including slack: 224,4 Daily Rate

Development time in weeks (6 pairs): 700,00 € 3,59 6,00 157.062,50 € 700,00€

Total umbrella tasks: 75,4 58.696,50 €

Product Owner (5/5): 750,00 € 18,0 5,00 13.462,50 € 6% 750,00€

Coach (5/5): 750,00 € 18,0 5,00 13.462,50 € 6% 750,00€

Guide (Testing and documentation) (5/5): 700,00 € 18,0 5,00 12.565,00 € 6% 700,00€

ScrumMaster (5/5): 900,00 € 18,0 5,00 16.155,00 € 6% 900,00€

MDI (1/5): 850,00 € 3,6 1,00 3.051,50 € 1% 850,00€

TOTAL: 299,8 215.759,00 € !

Estimated project elapse time 2/01/2009 ########## 719,76 €/d

89,97 €/h

v (2): 25%

Holidays: 17%

Coach development (y/n): Y

//localhost/C/Documents and Settings/verhegu/Local Settings/Temp/B-My.Fragility (Methods Documentation)/The Project Architects - Product Backlog Estimation Model 0.4.xls

17/11/2009

19

• Set other parameters:

– #Pairs

– Daily rate and #FTE per Umbrella task (roles)

– Start date

– (Velocity at process level, Coach development

participation and Normal absences %)

• Add Pre-game staging and Release Sprint

• Assess and iterate

– Results in total Days, Budget and Elapse time

– Additional info holds Overall day price, Role split %,

Detail budget per Story and Total process days per Story

17/11/2009

My.Fragility

374. My.Fragility

– Product Backlog Tracking model

• Create project

– Sprint Plan phasing and planned days

– Umbrella tasks

– Budget

• Create team

– Identify costs

17/11/2009

My.Fragility

384. My.Fragility

17/11/2009

20

• Tracking data

– Daily Sprint tracking Burndown

– Weekly Team tracking (spent effort at Role level)

– Weekly Project tracking (Product) Burndown

» Results in overiew to-date and end-of-project

17/11/2009

My.Fragility

394. My.Fragility

• Cases

– Telenet iDTV Back-end server platform

• JavaEE Platform / Integration hub

• Pilot project: Oct 2003 – Dec 2003 (Apr 2004)

– 920 d - 432 k€

– #User Stories: 59

– Presented as a case at JavaPolis 2003

17/11/2009

My.Fragility

404. My.Fragility

17/11/2009

21

• Commercial launch: Oct 2004 – Jan 2005

– 1.350 d – 617 k€

– #User Stories: 298 (historical total)

» 413 D-registrations

– #LOC: 85.233; #Classes: 1.681

• Consolidation: Jan 2006 – Apr 2006

– 732 d - 400 k€

– #User Stories: 105 (historical total)

17/11/2009

My.Fragility

414. My.Fragility

– www.wwaow.com (Unibind)

• Jun 2006 – Dec 2006

• 670 d – 363 k€

• #User Stories: 158 (historical total)

– www.police-on-web.be

• Oct 2006 – Dec 2006

• 450 d – 225 k€

• #User Stories: 54 (historical total)

17/11/2009

My.Fragility

424. My.Fragility

http://www.wwaow.com/

http://www.police-on-web.be/





17/11/2009

22

5. Q&A

Questions?

Appendix: eXtreme Programming

17/11/2009

23

• XP (eXtreme Programming):

17/11/2009 45Appendix: XP My.Fragility

root-cause analysis

weekly cycle quarterly cycle

shrinking teamspay-per-use

Appendix: references

17/11/2009

24

• Scrum:– Agile Software Development with Scrum –

Mike Beedle, Ken Schwaber

– Agile Project Management with Scrum –Ken Schwaber

– The enterprise and Scrum – Ken Schwaber

• Extreme programming:– Extreme Programming Explained (ed. 1, 2)

– Kent Beck

– Planning Extreme Programming – Kent Beck, Martin Fowler

– Extreme Programming Installed – Ron Jeffries, Ann Anderson, Chet Hendrickson

17/11/2009 47Appendix: References My.Fragility

• Agile & Iterative development:– Agile & Iterative Development – Craig Larman

– Balancing agility & discipline – Barry Boehm, Richard Turner

– User Stories Applied (for agile software development) – Mike Cohn

– Agile Software Development – Allistair Cockburn

– Agile Software Development Ecosystems – Jim Highsmith

• Lean:– The machine that changed the world: the story of lean production –

James Womack, Daniel Jones, Daniel Roos

– Lean software development: an agile toolkit – Mary Poppendieck, Tom Poppendieck

– Implementing Lean software development: from concept to cash – Mary Poppendieck, Tom Poppendieck

• Other:– Crossing the chasm: Marketing and selling technology products to

mainstream customers – Geoffrey Moore

– The chasm companion: A Field Guide to Crossing the Chasm and Inside the Tornado – Paul Wiefels

– The Mythical Man-Month – Frederick P. Brooks

17/11/2009

My.Fragility

48Appendix: References

17/11/2009

25

• Agile Software Development:– www.controlchaos.com (Ken Schwaber)

– www.scrum.org

– www.scrumalliance.org

– www.agilealliance.org

– www.xprogramming.com

– www.pairprogramming.com

– www.agilemodeling.com

– www.agiledata.org

– www.poppendieck.com (Lean Software Development)

– www.mountaingoatsoftware.com (Mike Cohn)

– www.martinfowler.com/articles/newMethodology.html

– www.infoq.com/minibooks/scrum-xp-from-the-trenches

– jeffsutherland.com/scrum/Sutherland2005FutureofScrum20050603.pdf Scrum Type-C!

17/11/2009

My.Fragility


• Open Source Tools:– Versioning of source code with „CVS‟: www.cvshome.org

– Build scripting via „Ant‟: ant.apache.org

– Continuous integration via „CruiseControl‟: cruisecontrol.sourceforge.net

– Unit-testing (Java) source code via „JUnit‟ (and xUnit extensions): www.junit.org

– Mock objects in JUnit tests via „EasyMock‟ and „Spring IOC‟: www.easymock.org, www.springframework.org

– Integration testing of web interfaces via „HttpUnit‟: www.httpunit.org

– Load-testing via „JMeter‟: jakarta.apache.org/jmeter

– Measuring code coverage with „Cobertura‟: cobertura.sourceforge.net or „JBlanket‟: csdl.ics.hawaii.edu/Tools/JBlanket

– „Eclipse‟ as integrated development environment: www.eclipse.org

– Automated GUI testing with Selenium: selenium.openqa.org

17/11/2009

My.Fragility


http://www.controlchaos.com/

http://www.scrumalliance.org/



http://www.agilealliance.org/

http://www.xprogramming.com/

http://www.pairprogramming.com/

http://www.agilemodeling.com/

http://www.agiledata.org/

http://www.poppendieck.com/

http://www.mountaingoatsoftware.com/

http://www.martinfowler.com/articles/newMethodology.html





http://www.infoq.com/minibooks/scrum-xp-from-the-trenches













http://jeffsutherland.com/scrum/Sutherland2005FutureofScrum20050603.pdf



http://www.cvshome.org/

http://www.ant.apache.org/

http://www.ant.apache.org/

http://www.junit.org/

http://www.easymock.org/

http://www.springframework.org/

http://www.httpunit.org/

http://www.jakarta.apache.org/jmeter

http://www.jakarta.apache.org/jmeter

http://www.csdl.ics.hawaii.edu/Tools/JBlanket

http://www.eclipse.org/

http://selenium.openqa.org/

17/11/2009

26

Appendix: A Brief Agile History

Year / Era Name Company / Project Characteristics

1930s W. EdwardsDemming

Bell – Quality Management PDSA- Plan Do Study Act

1950s X-15 hypersonic jet Non-software project applying IID

1958 Weinberg IBM Federal Systems

Division

Project Mercury (NASA)

Software project applying IID

½ day iterations (miterations)

Test First

Top-down dev with stubs

1968 Brian Randell

F.W. Zurcher

IBM Research

Report “Iterative multi-

level modeling: A

methodolgy for computer system design”

1970 Winston Royce Article “Managing the

development of largedevelopment systems”

Birth of Waterfall (despite advise

for 2 iterations „version

delivered‟ … „actually 2nd version‟)

1970 Harlan Mills IBM FSD

Article “Top down

programming in largesystems”

Suggestion of iterativerefinement

17/11/2009 52Appendix: A Brief Agile History My.Fragility

17/11/2009

27


1972 O‟Neill

(Dyer – McHenry)

IBM FSD

DoD USD Trident Submarine

US Department of Defense (DOD)

106 lines of code

4 iterations of 6 months

1972 TRW

DoD Ballistics missiledefense

5 iterations of several months

(but feedback driven refinements)

Mid 1970s Harlan Mills IBM FSD

DoD Light AirborneMultipurpose System

45 iterations of 1 month

1975 Victor Basili

Joe Turner

IEEE Paper “Iterative

enhancement: a practical

technique for software development”

First original paper devoted to IID

Base: 17 iterations over 20 months

1975 Fred Brooks Book “The Mythical Month”

1976 Tom Gilb Book “Software metrics” Focus on evolutionary delivery

1976 Harlan Mills IBM FSD

IEEE Paper “Software development”



1977 McHenry IBM FSD

Standardization on“integration engineering”

Trident approach (1972)

1977

(-1980)

NASA Space Shuttle software

17 iterations of 8 weeks (over 31 months)

1978 Tom Gilb „Computer weekly‟ columns

IID / Evolutionary project management

1980s DoD-Std-2167 Standards and governement

contracting constraints

Single-pass waterfall

1980s Artificial intelligenceprojects

Evolutionary prototyping

1985 Barry Boehm Spiral Model Method Risk-driven iterations

1986 Fred Brooks Paper “No silver bullet”

1986 Takeuchi and

Nonaka

Ken Schwaber

“The new new product

development game”

Scrum

Sashimi idea inherited by Scrum

30d iterations


17/11/2009

28


1987 TRW

Aviation project

“Command center

processing and display replacement system”

4-year project

6 iterations of 6 months

1987 Harlan Mills - Dyer IBM FSD

Cleanroom method

Evolutionary development

1988 DoD-Std-2167A Replacement of DoD-Std-2167

1988 Tom Gilb Book “Principles of

software engineering

management”

Evo method

Re-iteration and expansion of

“Software metrics” (1976)

1 Week iterations / quantifiable-measurable goals

1992 Philippe Kruchten Canadian Air TrafficControl System (1989)

Restart of catastrophy 2167A

Later evolved to RUP

1994 MIL-STD-498 Replacement of DoD-Std-2167A

“Removing the Waterfall Bias”

“Joint technical and management

reviews over „non-productive‟ formal reviews and audits”

1994 James Martin RAD standards Finally resulting in DSDM



1995 Ken Schwaber Oopsla workshop The „definite‟ Scrum

1996 Kent Beck C3-project Focus on communication,

simplicity and testing

Finally resulting in XP

1997 Peter Coad

Jeff De Luca

Large logistics system

(Singapore) from waterfallto IID

Finally resulting in Feature

Driven Development

1998 Standish Group 23.000 projects failure analysis

(refer to our presentation)

2000 DoD 5000.2 Replacement of MIL-STD-498

2001 17 process experts Utah ski resort Agile Alliance & Agile Manifesto

2002 Alistair Cockburn Agile Software Development

First book to use the brand „Agile‟

2002 Jim Highsmith Agile Software Development Ecosystems

Series by Cockburn-Highsmith

2002 Ken Schwaber Scrum Alliance Non-profit organization to

manage certications, courses,

articles and events


17/11/2009

29


2002 Ken Schwaber, Mike

Beedle

Agile Software

Development with Scrum

Original book on Scrum (first

method description)

2003 Craig Larman Agile & iterative

development: a manager‟sguide

Series by Cockburn-Highsmith

2004 Kent Beck eXtreme Programming

Explained (ed.2)

Kent Beck revises his milestone

book

2005 Jeff Sutherland The Future of Scrum Paper describing Scrum Type-A, B

and C

2005 Mike Cohn Agile Estimating and

Planning

Milestone book on the Agile

estimation process

17/11/2009

My.Fragility

57Appendix: A Brief Agile History

Thanks