agile productivity metrics - stickyminds · bio presentation better software conference june 26 –...

BIO PRESENTATION

Better Software Conference June 26 – 29, 2006 Las Vegas, NV USA

WK2

6/28/2006 10:00 AM

AGILE PRODUCTIVITY METRICS

Michael Mah QSM Associates, Inc.

Michael Mah Michael Mah is a contributing author of IT Measurement, Advice from the Experts and the upcoming book, Optimal Friction, People Dynamics at Work in the Information Age. Michael also publishes his writing on-line through the Cutter Consortium. His areas of expertise include organizational development, IT negotiation, software project estimation, productivity benchmarking, outsourcing, risk management, and project “runway prevention.” Michael has been a keynote and featured speaker for the Carnegie Mellon Software Engineering Process Group Conference, the Better Software Conference & EXPO, the Cutter Summit series, and numerous Project Management Institute and SEI SPIN chapter meetings.

4/19/2006 1Copyright QSM Associates, Inc.

Michael MahManaging Partner

QSM Associates, Inc.75 South Church Street

Pittsfield, MA 01201413-499-0988

Fax 413-447-7322e-mail: michael.mah@qsma.com

Website: www.qsma.comBlog: www.optimalfriction.com

Better Software Conference & ExpoJune 26 - 29, 2006

Agile Productivity Metrics:“XP and Productivity Measures – What

the Numbers Say”

4/19/2006 2Copyright QSM Associates, Inc.

Manifesto for Agile Software DevelopmentWe are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools working software over comprehensive documentation Customer collaboration over contract negotiation responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more.

© 2001 Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin, Steve Mellor, Ken Schwaber, Jeff Sutherland, Dave Thomas, Martin Fowler

4/19/2006 3Copyright QSM Associates, Inc.

“Frothy eloquence neither convinces nor satisfies me. I am from Missouri. You have got to show me.”

- Missouri Congressman Willard Duncan Vandiver, 1899

4/19/2006 4Copyright QSM Associates, Inc.

4/19/2006 5Copyright QSM Associates, Inc.

Industry Data from the QSM SLIM-Metrics Database

Spans 20+ yearsLarge heterogeneous databasecontains over 7,000+ projectsRepresents over 685+ million SLOC, 7+ million function points, over 600 languages, from 500+ organizations in 18 countriesAdding 200 – 400 projects/year

4/19/2006 6Copyright QSM Associates, Inc.

4/19/2006 7Copyright QSM Associates, Inc.

3 - DETERMINE PROCESSMETRICS & PROJECT POSITIONING

1 - COLLECT ANDVALIDATE PROJECT DATA

2 - ANALYZE PROJECTS USINGQSM REFERENCE DATABASE

4 - DOCUMENT RESULTS

Overview of Database

Number of Projects vs Division

Division

Desktop Faxes

High End Systems

Mobile Coms

Number of Projects0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5

5.0

3.0

3.0

Avg, Min, Max PI Grade vs Division

Division

Desktop Faxes

High End Systems

Mobile Coms

Number of Projects0 5 10 15 20 25 30 35 40

Average Value of Metrics

New Percent SLOC

Modified % SLOC

Unmodified % SLOC

Number of Projects0 5 10 15 20 25 30 35 40 45 50 55 60 65

56

12

35

Life Cycle Effort vs ESLOC

ESLOC (thousands)1 10 100 1000

Life Effort (M

HR

) (thousands)

0.01

0.1

1

10

100

ALL Systems Fax Corp 1997base Avg.

Agile Measurement Approach

4/19/2006 8Copyright QSM Associates, Inc.

“We’ve got to make our deadline – or kill our kids,”Sam told his team.

4/19/2006 9Copyright QSM Associates, Inc.

4/19/2006 10Copyright QSM Associates, Inc.

Industrial XP Environment

4/19/2006 11Copyright QSM Associates, Inc.

Industrial XP Environment

4/19/2006 12Copyright QSM Associates, Inc.

Industrial XP Environment

Photo courtesy of Joshua Kerievsky

4/19/2006 13Copyright QSM Associates, Inc.

Industrial XP Environment

Photo courtesy of Joshua Kerievsky

4/19/2006 14Copyright QSM Associates, Inc.

Traditional Release 1

Traditional Release – Whiteboard Sketch

4/19/2006 15Copyright QSM Associates, Inc.

Staffing & Probability Analysis

Avg Staff (people)<Single Goal - MBI 1.1946>

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26Jul'01

Aug Sep Oct Nov Dec Jan'02

Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan'03

Feb Mar Apr May Jun Jul Aug Sep

0

5

10

15

20

25Avg Staff (people)

876543 21

R&DC&T

Milestones 0 - CSR 1 - SRR 2 - HLDR 3 - LLDR 4 - CUT 5 - IC 6 - STC 7 - UAT 8 - FCR 9 - 99R 10 - 99.9R

Milestones 0 - CSR 1 - SRR 2 - HLDR 3 - LLDR 4 - CUT 5 - IC 6 - STC 7 - UAT 8 - FCR 9 - 99R 10 - 99.9R

SOLUTION PANEL <Single Goal - MBI 1.1946>

DurationEffortCost

Peak StaffMTTD

Start Date

C&T24.0347

589319.50.3

9/29/2001

Life Cycle27.0397

674119.50.3

7/1/2001

MonthsPM

$ (K)peopleDays

PI=17.0 MBI=1.2 Eff SLOC=376022

CONTROL PANEL <Single Goal - MBI 1.1946>

PI 13.6 20.4

17.0

Peak Staff 15.6 23.4

19.5

Eff SLOC (K) 301 451

376

Project: Condor

Digitized Replica Using SLIM™

4/19/2006 16Copyright QSM Associates, Inc.

Agile 1

Agile 2

Agile 3

XP Releases – Whiteboard Sketch

4/19/2006 17Copyright QSM Associates, Inc.

Staffing & Probability Analysis

Avg Staff (people)<XP Rel. 3.0>

1 2 3 4 5 6 7 8 9Apr'02

May Jun Jul Aug Sep Oct Nov Dec Jan'03

Feb Mar

0

2

4

6

8

10A

vgS

taff (people)876543 21

R&DC&T

Milestones0 - CSR1 - SRR2 - HLDR3 - LLDR4 - CUT5 - IC6 - STC7 - UAT8 - FCR9 - 99R10 - 99.9R

Milestones0 - CSR1 - SRR2 - HLDR3 - LLDR4 - CUT5 - IC6 - STC7 - UAT8 - FCR9 - 99R10 - 99.9R

SOLUTION PANEL <XP Rel. 3.0>

DurationEffortCost

Peak StaffMTTD

Start Date

C&T6.033

5618.41.8

8/31/2002

Life Cycle9.069

11738.41.8

6/1/2002

MonthsPM

$ (K)peopleDays

PI=20.7 MBI=3.8 Eff SLOC=67023

CONTROL PANEL <XP Rel. 3.0>

PI16.5 24.8

20.7

Peak Staff6.7 10.1

8.4

Eff SLOC (K)54 80

67

Project: Elan 3.0

Digitized Replica Using SLIMtm

4/19/2006 18Copyright QSM Associates, Inc.

Staffing & Probability Analysis

Avg Staff (people)<XP Rel. 3.2>

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18Jan'03

Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan'04

Feb Mar Apr May Jun Jul Aug Sep

0

2

4

6

8

10A

vgS

taff (people)87654321

R&DC&T

Milestones0 - CSR1 - SRR2 - HLDR3 - LLDR4 - CUT5 - IC6 - STC7 - UAT8 - FCR9 - 99R10 - 99.9R

Milestones0 - CSR1 - SRR2 - HLDR3 - LLDR4 - CUT5 - IC6 - STC7 - UAT8 - FCR9 - 99R10 - 99.9R

SOLUTION PANEL <XP Rel. 3.2>

DurationEffortCost

Peak StaffMTTD

Start Date

C&T15.088

14968.2-0.0

6/30/2003

Life Cycle19.010918548.2-0.0

3/1/2003

MonthsPM

$ (K)peopleDays

PI=14.8 MBI=1.3 Eff SLOC=76276

CONTROL PANEL <XP Rel. 3.2>

PI11.9 17.8

14.8

Peak Staff6.6 9.8

8.2

Eff SLOC (K)61 92

76

Project: Elan 3.2

Digitized Replica Using SLIMtm

4/19/2006 19Copyright QSM Associates, Inc.

4/19/2006 20Copyright QSM Associates, Inc.

David Tara

...

. . . . . . .

...

.. . .. .

.

..

. ...

. .. .

..

..

. ... . .

4/19/2006 21Copyright QSM Associates, Inc.

4/19/2006 22Copyright QSM Associates, Inc.

4/19/2006 23Copyright QSM Associates, Inc.

C&T Duration (Months) vs Effective SLOC

10 100 1000Effective SLOC (thousands)

1

10

100

C&

T Duration (M

onths)Agile 1

Agile 3

Agile 2

Traditional 1

Traditional 2

Agile 1

Agile 3

Agile 2

Traditional 1

Traditional 2

All Systems Special Project QSM 2002 Scientific Avg. Line Style 1 Sigma Line Style

Fast Schedules on Traditional ProjectsEven Faster on Agile Projects

Schedule Comparison

4/19/2006 24Copyright QSM Associates, Inc.

Errors SIT-FOC vs Effective SLOC

10 100 1000Effective SLOC (thousands)

1

10

100

1000

10000

Errors SIT-FOC

Traditional 1

Traditional 2

Agile 1

Agile 2

Agile 3

Traditional 1

Traditional 2

Agile 1

Agile 2

Agile 3

All Sy stems Special Project QSM 2002 Scientific Av g. Line Sty le 1 Sigma Line Sty le

High Defects on Traditional ProjectsLow Defects on Agile Projects

Defect Comparison

4/19/2006 25Copyright QSM Associates, Inc.

Example PI CalculationSize = 376,022 SLOC

Effort = 347 Person-Months

Time = 24 Months*Condor

PI = SIZE

TIME EFFORT= 17

4/19/2006 26Copyright QSM Associates, Inc.

Productivity Index (PI)(industry values by application type)

0 2 4 6 8 10 12 14 16 18 20 22 24

Productivity Index (PI) w/ ±1 Standard Deviation

Avionics

Business

Command and Control

Microcode

Process Control

Real Time

Scientific

System

Telecommunications

Information

Engineering

Real Time

4/19/2006 27Copyright QSM Associates, Inc.

Previous Performance

Current Performance

Percent Improvement

Project Cost $2.8 Million $1.1 Million 61%

Project Schedule 18 months 13.5 months 24%

CumulativeDefects

2,270 381 83%

Staffing 18 11 39%

* Using average project size of 150,000 lines of new and modified code

Before vs. After Summary

4/19/2006 28Copyright QSM Associates, Inc.

4/19/2006 29Copyright QSM Associates, Inc.

4/19/2006 30Copyright QSM Associates, Inc.

4/19/2006 31Copyright QSM Associates, Inc.

SLIM-Metrics

SLIM-Estimate

SLIM-ControlMetrics

Repository& Analysis

Size, Schedule,Cost & Quality

Estimating

StatisticalProcess

Control &Adaptive

Forecasting

Metrics Tools Used for this Analysis

4/19/2006 32Copyright QSM Associates, Inc.

“Without metrics, you’re just anotherperson with a different

opinion.”

4/19/2006 33Copyright QSM Associates, Inc.

RecommendedReading

Honore, Carl, “In Praise of Slowness, How A Worldwide Movement is Challenging the Cult of Speed”,© 2004 HarperSanFrancisco

Gladwell, Malcolm,“Blink; The Power of Thinking Without Thinking” © 2005 Little, Brown.

Mah, Michael, “The Making of the Agile IT Executive” Business IT Strategies Advisory Executive Report Vol 6 Number 10.© 2004 Cutter Information Corp.

Putnam, Lawrence H., and Myers, Ware, “Five Core Metrics,The Intelligence Behind Successful Software Management”© 2003 Dorset House Publishers.

4/19/2006 34Copyright QSM Associates, Inc.

RecommendedReading

Demarco, Tom and Tim Lister “Waltzing With Bears, Managing Risk on Software Projects” © 2003 Dorset House.

Ury, William, “Getting Past NO” © 1993 Bantam.

Fisher, Roger and Alan Sharp, “Getting It Done, How to Lead When You’re Not in Charge”, © 1998 HarperCollins.

Heen, Sheila, Doug Stone and Bruce Patton “Difficult Conversations - How to Discuss What Matters Most”,© 1999 Viking/Penguin.

4/19/2006 35Copyright QSM Associates, Inc.

Recommended Web Resources

Blogosphere:

www.optimalfriction.com

QSM Associates Web Library/Resource Center:

www.qsma.com

4/19/2006 36Copyright QSM Associates, Inc.

For Additional Information

Contact:

Michael MahManaging PartnerQSM Associates Inc.Clocktower Building75 So. Church St., Suite 600Pittsfield, MA 01201

Email: michael.mah@qsma.comWeb url: www.qsma.com