Business Value of
Agile MethodsUsing Cost of Quality (CoQ)
Dr. David F. Rico, PMP, CSMWebsite: http://davidfrico.com
LinkedIn: http://www.linkedin.com/in/davidfricoFacebook: http://www.facebook.com/profile.php?id=1540017424
2
AuthorDoD contractor with 25+ years of IT experienceB.S. Comp. Sci., M.S. Soft. Eng., & D.M. Info. Sys.Large gov’t projects in U.S., Far/Mid-East, & Europe
Published six books & numerous journal articlesExpertise in metrics, models, & cost engineeringAdjunct at Argosy, UMUC, & George WashingtonSix Sigma, CMMI, ISO 9001, DoDAF & DoD 5000Agile Program Management & Lean Development
3
AgendaOVERVIEW of BriefingIntro to Agile MethodsStudies of Agile MethodsCosts of Traditional MethodsCosts of Agile MethodsMetrics for Agile MethodsComparison of Agile MethodsSummary of Agile Methods
4
Purpose of BriefingProvide an overview of the business value of Agile Methods using ROI and Real Options:
Provide a brief introduction to agile methodsIllustrate some of the major agile methods/practicesSummarize the results of major cost/benefit studiesTalk a little bit about the cost of quality (CoQ)Introduce cost and benefit models for agile methodsDescribe metrics to estimate the ROI of agile methodsCompare the costs and benefits of agile methodsSummarize what we’ve learned about agile methods
5
What is Business Value?Val-ue (văl-'yōō): An amount, quantity, rate, magnitude, or desirability; Economic worth
An economic estimation of the tangible worth of the organizational assets such as buildings and equipmentAn appraisal of intangible assets such as knowledge, experience, skills, patents, processes, and methodsA technique for evaluating the costs and benefits of investments in a business, operations, or personnelThe economic impact of deploying a new product development approach such as agile methodologiesThe total life cycle costs of institutionalizing lean and agile project management techniques in an enterprise
6
Some of Today’s ChallengesChal-lenge (chăl-'ənj): Contest, competition, fight, defy, confront, or dispute; To question
21st century systems are more software-intensiveand highly-complex with numerous invisible partsTechnology is evolving at an exponential rate of change which severely limits the planning horizonGlobal competitiveness has intensified and new military threats are rapidly emerging all of the timeCustomers have unpredictable needs and necessitate decision-making flexibility throughout the projectToday’s 21st-century post-industrial information age knowledge workers need agile methods and tools
7
AgendaOverview of Briefing
INTRO to Agile MethodsStudies of Agile MethodsCosts of Traditional MethodsCosts of Agile MethodsMetrics for Agile MethodsComparison of Agile MethodsSummary of Agile Methods
8
What is Agility?A-gil-i-ty (ə-'ji-lə-tē) Quickness, lightness, and ease of movement; To be very nimble
The ability to create and respond to change in order to profit in a turbulent global business environmentThe ability to quickly reprioritize use of resources when requirements, technology, and knowledge shiftA very fast response to sudden market changes and emerging threats by intensive customer interactionUse of evolutionary, incremental, and iterativedelivery to converge on an optimal customer solutionMaximizing the business value with right-sized, just-enough, and just-in-time processes and documentation
9
What are Agile Methods?‘Adaptable’ system development methodologies‘Human-centric’ method for creating business value‘Alternative’ to large document-based methodologies
Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org
10
Essence of Agile MethodsHigh degree of customer & developer interactionHighly-skilled teams producing frequent iterationsRight-sized, just-enough, and just-in-time process
Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.
11
When to use Agile MethodsOn exploratory or research/development projectsWhen fast customer responsiveness is paramountIn organizations that are highly-innovative & creative
Highsmith, J. (2003). Agile project management: Principles and tools. Arlington, MA: Cutter Consortium.
12
Myths of Agile MethodsCommon myths still abound, although agile methods have been around for ~15 years:
Agile is only for software engineeringAgile doesn’t scale to large systemsAgile doesn't use project managementAgile doesn't have any requirementsAgile requires a traditional system architectureAgile doesn't have any documentationAgile isn't disciplined or measurableAgile has low quality, maintainability, and security
13
AgendaOverview of BriefingIntro to Agile Methods
STUDIES of Agile MethodsCosts of Traditional MethodsCosts of Agile MethodsMetrics for Agile MethodsComparison of Agile MethodsSummary of Agile Methods
14
Surveys of Agile MethodsNumerous surveys of Agile Methods since 2003AmbySoft and Version One collect annual dataGenerally include both hard and soft benefits
Rico, D. F. (2008). What is the return-on-investment of agile methods? Retrieved February 3, 2009, from http://davidfrico.com/rico08a.pdf
15
Agile (138 pt.) and traditional methods (99 pt.)Agile methods fare better in all benefits categoriesAgile methods 359% better than traditional methods
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
Studies of Agile Methods
16
Analysis of 23 agile vs. 7,500 traditional projectsAgile projects are 41% better than traditional onesXP (56%) and Scrum (26%) better than trad. projects
Mah, M. (2008). Measuring agile in the enterprise: Proceedings of the Agile 2008 Conference, Toronto, Canada.
Projects Using Agile Methods
17
Analysis of 29 agile vs. 7,500 traditional projectsAgile projects are 33% better than traditional onesRally projects are 28% better than traditional ones
Rally Software. (2009). The agile impact report. Boulder, CO: Author.
Projects Using Agile PM Tools
18
AgendaOverview of BriefingIntro to Agile MethodsStudies of Agile Methods
COSTS of Traditional MethodsCosts of Agile MethodsMetrics for Agile MethodsComparison of Agile MethodsSummary of Agile Methods
19
Cost of Quality (CoQ)1:10:100 ratio forms a basic model to estimate ROIDefects have negative multiplicative effect on costAgile methods leave fewer defects (higher ROI)
Boehm, B. W. (1981). Software engineering economics. Englewood Cliffs, NJ: Prentice-Hall.
Rel
a tiv
e C
o st
to F
ix E
rror
20
Traditional Cost ModelsCost estimation models still in use todayUsed to estimate effort of Traditional MethodsAdjusted average of 5,088 used for ROI estimation
Benediktsson, O., & Dalcher, D. (2005). Estimating size in incremental software development projects. Journal of Engineering Manufacture, 152(6), 253-259.
21
Total Lifecycle Costs0.51 hours/line of code for Traditional Methods10% defect inject rate (1,000 defects/10 KLOC)67% of defects in test (33% in maintenance)
Rico, D. F. (2004). ROI of software process improvement: Metrics for project managers and software engineers. Boca Raton, FL: J. Ross Publishing.In, H. P., et al. (2006). A quality-based cost estimation model for the product line life cycle. Communications of the ACM, 49(12), 85-88.McCann, B. (2007). The relative cost of interchanging, adding, or dropping quality practices. Crosstalk, 20(6), 25-28.
22
AgendaOverview of BriefingIntro to Agile MethodsStudies of Agile MethodsCosts of Traditional Methods
COSTS of Agile MethodsMetrics for Agile MethodsComparison of Agile MethodsSummary of Agile Methods
23
Agile Cost ModelsBased on 13 studies of Extreme Programming (XP)Also based on 7 studies of pair programming (PP)“Pair programming” had highest productivity
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
24
Agile Quality ModelsBased on 10 studies of Extreme Programming (XP)Also based on 6 studies of pair programming (PP)“Extreme Programming” had the highest quality
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
1. XP 10
2. TDD 2
3. PP 6
4. Scrum 3
5. Agile 21
No. Method Pts.
0.0032
0.6100
0.3250
0.4350
0.0032
Low
0.7466
2.1550
2.3550
3.9450
1.7972
Median
2.1450
3.7000
5.8500
8.5000
8.5000
High
0.7466 KLOC 100
2.1550 KLOC 100
2.3550 KLOC 100
3.9450 KLOC 100
1.7972 KLOC 100
Quality
25
Agile Lifecycle Cost ModelsCosts based on productivity and quality modelsDevelopment costs based on LOC ÷ productivity rateMaintenance costs based on defects × KLOC × MH
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
1. XP
2. TDD
3. PP
4. Scrum
5. Agile
No. Method Agile Lifecycle Cost Models
16.1575 0.7466
29.2800 2.1550
33.4044 2.3550
05.4436 3.9450
21.2374 1.7972
$249,653
$265,437
$578,202
$226,805
Costs
$136,548
26
Agile Lifecycle Benefit ModelsBenefits based on total traditional less agile costsTraditional costs based LOC × dev. + maint. effortTraditional costs credited testing effort applied
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
1. XP
2. TDD
3. PP
4. Scrum
5. Agile
No. Method Agile Lifecycle Benefit Models
(10,000 10.51 – 6,666.67 9) 100 – $136,548
(10,000 10.51 – 6,666.67 9) 100 – $249,653
(10,000 10.51 – 6,666.67 9) 100 – $265,437
(10,000 10.51 – 6,666.67 9) 100 – $578,202
(10,000 10.51 – 6,666.67 9) 100 – $226,805
$4,260,344
$4,244,560
$3,931,795
$4,283,192
Benefits
$4,373,449
27
AgendaOverview of BriefingIntro to Agile MethodsStudies of Agile MethodsCosts of Traditional MethodsCosts of Agile Methods
METRICS for Agile MethodsComparison of Agile MethodsSummary of Agile Methods
28
Measures of Business ValueA major principle of Agile Methods is creating valueROI is the measure of value within Agile MethodsThere are seven closely related ROI measures
Metric Definition Formula Costs
Sum of Costs Total amount of money spent ∑
=
n
iiCost
1
Benefits Sum of Benefits
Total amount of money gained ∑=
n
iiBenefit
1
B/CR Benefit to Cost Ratio
Ratio of benefits to costs Costs
Benefits
ROI Return on Investment
Ratio of adjusted benefits to costs %100×−
CostsCostsBenefits
NPV Net Present Value
Discounted cash flows ∑=
−+
Years
iYears
i CostsRateDiscount
Benefits1
0)1(
BEP Breakeven Point
Point when benefits exceed costs 1−CostsNewCostsOldCostsNew
ROA Real Options Analysis
Value gained from strategic delay ( ) ( ) YearsRateeCostsdNBenefitsdN ×−××−× 21
d1 = [ln(Benefits ÷ Costs) + (Rate + 0.5 × Risk2) × Years] ÷ Risk × √ Years, d2 = d1 − Risk × √ Years
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
29
Extreme ProgrammingCosts based on avg. productivity and qualityProductivity ranged from 3.5 to 43 LOC an hourCosts were $136,548, benefits were $4,373,449
∑ =
5
1i
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
30
Test Driven DevelopmentCosts based on avg. productivity and qualityProductivity ranged from 12 to 36 LOC an hourCosts were $249,653, benefits were $4,260,344
∑ =
5
1i
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
31
Pair ProgrammingCosts based on avg. productivity and qualityProductivity ranged from 16 to 87 LOC an hourCosts were $265,437, benefits were $4,244,560
∑ =
5
1i
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
32
ScrumCosts based on avg. productivity and qualityProductivity ranged from 4.7 to 5.9 LOC an hourCosts were $578,202, benefits were $3,931,795
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
33
Agile MethodsCosts based on avg. productivity and qualityProductivity ranged from 3.6 to 87 LOC an hourCosts were $226,805, benefits were $4,283,192
∑ =
5
1i
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
34
AgendaOverview of BriefingIntro to Agile MethodsStudies of Agile MethodsCosts of Traditional MethodsCosts of Agile MethodsMetrics for Agile Methods
COMPARISON of Agile MethodsSummary of Agile Methods
35
Productivity of Agile MethodsPP productivity 32X more than trad. methodsScrum productivity 5X more than trad. methodsAgile methods productivity 20X more than traditional
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
33.4044
29.28
21.2374
16.1575
5.4436
1.06190
8
16
24
32
40
PP TDD Agile XP Scrum CMMI®
Software Method
Pro
duct
ivit
y (L
OC
/Hou
r)
36
Quality of Agile MethodsXP quality 13X better than trad. methodsScrum quality 3X better than trad. methodsAgile methods quality 5X better than traditional
0.7466
1.7972 2.155 2.355
3.945
9.667
0
2.2
4.4
6.6
8.8
11
XP Agile TDD PP Scrum CMMI®
Software Method
Qua
lity
(Def
ects
/KLO
C)
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
37
Costs of Agile MethodsXP costs 8X less than traditional methodsScrum costs 2X less than traditional methodsAgile methods cost 5X less than traditional methods
$136,551$226,807 $249,653 $265,436
$578,202
$1,108,233
$0
$325,000
$650,000
$975,000
$1,300,000
XP Agile TDD PP Scrum CMMI®
Software Method
Cos
ts
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
38
Benefits of Agile MethodsXP benefits 1.5X more than traditional methodsScrum benefits 1.3X more than traditional methodsAgile methods benefits 1.4X more than trad. methods
$4,372,282$4,282,026 $4,259,180 $4,243,397
$3,930,631
$3,023,064
$2,750,000
$3,237,500
$3,725,000
$4,212,500
$4,700,000
XP Agile TDD PP Scrum CMMI®
Software Method
Ben
efit
s
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
39
ROI of Agile MethodsXP ROI 18X more than traditional methodsScrum ROI 3.4X more than traditional methodsAgile methods ROI 10X more than trad. methods
3,102%
1,788%1,606% 1,499%
580%
173%
0%
925%
1,850%
2,775%
3,700%
XP Agile TDD PP Scrum CMMI®
Software Method
Ret
urn
on In
vest
men
t
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
40
NPV of Agile MethodsXP NPV 2.4X more than traditional methodsScrum NPV 1.9X more than traditional methodsAgile methods NPV 2.3X more than trad. methods
$3,649,388$3,480,979 $3,438,351 $3,408,902
$2,825,313
$1,509,424
$1,000,000
$1,787,500
$2,575,000
$3,362,500
$4,150,000
XP Agile TDD PP Scrum CMMI®
Software Method
Net
Pre
sent
Val
ue
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
41
Real Options of Agile MethodsXP ROA 1.6X more than traditional methodsScrum ROA 1.4X more than traditional methodsAgile methods ROA 1.6X more than trad. methods
$4,265,936$4,105,884 $4,066,678 $4,040,377
$3,608,772
$2,633,052
$2,200,000
$2,825,000
$3,450,000
$4,075,000
$4,700,000
XP Agile TDD PP Scrum CMMI®
Software Method
Rea
l Opt
ions
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
42
Agile vs. Traditional MethodsAll of the methods were ordered by ROIAgile Methods had a high ROI value of 3,102%Traditional Methods had a high ROI value of 4,133%
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
43
AgendaOverview of BriefingIntro to Agile MethodsStudies of Agile MethodsCosts of Traditional MethodsCosts of Agile MethodsMetrics for Agile MethodsComparison of Agile Methods
SUMMARY of Agile Methods
44
SummaryAgility is the evolution of management thoughtConfluence of traditional and non-traditional ideasImprove performance by over an order-of-magnitude
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods: Maximizing ROI with just-in-time processes and documentation. Ft. Lauderdale, FL: J. Ross Publishing.
Systems engineering approaches
...
New product development approaches
Expertly designed to be fast and efficient
Intentionally lean and free of waste (muda)
Systematic highly-disciplined approaches
Capable of producing high quality systems
Right-sized, just-enough, and just-in-time tools
Intended to maximize business value for customers
45
New Book on Agile MethodsGuide to Agile Methods for business leadersCommunicates business value of Agile MethodsRosetta stone to Agile Methods for traditional folks
http://davidfrico.com/agile-book.htm (Description)http://www.amazon.com/dp/1604270314 (Amazon)
Table of Contents 1. Introduction to Agile Methods 2. Values of Agile Methods 3. History of Agile Methods 4. Antecedents of Agile Methods 5. Types of Agile Methods 6. Practices of Agile Methods 7. Agile Project Management 8. Agile Software Engineering 9. Agile Support Processes10. Agile Tools and Technologies11. Comparison of Agile Methods12. Agile Metrics and Models13. Surveys of Agile Methods14. Costs-Benefits of Agile Methods15. ROI Metrics of Agile Methods16. Measures of Agile Methods17. Costs of Agile Methods18. Benefits of Agile Methods19. ROI of Agile Methods20. NPV of Agile Methods21. Real Options of Agile Methods22. Business Value of Agile Methods23. Agile vs. Traditional Methods24. Future of Agile Methods