lean & agile systems engineering for systems of systems dr. david f. rico, pmp, csm website:...
TRANSCRIPT
Lean & AgileSystems
Engineeringfor Systems of Systems
Dr. David F. Rico, PMP, CSM
Website: http://davidfrico.comLinkedIn: http://www.linkedin.com/in/davidfrico
Facebook: http://www.facebook.com/profile.php?id=1540017424
2
Agenda
Introduction Systems Engineering
Systems Engineering Challenges Lean Systems Engineering
Agile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary
3
Author DoD contractor with 25+ years of IT experience B.S. Comp. Sci., M.S. Soft. Eng., & D.M. Info. Sys. Large gov’t projects in U.S., Far/Mid-East, & Europe
Published six books & numerous journal articles Expertise in metrics, models, & cost engineering Adjunct at George Washington, UMUC, & Argosy Six Sigma, CMMI, ISO 9001, DoDAF & DoD 5000 Agile Program Management & Lean Development
4
Purpose of Briefing Provide an overview of traditional, lean, and
agile systems engineering concepts: Define systems engineering, its purpose, and
identify major approaches to traditional systems development
Identify the strengths and weaknesses of traditional systems engineering for today’s ever changing world
Discuss lean and agile systems engineering as a means of managing ever increasing system complexity
Introduce mechanisms for scaling lean and agile systems engineering for larger systems of systems
Examine iterative testing techniques within agile systems engineering for verification and validation
5
AgendaIntroduction
Systems EngineeringSystems Engineering Challenges
Lean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary
6
What is Systems Engineering? Sys-tem (sĭs-'təm): Interacting, interrelated,
interdependent elements; A complex whole Interdisciplinary approach and means to
enable the realization of successful systems [INCOSE] Interdisciplinary tasks required to transform
customer needs into a system solution [IEEE] Interdisciplinary approach for transforming a set of
customer needs into a product solution [CMMI] Interdisciplinary approach for translating mission needs
into operational systems [DoD 5000] Interdisciplinary processes spanning the conception
of ideas through the retirement of a system [ISO]
7
Purpose of Systems Engineering Manage increasing system complexity (1950s) Optimize [sub]system performance (1960s) Improve system cost and quality (1970s)
Eisner, H. (2002). Essentials of project and systems engineering management. New York, NY: John Wiley & Sons.Blanchard, B. S., & Fabrycky, W. J. (2006). Systems engineering and analysis. Upper Saddle River, NJ: Pearson Prentice-Hall.
8
MIL-STD-1521B Created by U.S. Air Force in 1976 Framework for system and software reviews Standardized milestone reviews and technical audits
U.S. Department of Defense. (1985). Military standard: Technical reviews and audits for systems, equipments, and computer software (MIL-STD-1521B). Washington, DC: Air Force Systems Command (AFSC).
Note: ActualTime Framesof Activities
Must BeTailored to
Each Program
Test
DraftSystem
RequirementsSpecification
SystemRequirementsSpecification
HWCIDevelopmentSpecification
SoftwareTop Level
DesignDocument
SoftwareListing
HWCIProduct
Specification
SoftwareProduct
Specification
IRSDBDD
DraftHWCIProduct
Specification
Sourceand
ObjectCode
SystemRequirements
Review
SystemDesignReview Software
SpecificationReview
PreliminaryDesignReviewCSCI
PreliminaryDesignReviewHWCI
CriticalDesignReviewCSCI
CriticalDesignReviewHWCI
TestReadinessReview
FunctionalConfiguration
AuditHWCI
PhysicalConfiguration
AuditHWCI
FormalQualification
ReviewHWCI
FunctionalConfiguration
AuditCSCI
PhysicalConfiguration
AuditCSCI
SystemFunctional
ConfigurationAudit
SystemPhysical
ConfigurationAudit
SystemFormal
QualificationReview
Prepare Testand
EvaluationMaster Plan
PrepareSoftware
TestPlan
PrepareHWCITestPlan
PrepareSoftware
TestDescription
PrepareHWCITest
Procedures
PrepareSoftware
TestProcedures
PerformHWCI
SubsystemTests
PerformSoftware
Tests
PrepareSoftware
TestReports
PerformSystemTests
PrepareSystem
TestReports
ProgramActivity
ConceptExploration
Phase
Demonstrationand Validation
PhaseFull Scale Development Phase
TechnicalReviews
andAudits
Specifica-tions and
otherProducts
SystemFunctional
Identification
AllocatedIdentification
Developmental Configuration(HWCI Only)
Product Identification
SystemFunctionalBaseline
AllocatedBaseline
ProductBaseline
SoftwareRequirementsSpecification
IRDSoftwareDetailedDesign
Document
9
MIL-STD-498 Created by U.S. Navy in 1994 Consolidated multiple U.S. DoD standards Software process and documentation standard
Project Planning and
Oversight
System Requirements
Analysis
SystemDesign
Software Requirements
Analysis
Software Implementation and Unit
Testing
UnitIntegrationand Testing
CSCI Qualification
Testing
CSCI/HWCI Integrationand Testing
System Qualification
Testing
Preparing for Software Use
Preparing for Software TransitionPreliminary Detailed
Software DesignSoftwareActivity
QualityAssurance
Configura-tion
Manage-ment
Baseline Functional Allocated Product(Developmental Configuration)
FunctionalConfiguration
Audit(FCA)
PhysicalConfiguration
Audit(PCA)
FunctionalConfiguration
Audit(FCA)
PhysicalConfiguration
Audit(PCA)
Audit Audit Audit Audit Audit Audit Audit Audit Audit Audit Audit Audit Audit
J ointReview
FormalQualification
Review(FQR)
FormalQualification
Review(FQR)
SystemRequirements
Review(SRR)
SystemDesignReview(SDR)
SoftwareSpecification
Review(SSR)
PreliminaryDesignReview(PDR)
CriticalDesignReview(CDR)
TestReadiness
Review(TRR)
Program Definition and Risk ReductionConcept Exploration Engineering and Manufacturing DevelopmentAcquisition
Phase
TechnicalReview
· Walkthru· Inspection
· Walkthru· Inspection
· Walkthru· Inspection
· Walkthru· Inspection
· Walkthru· Inspection
· Walkthru· Inspection
· Walkthru· Inspection
· Walkthru· Inspection
· Walkthru· Inspection
· Walkthru· Inspection
· Walkthru· Inspection
· SPS· SVD· SUM· SIOM· SCOM· COM
· SSDD· CPM· FSM
· STD· STR
· STD· STR
· SDD· IDD· DBDD
· SDD· IDD· DBDD
· SRS· IRS
· SSDD· IDD· DBDD
· OCD· SSS· IRS
· SDP· STP· SIP· STrP
SoftwareProduct
U.S. Department of Defense. (1994). Military standard: Software development and documentation (MIL-STD-498). Arlington, VA: Space and Naval Warfare Center (SPAWAR).
10
ISO-15288 Created by ISO/IEC around 2002 Standardization of international practices Meant for complex, computer-based systems
International Organization for Standardization/International Electrotechnical Commission. (2002). Standard for systems engineering: System life cycle processes (ISO/IEC 15288). Geneva, Switzerland: Author.
11
CMMI Created by the SEI in 2002 Merger of SW-CMM, SA-CMM, IPD-CMM, etc. Used for systems engineering process improvement
CMMI Product Team. (2006). CMMI for development version 1.2 (CMU/SEI-2006-TR-008). Pittsburg, PA: Software Engineering Institute.
12
DoD Acquisition Lifecycle Created by the U.S. DoD around 2003 Latest evolution of acquisition best practices Meant for large-scale, multi-billion weapon systems
DAU. (2009). Integrated defense acquisition, technology, and logistics life cycle management framework. Retrieved October 9, 2009, from https://acc.dau.mil/ifc
13
Systems Engineering Benefits Study funded by Australian defense institute Almost 44 programs studied from 2001 to 2004 Systems engineering minimizes schedule overruns
Honour, Eric C. (2009). Demographics in measuring systems engineering return on investment (SE-ROI). Proceedings of the Joint 19th Annual International Symposium of INCOSE/Third Asia-Pacific Conference on Systems Engineering (INCOSE/APCOSE 2009), Singapore.
14
Systems Engineering Studies U.S. Air Force Center for Systems Engineering Case studies of 9 major U.S.Air Force Programs Programs had significant cost and technical issues
Air Force Institute of Technology (AFIT). (2009). Systems engineering case studies. Retrieved October 19, 2009, from http://www.afit.edu/cse/cases.cfm
PROGRAM NAME EST. COST EST. UNIT ACT. COST ACT. UNIT OVERRUN OTHER ISSUES
A-10 Thunderbolt $2,547,600,000 439 $5,511,490,000 360 21,005% Wing Reliability
B-2 Spirit $58,200,000,000 132 $45,300,000,000 20 57,949% Low Observability
C-5A Galaxy $3,413,200,000 120 $4,426,400,000 81 7,585% Maintenance
F-111 Aardvark $2,171,590,000 547 $8,652,000,000 547 298% Structural
RQ-4A Global Hawk $2,904,600,000 51 $3,816,700,000 51 31% Quality
GPS Navstar $814,400,000 28 $8,650,000,000 14 31,764% Sofware Reliability
HST Hubble $651,200,000 1 $1,458,900,000 1 124% Schedule, Mirror
K C-10 Extender $1,055,000,000 20 $1,090,600,000 20 3% Reliability
LGM-118A Peacekeeper $16,600,000,000 21 $16,600,000,000 21 0% Range
$9,817,510,000 151 $10,611,787,778 124 13,196% $85,655,686
15
AgendaIntroductionSystems Engineering
Systems Engineering Challenges Lean Systems Engineering
Agile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary
16
What is a Challenge? Chal-lenge (chăl-'ənj): Contest, competition,
fight, defy, confront, or dispute; To question 21st century systems are more software-intensive and
highly-complex with numerous invisible parts Technology is evolving at an exponential rate of
change which severely limits the planning horizon Global competitiveness has intensified and new
military threats are rapidly emerging all of the time Customers have unpredictable needs and necessitate
decision-making flexibility throughout the program Today’s post-industrial information age knowledge workers
need new systems engineering approaches
17
Information Age U.S. is no longer an industrial-age nation U.S. part of a group of post-industrial countries U.S. consists of information-age knowledge workers
Bell, D. (1999). The coming of post industrial society. New York, NY: Basic Books.
0%
20%
40%
60%
80%
100%
1860 1870 1880 1890 1900 1910 1920 1930 1940 1950 1960 1970 1980 1990
Perc
ent o
f Eco
nom
y
Information
Service
Industry
Agriculture
18
System Complexity is Growing 21st century systems are becoming more complex Number of physical parts are becoming smaller Nano-circuitry and software hide complexity
Moody, J. A., et al. (1997). Metrics and case studies for evaluating engineering designs. Upper Saddle River, NJ: Prentice-Hall.
19
Software Century Number of software-intensive systems is growing Yearly software industry revenue exceeds $3 trillion Poor software quality costing trillions in lost revenues
Dvorak, D. L. (2009). NASA study on flight software complexity. Pasadena, CA: Jet Propulsion Laboratory (JPL).
Software in Military Aircraft
8% 10%20%
35%45%
65%
80%
0%
20%
40%
60%
80%
100%
1960F-4
1964A-7
1970F-111
1975F-15
1982F-16
1990B-2
2000F-22
Perc
ent o
f Fun
ctio
nalit
y
20
Exponential Rate of Change Technology evolving at an ever increasing rate Nano-scale computers will become the norm soon Technological breakthroughs may climax in 25 years
Kurzweil, R. (2005). The singularity is near: When humans transcend biology. New York, NY: Penguin Group.
21
Crossing the Chasm New technology spreads very slowly There are a few innovators and early adopters Years and decades for most to adopt new technology
Moore, G. A. (1991). Crossing the chasm: Marketing and selling technology to mainstream customers. New York, NY: Harper Business.
22
Coping With Big Changes Humans can’t cope with large technological change Changes may be resisted for a long time (years) Big projects plunge organizations into chaos
Sidky, A. (2008). Becoming agile in an imperfect world. Washington, DC: Agile Project Leadership Network (APLN).
23
Global Market Competition Globalization has intensified market competition Domestic competition is no longer the major threat The trade deficit with the Far East is growing bigger
24
Cyber Threats are Growing Cyber threats have increased 10-fold in last decade 70% of cyber incidents perpetrated by U.S. citizens Cyber threats coming from Far East less than 3%
0
20,000
40,000
60,000
80,000
100,000
120,000
140,000
160,000
1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 20030
500
1000
1500
2000
2500
3000
3500
4000
4500
Incidents
Vulnerablities
25
Complex Systems are Unstable Large systems experience big downstream changes Project plans designed to cope with small changes Systems engineering not well-suited for changes
Jones, C. (1995). Patterns of software system failure and success. Boston, MA: International Thompson Computer Press.
Basic Assembly 575J CL 400Macro Assembly 400C 225Cobol 74 (Cobol I) 220FORTRAN 210Cobol 85 (Cobol II) 175Pascal 160PL/1 126RPG I 120RPG II/I I I 110Natural 100C++ 80J ava 80dBase III 60Focus 60Clipper 60Oracle 60Sybase 60dBase IV 55Perl 50J avaScript 50VBScript 50Shell Script 50SAS 50APL 50
26
High Project Failure Rates Failed and challenged projects hover around 70% High failure rate due to inability to cope with change Big projects exacerbate challenge and failure potential
Johnson, J., et al. (2009). Chaos summary 2009. Boston, MA: Standish Group International.
27
AgendaIntroductionSystems EngineeringSystems Engineering Challenges
Lean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary
28
What is Lean? Lean (lēn): Thin, slim, slender, narrow,
adequate, or just-enough; Without waste A customer-driven systems engineering process that
delivers the maximum amount of business value An economical systems engineering way of planning
and managing the development of complex systems A systems engineering process that is free of excess
waste, capacity, and non-value adding activities Just-enough, just-in-time, and right-sized systems
engineering processes, documentation, and tools A systems engineering approach that is adaptable to
change in customer needs and market conditions
29
Lean Thinking Term coined by John Krafcik of MIT in 1988 Taiichi Ohno of Toyota is credited with its ideas Toyota Production System was adapted from Ford
Womack, J. P., & Jones, D. T. (1996). Lean thinking: Banish waste and create wealth in your corporation. New York, NY: Free Press.Liker, J. K. (2004). The toyota way: 14 management principles from the world’s greatest manufacturer. New York, NY: McGraw Hill.Larman, C., & Vodde, B. (2008). Scaling lean and agile development: Thinking and organizational tools for large-scale scrum. Boston, MA: Addison-Wesley.
Respect for People· Don’t trouble your customer
· Develop people, then build products
· No wasteful work
· Teams and individuals evolve their own practices and improvements
· Build partners with stable relationships, trust, and coaching lean thinking
· Develop teams
Development Practices
· Long-term great engineers· Mentoring manager-engineer-teacher· Cadence· Cross-functional· Team room + visual management· Entrepreneurial chief / product manager· Set-based concurrent development· Create more knowledge
14 Lean Principles
Long-term philosophy, flow, pull, level workload, stop and fix, master norms, visual controls, tested technology, leaders-teachers from within, develop exceptional people, help partners be lean, go see, consensus and action, learning-reflection-kaizen
Continuous Improvement· Go see
· Kaizen
· Spread knowledge
· Small, relentless
· Retrospectives
· 5 whys
· Eyes for waste, variability, overburden, NVA (handoff, WIP, info scatter delay, multitasking, defects, wishful thinking…)
· Perfection challenge
· Work to flow (smaller batch sizes, low cycle time)
Sustainable shortest lead time. Best quality and value (to people and society).Most customer delight, lowest cost, high morale, and safety.
The Goal: Value
Foundation: Management SupportManagement applies and teaches lean thinking, and bases decisions on this long-term philosophy
30
Lean Six Sigma Created in late 1990s by Allied Signal and Maytag Combination of Six Sigma and Lean Thinking Focuses on eliminating waste vs. variation
George, M. L. (2002). Lean six sigma: Combining six sigma quality with lean speed. New York, NY: McGraw-Hill.
31
Lean Development Lean product development emerged in the 1980s Adaptation of Toyota Production System (TPS) “Toyota [New] Product Development System”
Clark, K. B., & Fujimoto, T. (1991). Product development performance: Strategy, organization, and management in the world auto industry. Boston, MA: Harvard Business School Press.
Lean Development
Frequent set-up changes
Lean Manufacturing
Short manufacturing throughput time
Reduced work-in-process inventory between manufacturing steps
Frequent transfer of small batches of parts between manufacturing steps
Reduced inventory requires slack resources and more information flow between steps
Adaptability to changes in volume, product mix, and product design
Broad task assignments for production workers gives higher productivity
Focus on quick problem solving and continuous process improvement
Simultaneous improvement in quality, delivery time, and manufacturing productivity
Frequent product changes
Short development time
Reduced information inventory between development steps
Frequent transfer of preliminary information between development steps
Reduced development time requires slack resources and information flow between stages
Adaptability to changes in product design, schedule, and cost targets
Broad task assignments for engineers (developers) gives higher productivity
Focus on frequent incremental innovation and continuous product and process improvement
Simultaneous improvement in quality, development time, and development productivity
32
Lean Systems Engineering Origin in MIT Lean Aerospace Initiative in 1992 Lean Systems Engineering WG formed in 2006 Lean Enablers for Systems Engineering in 2009
INCOSE. (2009). Lean enablers for systems engineering. Retrieved October 20, 2009, from http://www.incose.org/practice/techactivities/wg/leansewg
� Value
� Map the Value
Stream
� Flow
� Pull
� Perfection
� Respect for People
· Customer involvement
· Streamline development· Deconflict suppliers· Establish metrics
· Communicate effectively· Achieve smooth flow· Ensure program visibility· Use lead
· Pull tasks as-needed
· Appoint a chief· Eliminate waste· Perform continuous improvement
· Create a learning environment· Treat people as valuable assets
· Requirements engineering· Business value analysis
· Program planning· Map value streams· Frontload program
· Program monitoring· Clarify requirements· Frontload architecture· Coordinate activities
· Tailor program
· Continuous improvement· Strive for excellence· Perform lessons learned· Develop communication plan
· Human resources· Respect all people· Strive for technical excellence
33
Lean+ 10X Created by Charles Toups of Boeing in 2008 In-use by P-8A Poseidon and AEW&C System Adaptation of lean thinking for non-manufacturing
Brabant, E. M. (2009). Simple as. Retrieved October 20, 2009, from http://www.boeing.com/news/frontiers/i_ids01.pdf
34
Lean Engineering Benefits MIT has studied dozens of systems for last 15 years They applied criteria to determine if they were lean Numerous programs, past, present, and future
Murman, E., et al. (2002). Lean enterprise value: Insights from MIT's lean aerospace initiative. New York, NY: Palgrave.
35
AgendaIntroductionSystems Engineering
Systems Engineering ChallengesLean Systems Engineering
Agile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary
36
What is Agility? A-gil-i-ty (ə-'ji-lə-tē) Quickness, lightness, and
ease of movement; To be very nimble The ability to create and respond to change in order
to profit in a turbulent global business environment The ability to quickly reprioritize use of resources
when requirements, technology, and knowledge shift A very fast response to sudden market changes and
emerging threats by intensive customer interaction Use of evolutionary, incremental, and iterative
delivery to converge on an optimal customer solution Maximizing the business value with right-sized, just-
enough, and just-in-time processes and documentation
37
What are Agile Methods? ‘Adaptable’ software development methodologies ‘Human-centric’ method for creating business value ‘Alternative’ to large document-based methodologies
Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org
alsoknown as
CustomerCollaboration
Individuals &Interactions
WorkingSoftware
Respondingto Change
CustomerInteraction
High-Performance Teams
IterativeDevelopment
Adaptabilityor Flexibility
ContractNegotiation
Processes& Tools
ComprehensiveDocumentation
Followinga Plan
Agile Methods‘Values’
alsoknown as
alsoknown as
alsoknown as
valuedmore than
valuedmore than
valuedmore than
valuedmore than
Agile Methods‘Principles’
Traditional Methods‘Values’
38
Crystal Methods Created by Alistair Cockburn in 1991 Has 14 practices, 10 roles, and 25 products Scalable family of techniques for critical systems
Cockburn, A. (2002). Agile software development. Boston, MA: Addison-Wesley.
39
Scrum Created by Jeff Sutherland at Easel in 1993 Has 5 practices, 3 roles, 5 products, rules, etc. Uses EVM to burn down backlog in 30-day iterations
Schwaber, K., & Beedle, M. (2001). Agile software development with scrum. Upper Saddle River, NJ: Prentice-Hall.
40
Dynamic Systems Develop. Created by group of British firms in 1993 15 practices, 12 roles, and 23 work products Non-proprietary RAD approach from early 1990s
Stapleton, J. (1997). DSDM: A framework for business centered development. Harlow, England: Addison-Wesley.
41
Feature Driven Development Created by Jeff De Luca at Nebulon in 1997 Has 8 practices, 14 roles, and 16 work products Uses object-oriented design and code inspections
Palmer, S. R., & Felsing, J. M. (2002). A practical guide to feature driven development. Upper Saddle River, NJ: Prentice-Hall.
Develop anOverall Model
Build aFeatures List
Plan byFeature
Design byFeature
Build byFeature
Iteration
42
Extreme Programming Created by Kent Beck at Chrysler in 1998 Has 28 practices, 7 roles, and 7 work products Popularized pair programming and test-driven dev.
Beck, K. (2000). Extreme programming explained: Embrace change. Reading, MA: Addison-Wesley.
UserStories
ArchitecturalSpike
ReleasePlanning
IterationAcceptance
TestsSmall
Releases
Spike
TestScenarios
SystemMetaphor
CustomerApproval
LatestVersion
ReleasePlan
NextIteration
BugsNew
StoriesRequirements
UncertainEstimates
ConfidentEstimates
XPExtreme Programming
43
Side-Effects of Agile Methods Enable us to cross-the-chasm sooner or earlier Reduce chaos associated with large-scale change Reduce or divide the risk of change into small pieces
Sidky, A. (2008). Becoming agile in an imperfect world. Washington, DC: Agile Project Leadership Network (APLN).
44
Essence of Agile Methods High degree of customer & developer interaction Highly-skilled teams producing frequent iterations Right-sized, just-enough, and just-in-time process
Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.
Adaptability or Flexibility
High-Performance Teams
Iterative Development
Customer Interaction
45
AgendaIntroductionSystems Engineering
Systems Engineering ChallengesLean Systems EngineeringAgile Systems Engineering
Agile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary
46
What is a Practice? Prac-tice (prăk-'tĭs): Action, tool, technique, or
work instruction; Step-by-step procedure A set of one or more systems engineering techniques
to accomplish a specific action or desired outcome Standard or semi-formal best practices or rules-of-
thumb that are proven to be effective or efficient A suite of manual or automated tools or instruments
that are useful for system design and development An array of optional elements that may be employed on
an as-needed basis, i.e., right tool at the right time Value-adding action that may significantly enhance
productivity, quality, or other key performance metric
47
Release Planning Created by Kent Beck at Chrysler in 1998 Project plan with a 30-60-90-day timing horizon Disciplined and adaptable project management F/W
Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.
Release Plan Iteration P lan
UserWrite
StoriesUser
Estimate
StoriesUser
Split
StoriesUser
Order
StoriesUser
Analyze
StoriesDev.
Create
TasksDev.
Accept
TasksDev.
Estimate
Tasks
48
Onsite Customers Term coined by Kent Beck in 1999 Customer who sits with developers full-time Fast and efficient way to capture customer needs
Tabaka, J. (2006). Collaboration explained: Facilitation skills for software project leaders. Upper Saddle River, NJ: Addison Wesley.
49
User Stories Term coined by Kent Beck in 1999 Functions or features of value to customers Highly adaptable requirements engineering process
Cohn, M. (2004). User stories applied: For agile software development. Boston, MA: Addison-Wesley.
50
Pair Programming Term coined by Jim Coplien in 1995 Consists of two side-by-side programmers Highly-effective group problem-solving technique
Williams, L., & Kessler, R. (2002). Pair programming illuminated. Boston, MA: Pearson Education.
MOVE PEOPLE AROUND
REFACTOR MERCILESSLY
CONTINUOUS INTEGRATION
CREATE AUNIT TEST
We NeedHelp
ChangePair
SimpleCode
ComplexCode
New UnitTests
NewFunction-
ality
FailedUnitTest
PassedUnitTest
51
Refactoring Term coined by William Opdyke in 1990 Process of frequently rewriting source code Improves readability, maintainability, and quality
Fowler, M. (1999). Refactoring: Improving the design of existing code. Boston, MA. Addison-Wesley.
52
Test-Driven Development Term coined by Kent Beck in 2003 Consists of writing all tests before coding Ensures all source code is verified and validated
Beck, K. (2003). Test-driven development: By example. Boston, MA: Addison-Wesley.
53
Continuous Integration Term coined by Martin Fowler in 1998 Process of automated build/regression testing Evaluates impact of changes against entire system
Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley.
BuildIntegration
Server
VersionControlServer
BuildScripts
UsesWatches
BuildStatus
ProvidesDeveloper A
Developer B
Developer C
CommitsChanges
CommitsChanges
CommitsChanges
Compile Source Code
Run Unit Tests
Run Coverage Tests
Static Code Analysis
Build Database
Generate Help Files
Archive and Deploy
54
Agile Documentation Myth that voluminous documentation is needed Myth that agile methods do not use documentation Right-sized, just-in-time, and just enough documents
Rueping, A. (2003). Agile documentation: A pattern guide to producing lightweight documents for software projects. West Sussex, England: John Wiley & Sons.
Contracts
Document Type
Project P lans
Requirements
Architecture
Design
Coding
Tests
User guides
Quality Assurance
Agile Documentation
Performance-based, time-and-materials, level-of-effort
Release plans, iteration plans, story boards, agile repositories
User stories, wire frames, use cases, paper prototypes
Metaphors, spikes, system modeling language, DoDAF
Wire frames, design patterns, unified modeling language
Code patterns, program design language, coding comments
Unit, component, integration, system, and acceptance tests
XML documents, online help, Wikis, FAQs, video and audio clips
Performance, reliability, code structure analysis, and test reports
55
Which Practices Are In-Use Surveys of agile practices are conducted annually Release planning is the most often used practice Continuous integration is also a major practice
Version One. (2008). The state of agile development: Third annual survey. Alpharetta, GA: Author.
86%
75%
72%
65%
62%
60%
59%
59%
52%
49%
44%
35%
34%
31%
31%
0% 20% 40% 60% 80% 100%
Iteration planning
Daily s tandups
Release planning
Continuous integration
Automated builds
Burndown
Retrospectives
Refactoring
Velocity
Test-driven development
Open work area
Digital taskboard
On-s ite customer
Pair programming
Information radiators
56
AgendaIntroductionSystems Engineering
Systems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering Practices
Agile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary
57
What is Scalability? Scal-a-ble (skāl-'ə-bəl): To expand, grow,
stretch, raise, or intensify; Increase in size A systems engineering process that applies to projects of
varying size, scope, magnitude, and complexity A product development process that is tailorable to the
type, kind, and class of product under development A process that works well on range of products, from
small to large programs involving systems of systems An approach that enables control of time, cost, scope,
quality, and performance regardless of program type Systems engineering processes designed to maximize
business value under a wide variety of constraints
58
Multi-Level Teams Enables projects to plan for the future and present Decomposes capabilities into implementable pieces Unclogs the drainpipes to let the execution flow freely
Highsmith, J. (2010). Agile project management: Creating innovative products. Boston, MA: Pearson Education.
59
Multi-Level Backlog Enables multiple levels of abstraction to co-exist Allows customers and developers to communicate Makes optimum use of everyone’s time and resources
Highsmith, J. (2010). Agile project management: Creating innovative products. Boston, MA: Pearson Education.
n Feature Set· Related user stories that are grouped together
· Also called a Theme, i.e., implemented as entity
· Comprises 6 to 30 days worth of work
n Capability· High-level business or product function
· Also called an Epic, i.e., multiple feature sets
· Comprises 18-90 days worth of work
n User Story· Simple requirement written by customer or user
· A small unit of functionality having business value
· Comprises 2 to 10 days worth of work
60
Multi-Level Planning Enables multiple level enterprise plans to co-exist Allows stakeholders to build viewpoint-specific plans Ensures capabilities are delivered at regular intervals
n Release Plan· Feature set focused, subsystem architecture
· Strategy, objectives, and backlog
· 6 to 12 weeks
n Product Roadmap· Capability focused, enterprise architecture needs
· Vision, objectives, backlog
· 18 to 36 weeks
n Iteration Plan· User story focused, component-level architecture
· Implementation plan, objectives, and backlog
· 2 to 4 weeks
Highsmith, J. (2010). Agile project management: Creating innovative products. Boston, MA: Pearson Education.
61
Multi-Level Coordination Enables lean and agile methods to scale-up Allows enterprises to create large-scale programs Unleashes optimum productivity and overall control
Schwaber, K. (2004). Agile project management with scrum. Redmond, WA: Microsoft Press.
62
Multi-Level Governance Enables enterprises to achieve functional needs Allows programs to coordinate functional activities Ensures optimal technical performance is achieved
Ambler, S. W. (2009). Scaling agile software development through lean governance. Proceedings of the 2009 ICSE Workshop on Software Development Governance, Vancouver, Canada, 1-2.
Q
I
R
W
T
A
S
C
D
R
R
R
R
R
R
R
R
R
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
T
T
T
T
T
T
T
T
T
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
S
S
S
S
S
S
S
S
S
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
Q
I
R
W
T
A
S
C
D
R T S
63
Multi-Level PMO Enables enterprises to optimize project performance Allows enterprises to control and monitor programs Ensures projects are operating at peak capability
Augustine, S., & Cuellar, R. (2006). The lean-agile PMO: Using lean thinking to accelerate agile project delivery. Agile Project Management Executive Report, 7(10), 1-28.
64
Multi-Level Automation Enables enterprises to be flexible but disciplined Allows enterprises to distribute project work teams Ensures distributed project teams are collaborating
Project Management
Requirements
· DOORS· Requisite Pro· SLATE
Design
· Rhapsody· Telelogic System Architect· Rational System Architect
Coding
· Eclipse· Visual Studio· Sun Studio
Testing
· JUnit· NUnit· Xunit· CPPUnit
· Gtest· Fit· Fitnesse· Selenium
Quality Assurance
· CheckStyle· PMD· EMMA· Jdepend· Cobertura· Gcov
Configuration Mgt
· Subversion (SVN)· Concurrent Versions Sys.· ClearCase
Build Automation
· Ant· NAnt· Maven· Make
Continuous Integ.
· Cruise Control· Hudson· BuildBot
Collaboration
· WebEx· Skype· MeetMe· Wimba
Wiki
· MediaWiki· TracWiki· PhpWiki
Documentation
· NDoc· Javadoc· Doxygen· iText
· Version One· Rally· Scrum Works· VSTS
· Agile Team· Agile Enterprise· Scope Manager· Story Studio
· XP Plan It· Iterate· XP Tracker· Agilo
· XP CGI· XP Web· Xplanner· Ice Scrum
· Project Cards· Target Process· Xtreme Planner· Team System
· Community· Enterprise· Mingle· Hansoft
65
AgendaIntroductionSystems Engineering
Systems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering Scaling
Agile Systems Engineering TestingAgile Systems Engineering ValueSummary
66
What is Integration? In-te-gra-tion (ĭn-'tĭ-grā-shən): To add, group,
mix, or assemble; Act of combining A critical verification and validation step in the
systems engineering process for a complex new system A process of testing and evaluating units,
components, subsystems, systems, and systems of systems A key best practice that enables suppliers to deliver
operational systems to customers early and often A automated systems development process that lowers
the risks of developing large-scale complex systems A lean and efficient process that maximizes business
value by eliminating waste from traditional testing
67
What is Agile Testing? Traditional testing is a late, manual process Agile testing is an early and automated process The goal of agile testing is to deliver early and often
Traditional Testing
· Combining source files
· Combining software and environment
· Combining software and data
· Combining software and tests
· Combining developers
Agile Testing
· Code is frequently checked in
· Code is automatically retrieved
· Compilation is done automatically
· Tests are done automatically
· Code reports are generated
· Developers get instant feedback
· Code is automatically deployed or packaged for delivery
Grant, T. (2005). Continuous integration using cruise control. Northern Virginia Java Users Group (Novajug), Reston, Virginia, USA.
68
Agile Testing Process Developers check-in changes as they occur Server detects all changes and initiates testing Server compiles, tests, analyzes, builds, and deploys
Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley.
BuildIntegration
Server
VersionControlServer
BuildScripts
UsesWatches
BuildStatus
ProvidesDeveloper A
Developer B
Developer C
CommitsChanges
CommitsChanges
CommitsChanges
Compile Source Code
Run Unit Tests
Run Coverage Tests
Static Code Analysis
Build Database
Generate Help Files
Archive and Deploy
69
Agile Testing Technologies There are literally hundreds of agile testing tools There are tools for building, testing, and deploying Integration tools monitor repositories and initiate tests
Smart, J. (2009). Automated deployment with maven and friends: Going the whole nine yards. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA.
70
Agile Testing Statistics Fewer builds leave in higher bug counts A high number of builds eliminates the defects Goal is to have as many, early builds as possible
Lacoste, F. J. (2009). Killing the gatekeeper: Introducing a continuous integration system. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA, 387-392.
71
Scaling Agile Testing Agile testing slows down with very large systems Slow testing slows integration and increases bugs Agile testing can speed back up with proper attention
Kokko, H. (2009). Increase productivity with large scale CI: Reduce feedback cycle from weeks to 100 minutes. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA.
Wide Impact Tuning
· Fast builds – less changes – more green
· Parallelization of test runs
· ClearCase to subversion
· Pre-installing as much as possible
· Removal of randomness
· Compilation in memory
· Installation starting parallel with system build
Focused Impact Tuning
· More memory and CPUs
· Parallelize builds
· Replace 3rd party test libraries
· Reduce/remove timeouts in tests
· Select different tests
· Refactor code & components
· Tune the network & software
· Tune the database
72
Agile Testing Costs Most agile testing tools are “free” open source A build server is no more than a commodity PC Low overhead for new and subsequent setup time
Grant, T. (2005). Continuous integration using cruise control. Northern Virginia Java Users Group (Novajug), Reston, Virginia, USA.
þ
þ
þ
þ
þ
þ
þ
Free, Open Source Software
$500 for a dedicated build machine
4 hours configuration time for new user
2 hours for an experienced user
20 minutes to set up a new project
It becomes more valuable with use
Less than half the cost of traditional testing
73
Agile Testing Benefits Reduces the cost-of-change by 10 times Frequent builds dramatically lower defect levels Enables early “what-if” tests as well as late changes
Fredrick, J. (2008). Accelerate software delivery with continuous integration and testing. Proceedings of the Sixth Japan Symposium on Software Testing (JASST 2008), Tokyo, Japan.
þ 36% reduction in defect ratewhen integration/regression testing at each code check-in
þ 90% reduction in bugs reaching QAMajor municipal gas utility
þ
þ
þ
þ
þ
95% cut in cost of bugsLarge retail web site
90% cut in defect remediation costGlobal supplier of healthcare equipment
Faster time-to-marketMore features and higher quality
Agility in the marketplaceAdded new functionality 2 weeks before ship
Confidence in the process“Oozing Confidence”
74
Agile Testing is NextGen Manual testing is CMMI Capability Level 0 or 1 Agile testing is a CMMI Capability Level 5 practice It is planned, defined, measured, and it’s optimizing
Fredrick, J. (2008). Accelerate software delivery with continuous integration and testing. Proceedings of the Sixth Japan Symposium on Software Testing (JASST 2008), Tokyo, Japan.
75
Agile Testing Side-Effects Eliminates big-bang integration in the 11th hour Creates a repeatable and reliable testing process Evaluates system-wide changes throughout project
Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley.
76
AgendaIntroductionSystems Engineering
Systems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering Testing
Agile Systems Engineering ValueSummary
77
What is Business Value? Val-ue (văl-'yōō): An amount, quantity, rate,
magnitude, or desirability; Economic worth An economic estimation of the tangible worth of the
organizational assets such as buildings and equipment An appraisal of intangible assets such as knowledge,
experience, skills, patents, processes, and methods A technique for evaluating the costs and benefits of
investments in a business, operations, or personnel The economic impact of deploying a new product
development approach such as systems engineering The total life cycle costs of institutionalizing lean
and agile systems engineering techniques in an enterprise
78
Agile (138 pt.) and traditional methods (99 pt.) Agile methods fare better in all benefits categories Agile methods 359% better than traditional methods
Agile Methods Traditional MethodsLow Median HighCategory
ROI 240% 2,633% 8,852%
Satisfaction 70% 70% 70%
Quality 10% 70% 1,000%
Productivity 14% 122% 712%
Schedule 11% 71% 700%
Cost 10% 26% 70%
Low Median HighCategory
ROI 200% 470% 2,770%
Satisfaction -4% 14% 55%
Quality 7% 50% 132%
Productivity 9% 62% 255%
Schedule 2% 37% 90%
Cost 3% 20% 87%
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
Studies of Agile Methods
79
Productivity of Agile Methods PP productivity 32X more than trad. methods Scrum productivity 5X more than trad. methods Agile methods productivity 20X more than traditional
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
33.4044
29.28
21.2374
16.1575
5.4436
1.0619
0
8
16
24
32
40
PP TDD Agile XP Scrum CMMI®
Software Method
Pro
du
cti
vit
y (
LO
C/H
ou
r)
80
Quality of Agile Methods XP quality 13X better than trad. methods Scrum quality 3X better than trad. methods Agile methods quality 5X better than traditional
0.7466
1.79722.155 2.355
3.945
9.667
0
2.2
4.4
6.6
8.8
11
XP Agile TDD PP Scrum CMMI®
Software Method
Qu
ality
(D
efe
cts
/KL
OC
)
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
81
Costs of Agile Methods XP costs 8X less than traditional methods Scrum costs 2X less than traditional methods Agile methods cost 5X less than traditional methods
$136,551
$226,807 $249,653 $265,436
$578,202
$1,108,233
$0
$325,000
$650,000
$975,000
$1,300,000
XP Agile TDD PP Scrum CMMI®
Software Method
Co
sts
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
82
Benefits of Agile Methods XP benefits 1.5X more than traditional methods Scrum benefits 1.3X more than traditional methods Agile methods benefits 1.4X more than trad. methods
$4,372,282$4,282,026 $4,259,180 $4,243,397
$3,930,631
$3,023,064
$2,750,000
$3,237,500
$3,725,000
$4,212,500
$4,700,000
XP Agile TDD PP Scrum CMMI®
Software Method
Be
ne
fits
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
83
ROI of Agile Methods XP ROI 18X more than traditional methods Scrum ROI 3.4X more than traditional methods Agile methods ROI 10X more than trad. methods
3,102%
1,788%1,606%
1,499%
580%
173%
0%
925%
1,850%
2,775%
3,700%
XP Agile TDD PP Scrum CMMI®
Software Method
Re
turn
on
In
ve
stm
en
t
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
84
NPV of Agile Methods XP NPV 2.4X more than traditional methods Scrum NPV 1.9X more than traditional methods Agile methods NPV 2.3X more than trad. methods
$3,649,388$3,480,979 $3,438,351 $3,408,902
$2,825,313
$1,509,424
$1,000,000
$1,787,500
$2,575,000
$3,362,500
$4,150,000
XP Agile TDD PP Scrum CMMI®
Software Method
Ne
t P
rese
nt
Va
lue
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
85
Real Options of Agile Methods XP ROA 1.6X more than traditional methods Scrum ROA 1.4X more than traditional methods Agile methods ROA 1.6X more than trad. methods
$4,265,936$4,105,884 $4,066,678 $4,040,377
$3,608,772
$2,633,052
$2,200,000
$2,825,000
$3,450,000
$4,075,000
$4,700,000
XP Agile TDD PP Scrum CMMI®
Software Method
Re
al O
pti
on
s
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.
86
AgendaIntroductionSystems Engineering
Systems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering Value
Summary
87
Summary Agility is the evolution of management thought Confluence of traditional and non-traditional ideas Improve performance by over an order-of-magnitude
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods: Maximizing ROI with just-in-time processes and documentation. Ft. Lauderdale, FL: J. Ross Publishing.
þ
þ
þ
þ
þ
þ
þ
Systems engineering approaches
Agile methods and practices are ...
New product development approaches
Expertly designed to be fast and efficient
Intentionally lean and free of waste (muda)
Systematic highly-disciplined approaches
Capable of producing high quality systems
Right-sized, just-enough, and just-in-time tools
þ Intended to maximize business value for customers
88
New Book on Agile Methods Guide to Agile Methods for business leaders Communicates business value of Agile Methods Rosetta stone to Agile Methods for traditional folks
http://davidfrico.com/agile-book.htm (Description) http://www.amazon.com/dp/1604270314 (Amazon)
Table of Contents 1. Introduction to Agile Methods 2. Values of Agile Methods 3. History of Agile Methods 4. Antecedents of Agile Methods 5. Types of Agile Methods 6. Practices of Agile Methods 7. Agile Project Management 8. Agile Software Engineering 9. Agile Support Processes10. Agile Tools and Technologies11. Comparison of Agile Methods12. Agile Metrics and Models13. Surveys of Agile Methods14. Costs-Benefits of Agile Methods15. ROI Metrics of Agile Methods16. Measures of Agile Methods17. Costs of Agile Methods18. Benefits of Agile Methods19. ROI of Agile Methods20. NPV of Agile Methods21. Real Options of Agile Methods22. Business Value of Agile Methods23. Agile vs. Traditional Methods24. Future of Agile Methods