university of southern california center for software engineering cse usc engineering systems at...
TRANSCRIPT
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
Engineering Systems at USC:Center for Systems and Software Engineering
Barry Boehm
CESUN University Roundtable
April 2, [email protected]; http://csse.usc.edu
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
04-02-2008 ©USC-CSE 2
Outline
• CSE, SAE, and CSSE
• CSSE scope, objectives, and strategy
• Research and education examples
• Recent major developments– Integrating systems and software engineering– Value-based systems and software engineering
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
CSE, SAE, and CSSE• Center for Software Engineering (CSE)
– Large research program, Affiliates’ program– MSCS-SwE and PhD program
• Systems Architecting and Engineering (SAE) Program– Large MS program– Plans for PhD, Affiliates’ program
• Proposal to Affiliates– Set up SAE Affiliates’ program– Membership: $25K/year for either; $40K/year for both
• Affiliates’ response– We’re integrating SysE and SwE; you should do this too
• Result: Center for Systems and Software Engineering– Membership: $35K/year ($5K/year for small organizations)
04-02-2008 ©USC-CSE 3
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
10-08-2007 ©USC-CSE 4
Research, Transition, and Education Strategies
• Stakeholder Win-Win– Principals, affiliates, project clients, practitioners,
students• Opportunistic collaboration
– Focus on high-impact current, future problem areas– Principals’ criterion: shared interest in SysE/SwE
• Commercial or open-source versions of tools• Research/education integration via real-client projects
– Primarily USC campus, neighborhood e-services– Validate new methods and tools via project usage– Partial basis of 11 PhD dissertations
– Rqts. negotiation, formalization (2), COTS integration (2), Value-based methods (2), Agile methods (1), Quality tradeoffs (1), Risk analysis (1), Cost estimation (1), Testbeds (1)
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
10-08-2007 ©USC-CSE 5
Why Software Projects Fail
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
10-08-2007 ©USC-CSE 6
“Software Engineering:” The disciplines which distinguish the coding of a computer program from the development of a software system product
Requirements,
Architecture
Design,
Code
Implement,
Maintain
Computer Science CS Focus
User Applications
Economics
People
• Accommodate new tools and techniques•Web apps, Architecting tools, WikiWinWin, Spiral processes
• Integrate all these considerations–Via Incremental Commitment Model (ICM)–Necessarily emphasizes systems architecting, engineering
Stages
Issues
- Involves 9 times as much effort [Brooks, 1975]
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
2007-08 Software Engineering ProjectsBox Office Database System Jay McAdams 24th Street Theatre
Online Peer Review System Stephen Bucher USC, Engineering Writing Program
Thai CDC Software Tool Pheel Wang Thai Community Development Center
USC COINCOMO Ed Colbert, Vu Nguyen USC CSSE
Crystal Stairs Dashboard Rick Capella Crystal Stairs, Inc.
Family & Homeless Well-Being Project Jill Remelski St. Francis Center (SFC)
BTI Appraisal Projects Megan Tunnell O'Rourke BTI Appraisal
LAMAS Customer Service Application Celia Castellanos Los Angeles Music and Art School
Web-based service for TPC Foundation Pat Means Turning Point URBAN Magazine
REEO Database Scott Stimpfel Resources for Educational and Employment Opportunities
BID review System Keith Culling Background Investigation Division, LA city
THSA Website Project Poonchana Thitametakul USC THSA
EEO Section Database Art Irigoyen LA city
Conference Room Reservation System Linda Lanier LA city
Applicant-processing systems for CLAPD Dee Dee Coultas LA city
E-Mentoring program Carlene Davis Los Angeles Urban League
Social Networking Tools for Librarians Julie Kwan Social Networking Tools for Librarians
EZSIM II Ray Madachy USC-CSSE
Los Angeles County Generation Web Initiative Frank Cheng, Jeramy Gray LA County
Development Of Hunter-gatherer Database Chris Boehm USC
10-08-2007 ©USC-CSE 7
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
4/18/06 ©USC-CSE 8
First Six Weeks of Project Course
Domain Model
WinWin Taxonomy
Basic Conceptof Operation
Frequent
RisksStakeholders,
Primary win conditions
WinWin Negotiation
Model
IKIWISI Model,Prototypes,
Properties Models
EnvironmentModels
WinWin Agreements, Shared Vision
ViableArchitecture
Options
Updated Conceptof Operation
Life Cycle Planelements
Outstanding LCO risks
RequirementsDescription
LCO Rationale
Life Cycle Objectives (LCO) Package
Anchor PointModel
determinesidentifiesidentifiesdetermines
situates exercise exercise focususe of
focus use of determines
guidesdetermination of validate
inputs for
provides
initialize adopt identify identify
update update
achieveiterate to feasibility, consistency determines exit
criteria for validates readiness of
initializes
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
CSSE Principals (37)
• Core in CS, SAE, ISE (20)• Engineering: Aerospace, Civil, Electrical,
Mechanical (8)• Schools of Business, Public Policy (4) • Information Sciences Institute, Institute for
Creative Technology, CREATE Center (5)• 7 NAE members; 4 INCOSE Fellows; 9 IEEE
Fellows
10-08-2007 ©USC-CSE 9
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
Major Research Projects• Integrating systems and software engineering (OSD-SSE)
– 2007: Inhibitors and enablers study– 2008: IS&SE via Incremental Commitment Model guidebook
• Systems of systems engineering (Army FCS, OSD-SSE)– 2001- : Future Combat Systems S&SE support– 2007- : OSD-SSE Systems of Systems Guidebook support– 2007: DACS system of systems cost estimation framework
• Next-generation space systems (DARPA, NASA)– 2003- : Software reliability; processes; risk assessment
• Scaling up agile methods (Affiliates)– 2003- : Architecture-based agile methods
• Value-based science of design (NSF)– 2003- : Value-based theories of software, systems engineering
04-02-2008 ©USC-CSE 10
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
08/31/06 ©USC-CSE 11
FCS WinWin Spiral Model1b. StakeholdersIdentify SystemObjectives, Constraints,& Priorities (OC&Ps);Alternative SolutionElements
1a. IdentifySuccess-CriticalStakeholders
2a. EvaluateAlternativeswith respect toOC&Ps
2b.Assess,AddressRisks
3. ElaborateProduct andProcessDefinition4. Verify and Validate
Product and ProcessDefinitions
Stakeholders’
Commitment4
5
6
8
2
1
Stakeholders’Review
7
3
L COL CA
Build 2
Build 3
Progress Through Steps
Bl1
Driven By:
Success-critical
stakeholders’ win
conditions
Risk Management
Spiral anchor point
milestones
Feasibility Rationale
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
10-08-2007 ©USC-CSE 12
How much Architecting is Enough?- A COCOMOII Analysis
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60
Percent of Time Added for Architecture and Risk Resolution
Per
cen
t of T
ime
Ad
ded
to O
vera
ll S
ched
ule
Percent of Project Schedule Devoted to Initial Architecture and Risk Resolution
Added Schedule Devoted to Rework(COCOMO II RESL factor)
Total % Added Schedule
10000KSLOC
100 KSLOC
10 KSLOC
Sweet Spot
Sweet Spot Drivers:
Rapid Change: leftward
High Assurance: rightward
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
10-08-2007 ©USC-CSE 13
Tradeoffs Among Cost, Schedule, and Reliability:Want $5.5M, 20 months, 10K hours MTBF
0
1
2
3
4
5
6
7
8
9
0 10 20 30 40 50
Development Time (Months)
Co
st
($M
)
(VL, 1)
(L, 10)
(N, 300)
(H, 10K)
(VH, 300K)
-- Cost/Schedule/RELY:
“pick any two” points
(RELY, MTBF (hours))
•Cost, schedule, quality: pick any two?•For 100-KSLOC set of features•Can “pick all three” with 77-KSLOC set of features
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
08/31/06 ©USC-CSE 14
COSYSMO
SizeDrivers
EffortMultipliers
Effort
Calibration to 42 projects
# Requirements# Interfaces# Scenarios# Algorithms
+3 adjustment factors
- Application factors-8 factors
- Team factors-6 factors
COSYSMO Operational Concept
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
04-02-2008 ©USC-CSE 15
Outline
• CSE, SAE, and CSSE
• CSSE scope, objectives, and strategy
• Research and education examples
• Recent major developments– Integrating systems and software engineering– Value-based systems and software engineering
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008 ©USC-CSSE 16
2007 OSD-SSE Study Scope and Context• Statement of Work
– Evaluate current DoD SysE, SwE processes and standards• Primarily SEP Guide, DAG Ch. 4 (SysE), DoDI 5000.2• Identify deltas, issue areas, recommendations
– Evaluate ICM for integration applicability– Evaluate relevant DoD guidance, education, and training
• Provide recommendations based on evaluations• Context: current and future DoD S&SE challenges
– Multi-owner, multi-mission systems of systems– Rapid pace of change– Always-on, never-fail systems– Need to turn within adversaries’ OODA loop
• Observe, orient, decide, act– Within asymmetric adversary constraints
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
• An omniscient authority pre-establishes the requirements, preferred system concept, etc. (A4.1, 4.2, 4.4)
• Technical solutions are mapped and verified with respect to these (A4.1, 4.2, 4.4)
• They and technology do not change very often (A4.2, 4.5, 6.2)– Emphasis on rigor vs. adaptability
• The program has stable and controllable external interfaces (A4.5)• Project organization is function-hierarchical and WBS-oriented
(A3.1, 3.2)• All requirements are equally important (A4.2)• Reviews event/product-based vs. evidence-based (A5.2)• The program’s critical path can be defined up front (A6.1)• Can achieve optimum readiness at minimum life-cycle cost (A6.5)
– No slack for contingencies03/19/2008 ©USC-CSSE 17
Current SysE Guidance: Outdated Assumptions
Example: SysE Plan Preparation GuideSometimes OK for hardware; generally not for software
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008 ©USC-CSSE 18
System/Software Architecture Mismatches- Maier, 2006
• System Hierarchy
– Part-of relationships; no shared parts
– Function-centric; single data dictionary
– Interface dataflows
– Static functional-physical allocation
• Software Hierarchy
– Uses relationships; layered multi-access
– Data-centric; class-object data relations
– Interface protocols; concurrency challenges
– Dynamic functional-physical migration
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008 ©USC-CSSE 19
• Fractionated, incompatible sensor data management
• “Touch Football” interface definition earned value– Full earned value taken for defining interface dataflow
– No earned value left for defining interface dynamics• Joining/leaving network, publish-subscribe, interrupt handling,
security protocols, exception handling, mode transitions
– Result: all green EVMS turns red in integration
Examples of Architecture Mismatches
…
Sensor 1
SDMS1
Sensor 2
SDMS2
Sensor 3
SDMS3
Sensor n
SDMSn
……
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008 ©USC-CSSE 20
The Incremental Commitment Life Cycle Process: OverviewStage I: Definition Stage II: Development and Operations
Anchor Point Milestones
Anchor Point Milestones
Synchronize, stabilize concurrency via FEDsSynchronize, stabilize concurrency via FEDs
Risk patterns determine life cycle process
Risk patterns determine life cycle process
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008 ©USC-CSSE 21
Anchor Point Feasibility Evidence Description
• Evidence provided by developer and validated by independent experts that:
If the system is built to the specified architecture, it will– Satisfy the requirements: capability, interfaces, level of
service, and evolution
– Support the operational concept
– Be buildable within the budgets and schedules in the plan
– Generate a viable return on investment
– Generate satisfactory outcomes for all of the success-critical stakeholders
• All major risks resolved or covered by risk management plans
• Serves as basis for stakeholders’ commitment to proceed
Can be used to strengthen current schedule- or event-based reviews
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008 ©USC-CSSE 22
Incremental Commitment in Gambling
• Total Commitment: Roulette– Put your chips on a number
• E.g., a value of a key performance parameter– Wait and see if you win or lose
• Incremental Commitment: Poker, Blackjack– Put some chips in– See your cards, some of others’ cards– Decide whether, how much to commit to proceed
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008 ©USC-CSSE 23
Scalable remotely controlled operations
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008 ©USC-CSSE 24
Total vs. Incremental Commitment – 4:1 RPV
• Total Commitment– Agent technology demo and PR: Can do 4:1 for $1B
– Winning bidder: $800M; PDR in 120 days; 4:1 capability in 40 months
– PDR: many outstanding risks, undefined interfaces
– $800M, 40 months: “halfway” through integration and test
– 1:1 IOC after $3B, 80 months
• Incremental Commitment [number of competing teams]– $25M, 6 mo. to VCR [4]: may beat 1:2 with agent technology, but not
4:1
– $75M, 8 mo. to ACR [3]: agent technology may do 1:1; some risks
– $225M, 10 mo. to DCR [2]: validated architecture, high-risk elements
– $675M, 18 mo. to IOC [1]: viable 1:1 capability
– 1:1 IOC after $1B, 42 months
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008 ©USC-CSSE 25
The Incremental Commitment Life Cycle Process: OverviewStage I: Definition Stage II: Development and Operations
Anchor Point Milestones
Anchor Point Milestones
Concurrently engr. OpCon, rqts, arch, plans, prototypes
Concurrently engr. OpCon, rqts, arch, plans, prototypes
Concurrently engr. Incr.N (ops), N+1 (devel), N+2 (arch)
Concurrently engr. Incr.N (ops), N+1 (devel), N+2 (arch)
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008 ©USC-CSSE 26
ICM HSI Levels of Activity for Complex Systems
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008 ©USC-CSSE 27
Different Risk Patterns Yield Different Processes
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008
Common Risk-Driven Special Cases of the ICMSpecial Case Example Size,
ComplexityChange Rate %/Month
Criticality NDI Support Org, Personnel Capability
Key Stage I Activities : Incremental Definition Key Stage II Activities: Incremental Development, Operations
Time per Build; per Increment
1. Use NDI Small Accounting Complete Acquire NDI Use NDI
2. Agile E-services Low 1 – 30 Low-Med Good; in place
Agile-readyMed-high
Skip Valuation , Architecting phases Scrum plus agile methods of choice <= 1 day; 2-6 weeks
3. Architected Agile
Business data processing
Med 1 – 10 Med-High Good; most in place
Agile-readyMed-high
Combine Valuation, Architecting phases. Complete NDI preparation
Architecture-based Scrum of Scrums 2-4 weeks; 2-6 months
4. Formal Methods
Security kernel; Safety-critical LSI chip
Low 0.3 Extra High None Strong formal methods experience
Precise formal specification Formally-based programming language; formal verification
1-5 days;1-4 weeks
5. HW component with embedded SW
Multi-sensor control device
Low 0.3 – 1 Med-Very High
Good; In place
Experienced; med-high
Concurrent HW/SW engineering. CDR-level ICM DCR
IOC Development, LRIP, FRP. Concurrent Version N+1 engineering
SW: 1-5 days; Market-driven
6. Indivisible IOC Complete vehicle platform
Med – High
0.3 – 1 High-Very High
Some in place Experienced; med-high
Determine minimum-IOC likely, conservative cost. Add deferrable SW features as risk reserve
Drop deferrable features to meet conservative cost. Strong award fee for features not dropped
SW: 2-6 weeks;Platform: 6-18 months
7. NDI- Intensive Supply Chain Management
Med – High
0.3 – 3 Med- Very High
NDI-driven architecture
NDI-experienced; Med-high
Thorough NDI-suite life cycle cost-benefit analysis, selection, concurrent requirements/ architecture definition
Pro-active NDI evolution influencing, NDI upgrade synchronization
SW: 1-4 weeks; System: 6-18 months
9. Hybrid agile / plan-driven system
C4ISR Med – Very High
Mixed parts: 1 – 10
Mixed parts; Med-Very High
Mixed parts Mixed parts Full ICM; encapsulated agile in high change, low-medium criticality parts (Often HMI, external interfaces)
Full ICM ,three-team incremental development, concurrent V&V, next-increment rebaselining
1-2 months; 9-18 months
9. Multi-owner system of systems
Net-centric military operations
Very High Mixed parts: 1 – 10
Very High Many NDIs; some in place
Related experience, med-high
Full ICM; extensive multi-owner team building, negotiation
Full ICM; large ongoing system/software engineering effort
2-4 months; 18-24 months
10. Family of systems
Medical Device Product Line
Med – Very High
1 – 3 Med – Very High
Some in place Related experience, med – high
Full ICM; Full stakeholder participation in product line scoping. Strong business case
Full ICM. Extra resources for first system, version control, multi-stakeholder support
1-2 months; 9-18 months
C4ISR: Command, Control, Computing, Communications, Intelligence, Surveillance, Reconnaissance. CDR: Critical Design Review. DCR: Development Commitment Review. FRP: Full-Rate Production. HMI: Human-Machine Interface. HW: Hard ware. IOC: Initial Operational Capability. LRIP: Low-Rate Initial Production. NDI: Non-Development Item. SW: Software
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008 ©USC-CSSE 29
Example ICM Commercial Application:Symbiq Medical Infusion Pump
Winner of 2006 HFES Best New Design AwardDescribed in NRC HSI Report, Chapter 5
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
04-02-2008 ©USC-CSE 30
Outline
• CSE, SAE, and CSSE
• CSSE scope, objectives, and strategy
• Research and education examples
• Recent major developments– Integrating systems and software engineering– Value-based systems and software engineering (VBSEE)
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
06/25/07 © USC-CSE 31
VBSEE Motivation and Context
• INCOSE Definition of “Systems Engineering”– An interdisciplinary approach and means to enable the
realization of successful systems– A systems engineering theory should provide criteria and
guidance for realizing successful systems
• Addressing the Intellectual Content of Systems Engineering– How distinguished from other engineering disciplines– How related to other theories with intellectual content
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
06/25/07 © USC-CSE 32
Intellectual Content of Systems Engineering: Distinguishing Features
• The intellectual content of most engineering disciplines is component-oriented and value-neutral– Ohm’s Law, Hooke’s Law, Newton’s Laws
• The intellectual content of systems engineering is focused on:– How people and components can be combined into a cost-
effective system• System definition, architecting, analysis, planning,
evaluation, improvement• Cost-effectiveness in terms of value to stakeholders
• The ability to address value considerations is an essential qualification for a theory of systems engineering
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
06/25/07 © USC-CSE 33
Theory W: Enterprise Success Theorem– And informal proof
Theorem: Your enterprise will succeed if and only if
it makes winners of your success-critical stakeholders
• Proof of “if”:Everyone that counts is a winner.Nobody significant is left to complain.
• Proof of “only if”:
Nobody wants to lose.Prospective losers will refuse to participate, or will counterattack.The usual result is lose-lose.
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
06/25/07 © USC-CSE 34
Theory W: WinWin Achievement Theorem
Making winners of your success-critical stakeholders requires:
i. Identifying all of the success-critical stakeholders (SCSs).
ii. Understanding how the SCSs want to win.
iii. Having the SCSs negotiate a win-win set of product and process plans.
iv. Controlling progress toward SCS win-win realization, including adaptation to change.
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
06/25/07 © USC-CSE 35
VBSET Component Theories• Theory W (Stakeholder win-win)
– Enterprise Success Theorem, Win-Win Achievement Theorem
• Dependency Theory (Product, process, people interdependencies)– Systems architecture/performance theory, costing and
scheduling theory; organization theory
• Utility Theory– Utility functions, bounded rationality, Maslow need
hierarchy, multi-attribute utility theory
• Decision Theory– Statistical decision theory, game theory, negotiation theory,
Theory of Justice
• Control Theory– Observability, predictability, controllability, stability theory
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
08/2007 ©USC-CSSE 36
Initial VBSE Theory: 4+1 Process– With a great deal of concurrency and backtracking
Utility Theory
Theory W:SCS Win-Win
Decision Theory
Dependency Theory
Control Theory
6a, 7c. State measurement, prediction, correction; Milestone synchronization
5a. Investment analysis, Risk analysis
1. Protagonist goals3a. Solution exploration7. Risk, opportunity, change management
5a, 7b. Prototyping
2a. Results Chains3b, 5a, 7b. Cost/schedule/performance tradeoffs
2. Identify SCSs
3b, 7a. Solution Analysis
5a, 7b. Option, solution development & analysis
4. SCS expectations management
3. SCS Value Propositions(Win conditions)
SCS: Success-Critical Stakeholder
6, 7c. Refine, Execute, Monitor & Control Plans
5. SCS Win-Win Negotiation
Provides process for executing ICM Exploration phase
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
06/25/07 © USC-CSE 37
Value-Based Testing: Empirical Data and ROI— LiGuo Huang, ISESE 2005
-1.5
-1
-0.5
0
0.5
1
1.5
2
0 10 20 30 40 50 60 70 80 90 100
% Tests Run
Re
turn
On
In
ve
stm
en
t (R
OI)
Value-Neutral ATG Testing Value-Based Pareto Testing
% of Valuefor
CorrectCustomer
Billing
Customer Type
100
80
60
40
20
5 10 15
Automated test generation (ATG) tool
- all tests have equal value
Bullock data– Pareto distribution% of
Valuefor
CorrectCustomer
Billing
Customer Type
100
80
60
40
20
5 10 15
Automated test generation (ATG) tool
- all tests have equal value
% of Valuefor
CorrectCustomer
Billing
Customer Type
100
80
60
40
20
5 10 15
Automated test generation (ATG) tool
- all tests have equal value
Bullock data– Pareto distribution
(a)
(b)
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
04/19/07 38
By Number P-value % Gr A higher
By Impact P-value % Gr A higher
Average of Concerns
0.202 34 Average Impact of Concerns
0.049 65
Average of Problems
0.056 51 Average Impact of Problems
0.012 89
Average of Concerns per hour
0.026 55 Average Cost Effectiveness of Concerns
0.004 105
Average of Problems per hour
0.023 61 Average Cost Effectiveness of Problems
0.007 108
• Group A: 15 IV&V personnel using VBR procedures and checklists
• Group B 13 IV&V personnel using previous value-neutral checklists– Significantly higher numbers of trivial typo and grammar faults
Experiment: Reviews of Prioritized Ops Concept, Requirements, and Architecture DescriptionsExperiment: Reviews of Prioritized Ops Concept, Requirements, and Architecture Descriptions
Value-Based Reading (VBR) Experiment— Keun Lee, ISESE 2005
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008
Conclusions: IS&SE with ICM
• Current DoD (and other) SysE guidance seriously inhibits SwE– Largely sequential definition, design, development, integration, and test– Slows agility, ability to turn inside adversaries’ OODA loop– Functional “part-of” vs. layered “served by” product hierarchy– Static vs. dynamic interfaces: messages vs. protocols – One-size-fits-all process guidance inhibits balancing agility and assurance
• Incremental Commitment Model (ICM) enables better IS&SE– Integrates hardware, software, human factors aspects of SysE – Concurrent exploration of needs, opportunities, solutions
• Enabled by value-based systems engineering theory and process– Successfully addresses issues above in commercial practice– Only partially proven in DoD practice (examples: CrossTalk Top-5 projects)– Key practices applied to help major programs (example: Future Combat Systems)– Being adopted by major programs (example: Missile Defense Agency)– Effort underway to work with adopters to develop DoD ICM Guidebook, in
coordination with other USD(AT&L)SSE initiatives
©USC-CSSE 39
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
2008 SOW: DoD ICM Guidebook
• Develop a draft Guidebook for next-generation DoD IS&SE based on the ICM– In collaboration with DoD and industry participants
• Collaborate with selected DoD and industry parties in experimental tailoring of draft Guidebook elements
• Hold a series of Guidebook workshops– Mar 19-20 (USC), July 15-17 (DC area), Oct 29-30 (USC)
• Coordinate Guidebook with other IS&SE initiatives– Systems of systems engineering, systemic analysis,
acquisition, education, assessment, research and technology
03/19/2008 ©USC-CSSE 40
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008 ©USC-CSSE 41
References - IBeck, K., Extreme Programming Explained, Addison Wesley, 1999.Boehm, B., “Some Future Trends and Implications for Systems and Software Engineering Processes”,
Systems Engineering 9(1), pp. 1-19, 2006. Boehm, B., Brown, W., Basili, V., and Turner, R., “Spiral Acquisition of Software-Intensive Systems of
Systems, CrossTalk, Vol. 17, No. 5, pp. 4-9, 2004.Boehm, B. and Lane J., "21st Century Processes for Acquiring 21st Century Software-Intensive Systems
of Systems." CrossTalk: Vol. 19, No. 5, pp.4-9, 2006. Boehm, B., and Lane, J., “Using the ICM to Integrate System Acquisition, Systems Engineering, and
Software Engineering,” CrossTalk, October 2007, pp. 4-9.
Boehm, B., and Lane, J., “A Process Decision Table for Integrated Systems and Software Engineering,” Proceedings, CSER 2008, April 2008.
Boehm, B., “Future Challenges and Rewards for Software Engineers,” DoD Software Tech News, October 2007, pp. 6-12.
Boehm, B. et al., Software Cost Estimation with COCOMO II, Prentice Hall, 2000.Boehm, B., Software Engineering Economics, Prentice Hall, 2000.Carlock, P. and Fenton, R., "System of Systems (SoS) Enterprise Systems for Information-Intensive
Organizations," Systems Engineering, Vol. 4, No. 4, pp. 242-26, 2001.Carlock, P., and J. Lane, “System of Systems Enterprise Systems Engineering, the Enterprise
Architecture Management Framework, and System of Systems Cost Estimation”, 21st International Forum on COCOMO and Systems/Software Cost Modeling, 2006.
Checkland, P., Systems Thinking, Systems Practice, Wiley, 1980 (2nd ed., 1999).
Department of Defense (DoD), Defense Acquisition Guidebook, version 1.6, http://akss.dau.mil/dag/, 2006.
Department of Defense (DoD), Instruction 5000.2, Operation of the Defense Acquisition System, May 2003.
Department of Defense (DoD), Systems Engineering Plan Preparation Guide, USD(AT&L), 2004.
Electronic Industries Alliance (1999); EIA Standard 632: Processes for Engineering a System
Galorath, D., and Evans, M., Software Sizing, Estimation, and Risk Management, Auerbach, 2006.
Hall, E.T., Beyond Culture, Anchor Books/Doubleday, 1976.
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008 ©USC-CSSE 42
References -IIHighsmith, J., Adaptive Software Development, Dorset House, 2000.
International Standards Organization, Information Technology Software Life Cycle Processes, ISO/IEC 12207, 1995
ISO, Systems Engineering – System Life Cycle Processes, ISO/IEC 15288, 2002.
Jensen, R. “An Improved Macrolevel Software Development Resource Estimation Model,” Proceedings, ISPA 5, April 1983, pp. 88-92.
Krygiel, A., Behind the Wizard’s Curtain; CCRP Publication Series, July, 1999, p. 33Lane, J. and Boehm, B., "System of Systems Cost Estimation: Analysis of Lead System Integrator
Engineering Activities", Information Resources Management Journal, Vol. 20, No. 2, pp. 23-32, 2007.
Lane, J. and Valerdi, R., “Synthesizing SoS Concepts for Use in Cost Estimation”, Proceedings of IEEE Systems, Man, and Cybernetics Conference, 2005.
Lientz, B., and Swanson, E.B., Software Maintenance Management, Addison Wesley, 1980. Madachy, R., Boehm, B., Lane, J., "Assessing Hybrid Incremental Processes for SISOS Development",
USC CSSE Technical Report USC-CSSE-2006-623, 2006. Maier, M., “Architecting Principles for Systems-of-Systems”; Systems Engineering, Vol. 1, No. 4 (pp 267-
284).Maier, M., “System and Software Architecture Reconciliation,” Systems Engineering 9 (2), 2006, pp. 146-
159.Northrop, L., et al., Ultra-Large-Scale Systems: The Software Challenge of the Future, Software
Engineering Institute, 2006. Pew, R. W., and Mavor, A. S., Human-System Integration in the System Development Process: A New
Look, National Academy Press, 2007.Putnam, L., “A General Empirical Solution to the Macro Software Sizing and Estimating Problem,” IEEE
Trans SW Engr., July 1978, pp. 345-361.Rechtin, E. Systems Architecting, Prentice Hall, 1991.Schroeder, T., “Integrating Systems and Software Engineering: Observations in Practice,” OSD/USC
Integrating Systems and Software Engineering Workshop, http://csse.usc.edu/events/2007/CIIForum/pages/program.html, October 2007.
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008©USC-CSSE
List of AcronymsB/L Baselined
C4ISR Command, Control, Computing, Communications, Intelligence, Surveillance, Reconnaissance
CD Concept Development
CDR Critical Design Review
COTS Commercial Off-the-Shelf
DCR Development Commitment Review
DI Development Increment
DoD Department of Defense
ECR Exploration Commitment Review
EVMS Earned Value Management System
FCR Foundations Commitment Review
FDN Foundations, as in FDN Package
FED Feasibility Evidence Description
FMEA Failure Modes and Effects Analysis
FRP Full-Rate Production
GAO Government Accountability Office
GUI Graphical User Interface
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008©USC-CSSE
List of Acronyms (continued)
HMI Human-Machine Interface
HSI Human-System Interface
HW Hardware
ICM Incremental Commitment Model
IOC Initial Operational Capability
IRR Inception Readiness Review
IS&SE Integrating Systems and Software Engineering
LCA Life Cycle Architecture
LCO Life Cycle Objectives
LRIP Low-Rate Initial Production
MBASE Model-Based Architecting and Software Engineering
NDI Non-Developmental Item
NRC National Research Council
OC Operational Capability
OCR Operations Commitment Review
OO&D Observe, Orient and Decide
OODA Observe, Orient, Decide, Act
O&M Operations and Maintenance
University of Southern CaliforniaCenter for Software EngineeringC S E
USC
03/19/2008©USC-CSSE
List of Acronyms (continued)
PDR Preliminary Design Review
PM Program Manager
PR Public Relations
PRR Product Release Review
RUP Rational Unified Process
SoS System of Systems
SoSE System of Systems Engineering
SSE Systems and Software Engineering
SW Software
SwE Software Engineering
SysE Systems Engineering
Sys Engr Systems Engineer
S&SE Systems and Software Engineering
USD (AT&L) Under Secretary of Defense for Acquisition, Technology, and Logistics
VCR Validation Commitment Review
V&V Verification and Validation
WBS Work Breakdown Structure
WMI Warfighter-Machine Interface