© 2003 by carnegie mellon university page 1 process principles for cots-based systems 5 september...
Post on 19-Dec-2015
215 views
TRANSCRIPT
© 2003 by Carnegie Mellon University page 1
Process Principles for COTS-Based Systems
5 September 2003
Software Engineering InstituteCarnegie Mellon UniversityPittsburgh PA 15213
Sponsored by the U.S. Department of Defense
Tricia Oberndorf
© 2003 by Carnegie Mellon University page 2
Agenda
• COTS Background
• A/APCS Framework
• CMMI Background
• Implications of Using CMMI for COTS-Based Systems
© 2003 by Carnegie Mellon University page 3
COTS Background
Building systems today:• Few entirely custom-built to order• Commercial products expected to play major role
Diverse things influence the shape and function of a COTS-based system (CBS):• Stakeholder needs• Product characteristics and marketplace dynamics• Degree of interaction with legacy systems
Together, these compel many changes in system development and maintenance methods for CBSs.
© 2003 by Carnegie Mellon University page 4
What is a COTS-based system?A COTS product is one that is …• Sold, leased, or licensed to the general public• Offered by a vendor trying to profit from it• Is supported and evolved by the vendor,
who retains intellectual property rights• Available in multiple, identical copies• Used without modification of its internals
A COTS-based system is one that … • Uses one or more off-the-shelf
components from one or more suppliers plus any needed custom components
• Is integrated to achieve new or expanded system functionality
Lega
cy
Reuse
ass
ets
ComponentsComponents
COTS-Based System
COTS-Based System
??????
COTS
pro
duct
sCus
tom
© 2003 by Carnegie Mellon University page 5
Conceptual Bases: Requirements1
“Nail down the requirements first” will not work – the marketplace will not cooperate• Think of it as a new sphere of influence:
Constraints come from stakeholder needsAND marketplace imperatives
System that is created mustaccommodate competing sources of gravity
© 2003 by Carnegie Mellon University page 6
Conceptual Bases: Requirements2
Requirements (cont.)• Must distinguish between negotiable and non-
negotiable• Keep the non-negotiable set as small as possible• Understand how to prioritize and trade-off the
negotiables
A risk-driven spiral approach is key:• Frequent use of prototypes• Considerable interaction with system’s end users• Gradual refinement of understanding of system
© 2003 by Carnegie Mellon University page 7
Conceptual Bases: Knowledge
CBS development and maintenance is dependent on an evolving Body of Knowledge.
Architecture anddesign constraints
End-userbusiness processes
Acquisitionconstraints
BoK
Prioritized negotiables Marketplace offerings
Evolving System View
Non-negotiables
Legacy system context
Risks
© 2003 by Carnegie Mellon University page 8
Marketplace Affects CBS Approach
Traditional Engineering Approach
Requirements
Architecture & Design
Implementation
Requirements-driven Negotiation-driven
Required CBS Approach
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
© 2003 by Carnegie Mellon University page 9
The Process Framework
Based on these realizations and concepts, we have defined a framework that sets out the basics of a process that would be appropriate for the creation of a COTS-based system:
A/APCS: The Assembly/Acquisition Process for
COTS-based Systems
© 2003 by Carnegie Mellon University page 10
Process Overview
Three classes of activities:• Iterative: short-term, engineering-oriented
- Discovery: gather and refine system knowledge- Assembly: construct a prototype- Assessment: determine success of iteration & plan
• Pervasive: long-term, organizational scope- e.g., CM, license management, vendor relationship
management, contract tracking and oversight• Executive: event-driven, deal with decision making
- e.g., cost estimating, contract negotiation, project oversight
Today we focus on the Iterative.
© 2003 by Carnegie Mellon University page 11
Discovery
Gather and Refine activities occur simultaneously.
Gather:• Sources take many forms.
- Stakeholders, business processes, legacy systems, COTS marketplace
• Each is independent of the others.• There is no optimal order.
Refine:• Analysis and harmonization of the gathered knowledge• Yields the technical definition of the emerging system• Finds gaps, negotiates conflicts
© 2003 by Carnegie Mellon University page 12
Discovery Activities
Data from: stakeholders products …
RefineGather
Negotiate conflicts
Gap - need more data
Continue Discovery
Knowledge is sufficient for constructing executable
Go on?
Harmonized data
Mismatch
Agreement
Gather data zGather data y
Gather data xAnalyze
© 2003 by Carnegie Mellon University page 13
Assembly
Produces a prototype that reflects what’s been discovered
Reflects a traditional sequential process• Guided by local “ candidate requirements”
- Iteration Objectives, Detailed Iteration Plan plus hypotheses and desired behaviors expect to derive from prototype
- Vets “candidate requirements”• Produces an executable version of full system
- Likely to have less than complete functionality- Executable, not paper or specification or small piece
Does not preclude prototyping “in the small” throughout
© 2003 by Carnegie Mellon University page 14
Evaluation and Assessment
Occur at two levels:• Evaluation of the prototype in its own context
- Measures actual outcome against expected outcome- Considers degree to which prototype satisfies its
local “requirements”• Assessment of the iteration itself
- Addresses issues at project (not iteration) level- Considers whether objectives were good ones- Determines whether iteration results indicate
changes in project schedule, budget, etc.
© 2003 by Carnegie Mellon University page 15
On-going Iterations of A/APCS
Prototype
Deployed System
Body of Knowledge
Body ofKnowledge
BoK
© 2003 by Carnegie Mellon University page 16
Key Aspects of COTS Approach
• Simultaneous Definition and Trades• Parallel Business Process Engineering• Negotiable Requirements• Flexible Architecture• Current Market Knowledge• Integral Programmatics and Risks• Continuous Stakeholder Involvement• Disciplined Spiral or Iterative Practices• Frequent Executables
© 2003 by Carnegie Mellon University page 17
CMMIGuidance for improving an organization’s processes and ability to develop, acquire, and maintain products or services
Provides guidance for developing processes • is not processes or process descriptions • will not map one to one with the processes in your organization
Integrates four disciplines (bodies of knowledge)• System Engineering• Software Engineering• Integrated Product and Process Development• Supplier Sourcing
Expects all practices to be interpreted using knowledge of• CMMI• the organization • the business environment
CMMI process areas must be interpreted for each targeted domain
© 2003 by Carnegie Mellon University page 18
Presentation Format
Work Process Implications• High-level guidance for selected CMMI Process Areas or
disciplines- Highlight particular relevance- Provide unique interpretation- Suggest new process areas- Correct misconceptions
• Guidance for additional CMMI Process Areas for later reference
• Important process considerations for a COTS-based system approach
Take-away message for each aspect
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
© 2003 by Carnegie Mellon University page 19
Simultaneous Definition and Trades
Selected Work Process Implications• Decision Analysis and Resolution – continuous stakeholder negotiations require
robust, agreed-upon decision processes• Technical Solution – alternative solutions (including COTS product selection) are
analyzed continuously to accommodate new information• Risk Management – a key project risk is over-defining one sphere without adequate understanding of
implications on other spheres • Project Planning – project activities start concurrently with extensive interaction among them from
project start until the system is retired • Verification and Validation – continuous determination that information in each sphere is sufficient,
complete, and meets operational needs
• Balanced consideration of four diverse spheres of influence- Simultaneous discovery of information in each- Decisions in one inform and constrain decisions in others- Continuous identification and analysis of trades- Trades negotiated as definition of the solution evolves
Continuously reconcile what stakeholders want with what is available
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
© 2003 by Carnegie Mellon University page 20
Parallel Business Process Engineering
Selected Work Process Implications
• CMMI does not address changes to end-user business processes; expand concepts in Organizational Process Focus to plan and implement end-user business process improvement
• Organizational Environment for Integration – a shared vision of success among stakeholders, with suitable incentives and leadership, is critical to aligning business processes with alternative solutions
• Project Planning/Integrated Project Management for IPPD – implementing agreed upon business processes must be integrated in system planning
• End-users must be willing/able to change business processes to match those in COTS products
• Business process changes must be explicitly managed and coordinated as part of the project
• Business processes may continue to change with new COTS releases or new COTS products
Align business process engineering with system engineering
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
© 2003 by Carnegie Mellon University page 21
Negotiable Requirements
Selected Work Process Implications
• Requirements Development – must prioritize requirements to aid tradeoffs- stakeholders agree on minimum set of critical “must-have” requirements - evolvability is a high-priority, required quality attribute
• Requirements Management – disciplined and controlled management of requirements begins at project start to track identified and negotiated trades
• Project Planning/Integrated Project Management for IPPD – managing the project to encourage and reinforce the continual discovery of requirements while establishing sufficient stability to deliver a solution is challenging
• Requirements must be flexible enough to leverage available and projected COTS products
• Committing to a requirement is premature until COTS product behavior is understood
• As marketplace continues to change, requirements must be renegotiated
Requirements formation is a journey of discovery
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
© 2003 by Carnegie Mellon University page 22
Flexible Architecture
Selected Work Process Implications
• Technical Solution – each COTS product release requires evaluation and analysis of alternatives to support any incorporation decision-Describe behavior and linkage among system components for each
• Product Integration – composing and evaluating executable representations from project start is critical to verify and validate architecture suitability and evolvablity
• Project Planning/Integrated Project Management for IPPD – appropriate skills and resources are necessary to form, evaluate, and maintain system flexibility
- include effort to create and maintain “wrappers” or “glue” and re-integrate solution as COTS products change
• Architecture is considered early; evolves and is maintained until the system retired
• Potential drivers of change accommodated in architecture
Architecture is created and maintained as a corporate asset
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
© 2003 by Carnegie Mellon University page 23
Current Market Knowledge
Selected Work Process Implications
• Supplier Agreement Management – license agreements with COTS vendors must be negotiated to meet project needs
• Integrated Supplier Management – partnering with key vendors is critical (not just supplier management)- vendors seldom allow process monitoring; use hands-on evaluation- relationships with vendors’ other customers helps to amplify leverage
• Technical Solution – modification of COTS products introduces long term maintenance considerations and sizeable risk to project; avoid if possible
• Project Planning – significant resources may be required to monitor the marketplace and conduct COTS product evaluation; including experimentation facilities
• Anticipate and track changes to relevant market segments until system retired
• Anticipate and prototype system changes from updates to COTS products critical to the system
• Influence (not direct) COTS product changes, technology investments, and standards development
Marketplace is proactively monitored
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
© 2003 by Carnegie Mellon University page 24
Integral Programmatics and Risks
Selected Work Process Implications
• Technical Solution – engineering trades should include risk, cost, schedule and other programmatic factors associated with each alternative solution
• Project Planning – estimates of work product and task attributes should be generated for each alternative
• Analysis of alternative solutions includes team skills and expertise, cost, schedule, and associated risks for - building, fielding, and supporting the system - implementing any needed changes to business processes
(functional, operational, support)
Programmatic factors shape technical alternatives
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
© 2003 by Carnegie Mellon University page 25
Continuous Stakeholder Involvement
Selected Work Process Implications
• IPPD discipline – integrated teaming among disparate stakeholders throughout the development and maintenance is essential
• Validation – end users must always be involved in validating solution suitability
• Integrated Project Management for IPPD – accommodates resources for stakeholder involvement
- necessary changes to end-user operational processes must be explicitly and continuously managed and coordinated with solution development
Required stakeholder commitment may be unprecedented
• Significant commitment from all stakeholders required- identify, evaluate, and select alternative solutions - confirm results of any and all negotiations - agree evolving system meets their needs
• Stakeholders must reflect full diversity of interests
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
© 2003 by Carnegie Mellon University page 26
Disciplined Spiral or Iterative Practices• Continuously determine a compatible and feasible
set of: business processes, requirements, plans, design, COTS products, and other components
- Enterprise business objectives drive solution definition- Risk considerations drive degree of detail - Marketplace dynamics drive development and maintenance
processes
Spiral development facilitates developing a viable solution
Selected Work Process Implications
• Project Planning – if not already implemented, extensive effort may be needed to revamp planning and engineering processes for a spiral development approach
• Risk Management – tracking effectiveness of risk mitigation is key- highest priority remaining risks should be used to (re) direct and manage
the project
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
© 2003 by Carnegie Mellon University page 27
Frequent Executables• Frequent executable representations reduce risk
and reduce misunderstandings- Provide critical insight into the solution’s behavior- Explore critical system attributes - Validate end user business process - Verify technical viability
Executable representations demonstrate stakeholder buy-in
Selected Work Process Implications
• Requirements Development, Technical Solution, and Product Integration – each iteration should produce an executable representation reflecting current understanding of requirements, COTS products, business processes, and alternative designs explored and negotiated
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
© 2003 by Carnegie Mellon University page 28
Conclusion
CBS:• changes many things about your approach• requires even more discipline than custom
development
Although it needs interpretation, CMMI is a sound basis for CBS process improvement. • Applying CMMI to CBSs is more than just Supplier
Management:- All four disciplines- All process area categories
• Process areas will need to be integrated and applied differently
• Must add in your own activities to plan, design, and deploy changes to enterprise processes
© 2003 by Carnegie Mellon University page 29
For more information:
TR coming: CMU/SEI-2003-TR-022
www.sei.cmu.edu/cbs
Tricia Oberndorf Lisa BrownswordDirector, Dynamic Systems Sr. MTS, CBS (ISIS)412-268-6138 [email protected] [email protected]