© 2003 by carnegie mellon university page 1 processes for cots-based systems la spin 3 september...
Post on 19-Dec-2015
216 views
TRANSCRIPT
© 2003 by Carnegie Mellon University page 1
Processes for COTS-Based Systems
LA SPIN3 September 2003
Software Engineering InstituteCarnegie Mellon UniversityPittsburgh PA 15213
Sponsored by the U.S. Department of Defense
Tricia Oberndorf
© 2003 by Carnegie Mellon University page 2
Outline
• Background
• Process Overview
• High-level Process
• Example Low-level Activities Descriptions
• Converting This to a Process: EPIC
© 2003 by Carnegie Mellon University page 3
Background
Building systems today:• Few custom-built to order• Commercial products expected to play major role
Diverse things influence the shape and function of a COTS-based system (CBS):• Stakeholder needs• Characteristics of products and marketplace• Degree of interaction with legacy systems
Together, these compel many changes in system development and maintenance methods for CBSs.
© 2003 by Carnegie Mellon University page 4
COTS Spheres of Influence
VendorsMarket segmentsAvailable COTS ProductsEmerging technology
MarketplaceMarketplace
Use casesQuality attributesBusiness model
Stakeholder Needs/ Business Processes
Stakeholder Needs/ Business Processes
PatternsInterfacesInfrastructure
Architecture/ Design
Architecture/ Design
Risk managementProject management
Change managementLicense negotiation
Programmatics/Risk
COSTS
Programmatics/Risk
COSTS
© 2003 by Carnegie Mellon University page 5
Marketplace Affects CBS Approach
Traditional Engineering Approach
Requirements
Architecture & Design
Implementation
Requirements-driven Negotiation-driven
Required CBS Approach
Simultaneous Definition
and TradeoffsMarketplace
Stakeholder Needs/Business Processes
Architecture Design
Programmatics/Risk
© 2003 by Carnegie Mellon University page 6
Conceptual Bases: Requirements1
“Nail down the requirements first” will not work – the marketplace will not cooperate• Think of it as a new sphere of influence:
Constraints come from stakeholder needsAND marketplace imperatives
System that is created mustaccommodate competing sources of gravity
© 2003 by Carnegie Mellon University page 7
Conceptual Bases: Requirements2
Requirements (cont.)• Must distinguish between negotiable and non-
negotiable• Keep the non-negotiable set as small as possible• Understand how to prioritize and trade-off the
negotiables
A risk-driven spiral approach is key:• Frequent use of prototypes• Considerable interaction with system’s end users• Gradual refinement of understanding of system
© 2003 by Carnegie Mellon University page 8
Conceptual Bases: Knowledge
CBS development and maintenance is dependent on an evolving Body of Knowledge.
Architecture anddesign constraints
End-userbusiness processes
Acquisitionconstraints
BoK
Prioritized negotiables Marketplace offerings
Evolving System View
Non-negotiables
Legacy system context
Risks
© 2003 by Carnegie Mellon University page 9
Process Overview
Three classes of activities:• Iterative: short-term, engineering-oriented
- Discovery: gather and refine system knowledge- Assembly: construct a prototype- Assessment: determine success of iteration & plan
• Pervasive: long-term, organizational scope- e.g., CM, license management, vendor relationship
management, contract tracking and oversight• Executive: event-driven, deal with decision making
- e.g., cost estimating, contract negotiation, project oversight
Today we focus on the Iterative.
© 2003 by Carnegie Mellon University page 10
Discovery
Gather and Refine activities occur simultaneously.
Gather:• Sources take many forms.
- Stakeholders, business processes, legacy systems, COTS marketplace
• Each is independent of the others.• There is no optimal order.
Refine:• Analysis and harmonization of the gathered knowledge• Yields the technical definition of the emerging system• Finds gaps, negotiates conflicts
© 2003 by Carnegie Mellon University page 11
Discovery Activities
Data from: stakeholders products …
RefineGather
Negotiate conflicts
Gap - need more data
Continue Discovery
Knowledge is sufficient for constructing executable
Go on?
Harmonized data
Mismatch
Agreement
Gather data zGather data y
Gather data xAnalyze
© 2003 by Carnegie Mellon University page 12
Assembly
Produces a prototype that reflects what’s been discovered
Reflects a traditional sequential process• Guided by local “ candidate requirements”
- Iteration Objectives, Detailed Iteration Plan plus hypotheses and desired behaviors expect to derive from prototype
- Vets “candidate requirements”• Produces an executable version of full system
- Likely to have less than complete functionality- Executable, not paper or specification or small piece
Does not preclude prototyping “in the small” throughout
© 2003 by Carnegie Mellon University page 13
Evaluation and Assessment
Occur at two levels:• Evaluation of the prototype in its own context
- Measures actual outcome against expected outcome- Considers degree to which prototype satisfies its
local “requirements”• Assessment of the iteration itself
- Addresses issues at project (not iteration) level- Considers whether objectives were good ones- Determines whether iteration results indicate
changes in project schedule, budget, etc.
© 2003 by Carnegie Mellon University page 14
On-going Iterations of A/APCS
Prototype
Deployed System
Body of Knowledge
Body ofKnowledge
BoK
© 2003 by Carnegie Mellon University page 15
Top-Level View of A/APCS
Manage Program: based on global program constraints, generates iteration objectives and fielding decisions and forwards the current System View to the next iteration
Perform Iteration: based on iteration objectives, the market-place, and current System View, generates a new version of the system, evaluates it, and updates the new System View
Assess Iteration: based on iteration objectives and the evaluation of the system, assess the overall iteration
© 2003 by Carnegie Mellon University page 16
Detail of Perform Iteration
Manage Program
Assess Iteration
PlanIteration
1
ConstructSystem
2
Evaluate System
3
Perform Iteration*
© 2003 by Carnegie Mellon University page 17
Detail of Construct SystemDetailed iteration plan
“Go” decisionfor Assembly
System view (defined in this iteration)
Discovery activities
System (for
evaluation and potential deployment)
Consolidateddata to analyze
Description of needed knowledgeGather
Data2.1
RefineData
2.2
AssembleCandidateSystem 2.3
*System view (defined in previous iteration)
Replanning needed
COTS products
© 2003 by Carnegie Mellon University page 18
Low-level Activities of Refine Data
Description of needed knowledge
System View (defined in previous iterations)
Emerging revisions to System View
Additional knowledge
neededGaps
Resolutions
Resolutions & Gaps
Data to analyze
Detailed iteration plan
Agreements
(to all)
System view
Agreements, Gaps &Conflicts
Conflicts
Gaps
Assembly “Go”
Replanning needed
Analyze Data
Negotiate Conflicts
Form System View
Determine Data to be Gathered
2.2.1
2.2.2
2.2.3
2.2.4
© 2003 by Carnegie Mellon University page 19
Low-level Activity Descriptions
Construct System (Step 2)2.1 Gather Data:Gather Stakeholder InputsGather COTS Product Data Gather Business Process DataGather External System Data
2.2 Refine Data: Analyze DataNegotiate ConflictsDetermine Data to be GatheredForm System View
2.3 Assemble Candidate System:Create Detailed DesignPerform Needed ModificationsWrite Custom CodeIntegrate ComponentsTest
© 2003 by Carnegie Mellon University page 20
Example: Gather COTS Product DataPurpose: Gather product data by evaluating COTS productsRole: Evaluators, stakeholdersInputs/Entry Conditions:Description of the kind of product data that is needed, including the level of detail needed as well as gaps that must be filled. This description forms the requirements for the product evaluation.Outputs/Exit Conditions: Product data to the requisite level of detail, consolidated and formattedDescription:This activity consists of methodically evaluating one or more COTS products. As with all of the “gather” activities, the input is the Need for Knowledge generated by the Refine Data activity. Appropriate evaluation criteria for this product data gathering effort are established; these then provide the basis for the quantification methods that guide the data collection effort.
© 2003 by Carnegie Mellon University page 21
Example (cont.)Notes on Planning: …
Actions:1. Define criteria2. Prioritize criteria3. Prepare for evaluation4. Collect data from the sources5. Record results6. Consolidate and format data (described in section 7.1.5) Notes on the Actions:1. A criterion consists of both a capability statement …2. It is essential to rank criteria by importance. …3. One necessary preparation is to assemble the needed …4. Different data collection techniques may be needed. …5. The raw scores earned by each of the examined products …6. Product evaluation data is often diverse and …
© 2003 by Carnegie Mellon University page 22
Converting This to a Process
While developing this framework, we engaged with a customer who needed a process PDQ.
We took these principles and, using RUP as a backbone, turned them into a process.
Original name: ITSEP
New name: Evolutionary Process for Integrating COTS-based systems (EPIC)• A negotiation-driven process that helps identify and
manage the differences between what users want and what COTS products can deliver
• Commercial and government EPIC pilots underway
© 2003 by Carnegie Mellon University page 23
EPIC Conceptual Framework
TimeTrade Space
Simultaneous Definition
and Tradeoffs
Marketplace
Stakeholder Needs/Business Processes
Architecture/ Design
Programmatics/Risk
• Trades are negotiation-driven with knowledge of marketplace • Requirements formed based on knowledge of market/architecture • Continuous awareness of end-user process changes• Project business and contractual approaches support trades
Trade Space
Decisions
Converge
© 2003 by Carnegie Mellon University page 24
Knowledge Grows Incrementally
• Risk-based spiral development focuses and integrates diverse information
- Prioritized stakeholder needs, end-user processes- Organization and system architecture, design constraints- Identified risks, programmatic constraints- Marketplace offerings, product characteristics, other buyer
usage
• Frequent, evolving executable representations demonstrate current understanding
Executable
Time
© 2003 by Carnegie Mellon University page 25
Continuous Stakeholder Negotiation
• Quick resolution to discovered mismatches- Business process owners and end users- System engineers and developers- Vendors and suppliers- Project managers- Change agents
Time
Stakeholder Buy-in Increases
• Stakeholder needs mature with increased understanding of marketplace implications
• Business processes change to leverage available products• End users committed to system solution
© 2003 by Carnegie Mellon University page 26
EPIC: A Negotiation-Driven Approach
Drive strategic vision to
sustainable solution
Time
Solu
tio
n A
ltern
atives Stakeholder needs
Business processes
ProgrammaticsRisk
Simultaneous Definition
and TradeoffsMarketplace Architecture
Design
Time
Solu
tio
n A
ltern
atives Stakeholder needs
Business processes
ProgrammaticsRisk
Simultaneous Definition
and TradeoffsMarketplace Architecture
Design
Increase stakeholder buy-in and reconcile end-user processes with
COTS-based system
Accumulate knowledge through risk-based spirals
Coordinates operational change, system & software engineering, and project management to field initial capability in 6-18 months
© 2003 by Carnegie Mellon University page 27
Iterations Redefined for CBS
Assess iteration
Gather information
SimultaneousDefinition
and Tradeoffs
(1 to 8 weeks per outer cycle)
Plan iterationAssemble executable representation
Executable
Refine into harmonized set
by identifying and analyzing mismatches and negotiating among stakeholders
© 2003 by Carnegie Mellon University page 28
Phases Bounded by Anchor Points
LifeCycle Architecture
Initial Operational Capability
Elaboration Construction Transition
Refine, experiment & select solution
Try/select COTS
Prototype business process changes
Implement selected solution
Apply/track COTS
Prepare to change business process
Field and support solution
Track/update COTS
Change business process
… … …
LifeCycle Objectives
Inception
Gather and define project scope
Survey/try COTS
Agree to business process changes
…Plan iteration
Gather information
Assess iteration
Refine into harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
Assemble executablePlan iteration
Gather information
Assess iteration
Refine into harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
Assemble executablePlan iteration
Gather information
Assess iteration
Refine into harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
Assemble executablePlan iteration
Gather information
Assess iteration
Refine into harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
Assemble executablePlan iteration
Gather information
Assess iteration
Refine into harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
Assemble executablePlan iteration
Gather information
Assess iteration
Refine into harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
Assemble executablePlan iteration
Gather information
Assess iteration
Refine into harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
Assemble executablePlan iteration
Gather information
Assess iteration
Refine into harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
Assemble executable
6 to 18 months
© 2003 by Carnegie Mellon University page 29
Inception Phase: Goal
Achieve concurrence among stakeholders on project’s life-cycle objectives
• Match segments of marketplace with critical use cases• Identify implications of potential business process
changes• Establish acquisition strategy for appropriate vendor
relationships
Establish feasibility for the project through the business case that shows there is one or more candidate solutions
Survey & Try COTS
© 2003 by Carnegie Mellon University page 30
Inception Phase: Activities
• Gather high-level information (critical use cases)- “Must have” stakeholder needs and required
business processes- Market drivers, technologies, products- Architectural constraints and design alternatives- Programmatic constraints and risks- Incentives /inhibitors to business process changes
• Refine information to form high-level candidate solutions
• Assemble executable representation(s)
Plan iteration
Gather information
Assess iteration
Refine into harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
Assemble executable
© 2003 by Carnegie Mellon University page 31
Elaboration Phase: Goal
Define a high-fidelity solution with predictable cost/schedule
• Select and acquire COTS products • Business process implementation supported by all user
roles
Achieve sufficient stability of the solution across architecture, requirements, and marketplace
Mitigate development and rollout risks
Try & Select COTS
© 2003 by Carnegie Mellon University page 32
Elaboration Phase: Activities• Gather detailed knowledge Refine knowledge
to form a stable baseline- Gaps/mismatches identified and negotiated- COTS integration mechanisms defined and
validated - Business process changes implementation
refined- Balanced across all 4 spheres
• Assemble architectural prototypes- Business process changes prototyped
...
Amplify each candidate solution
Amplify selected solution
Plan iteration
Gather information
Assess iteration
Refine into harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
Assemble executable
Plan iteration
Gather information
Assess iteration
Refine into harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
Assemble executable
© 2003 by Carnegie Mellon University page 33
Construction Phase: Goal
Achieve a product quality release ready for the user community
Prepare functional units for needed business process changes• Organization changes• Training
Manage vendor relationships
Balance development stability and potential obsolescence with marketplace volatility
Apply & Track COTS
© 2003 by Carnegie Mellon University page 34
Construction Phase: Activities
• Gather marketplace information continually
• Refine selected solution as necessary
• Assemble selected solution for fielding- Develop any custom components- Tailor products (non-source code
customizations) - Develop production quality COTS integration
code/data sets- Implement business process changes for initial
fielding
Plan iteration
Gather information
Assess iteration
Refine into harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
Assemble executable
© 2003 by Carnegie Mellon University page 35
Transition Phase: Goal
Roll out and support selected solution set to the user community• Initial fielding (beta testing)• Full fielding• On-going support until solution release retired
Implement business process changes across user community
Balance operational stability with marketplace volatility
Manage relationships with vendors
Track & Update COTS
© 2003 by Carnegie Mellon University page 36
Transition Phase: Activities
• Gather marketplace information continually
• Refine solution to accommodate the impact of new products, releases, or technology
• Assemble maintenance releases- Re-tailor products (non-source code
customizations)- Re-develop COTS integration code/data sets- Re-integrate and re-test
Plan iteration
Gather information
Assess iteration
Refine into harmonized set
ExecutableExecutable
Programmatics/Risk
Business Processes
Architecture/ •Design
Marketplace
Stakeholder needs/
SimultaneousDefinition
and Tradeoffs
Assemble executable
© 2003 by Carnegie Mellon University page 37
Closing
This presentation has discussed:• a framework for COTS-based systems processes• a specific process based in that framework
The principles are broadly applicable, and you are invited to try them out.
© 2003 by Carnegie Mellon University page 38
For More Information
David J. CarneySEIPittsburgh, [email protected]
Tricia OberndorfSEIPittsburgh, [email protected]
Patrick PlaceSEIPittsburgh, [email protected]
Ceci AlbertSEIWashington, [email protected]
Lisa BrownswordSEIWashington, [email protected]
© 2003 by Carnegie Mellon University page 39
Acronyms
A/APCS Assembly/Acquisition Process for COTS-based Systems
CBS COTS-based systemCM configuration managementCOTS commercial off-the-shelfEPIC Evolutionary Process for Integrating COTS-
based systems ITSEP Integrating Technology by a Structured
Evolutionary ProcessSEI Software Engineering Institute
© 2003 by Carnegie Mellon University page 41
Guiding Principles1
• The requirements process must celebrate flexibility: requirements definition must be delayed and negotiated, and the true requirements minimized.
• Use of any COTS product necessitates documenting, and probably re-engineering of existing end-user process.
• Any COTS–based system will be in a necessary state of continual system re-formation and evolution throughout its useful lifetime.
© 2003 by Carnegie Mellon University page 42
Guiding Principles2
• Hands-on product evaluations are mandatory; these must be budgeted, scheduled, and accepted as a fundamental project activity, as central as requirements and design.
• Prototyping of the integrated collection of COTS products from the earliest possible moment throughout system lifetime is a basic necessity.
• Satisfactory contractual commitments can only result from the active participation of end users, testers, and other stakeholders, continuously throughout the entire period from project inception through sustainment and maintenance.