cbs process wg. participants david carney (chair) m. al-said (scribe) tony jordano kyung whan lee...
Post on 21-Dec-2015
217 views
TRANSCRIPT
CBS Process WGCBS Process WG
ParticipantsParticipants
David Carney David Carney (chair)(chair)
M. Al-Said (scribe)M. Al-Said (scribe)
Tony JordanoTony Jordano
Kyung Whan LeeKyung Whan Lee
Jeffrey PoulinJeffrey Poulin
David KlappholzDavid Klappholz
Glenn BergGlenn Berg
Jongmoon BaikJongmoon Baik
Rami RazoukRami RazoukGeorge HulingGeorge HulingTim SpinneyTim SpinneySteve CrossSteve CrossMike MooreMike MooreCo-chairs:Co-chairs:
Betsy ClarkBetsy ClarkDan Port Dan Port Tricia OberndorfTricia Oberndorf
Working ApproachWorking Approach
Homework: come in with 3 issuesHomework: come in with 3 issues Brainstormed, collapsed, clusteredBrainstormed, collapsed, clustered Prioritized by importance (practitioner Prioritized by importance (practitioner
& researcher) & difficulty& researcher) & difficulty Formulated process-oriented Formulated process-oriented
statementsstatements Discussed prospective CeBASE Top 10 Discussed prospective CeBASE Top 10
ListList
Issue 1: CBS Lifecycle Issue 1: CBS Lifecycle Models and Processes - 1Models and Processes - 1
Is there a new lifecycle Is there a new lifecycle modelmodel for for CBSs? CBSs?
Can one (or more) of the existing Can one (or more) of the existing lifecycle models be tailored to lifecycle models be tailored to CBSs?CBSs?
How do we define a CBS life-cycle How do we define a CBS life-cycle processprocess??
Issue 1: CBS Lifecycle Issue 1: CBS Lifecycle Models and Processes - 2Models and Processes - 2
Both of these questions in light of:Both of these questions in light of: A requirements-driven (COTS-supported) A requirements-driven (COTS-supported)
vs. COTS-driven paradigm vs. COTS-driven paradigm Strategies for handling different kinds of Strategies for handling different kinds of
market segmentsmarket segments CBS time to market pressuresCBS time to market pressures Level of service quality needsLevel of service quality needs ……..How do you identify How do you identify [-> metrics group][-> metrics group] and and
factor in multiple CBS cost drivers?factor in multiple CBS cost drivers?
Issue 2: Post-Deployment Issue 2: Post-Deployment ProcessProcess
In what ways does the CBS post-In what ways does the CBS post-deployment process differ from:deployment process differ from:
the CBS development process the CBS development process the custom post-deployment the custom post-deployment
process?process?
In what ways do the differences affect In what ways do the differences affect cost estimation for CBSs?cost estimation for CBSs?
Issue 3: Issue 3: Release PlanningRelease Planning
What are the process implications of What are the process implications of planning for system releases where COTS planning for system releases where COTS product:product:
releases are not synchronized with each otherreleases are not synchronized with each other releases are not synchronized with your systemreleases are not synchronized with your system end-of-life occurs during your system lifeend-of-life occurs during your system life
Planning includes cost estimation, Planning includes cost estimation, scheduling, determining system release scheduling, determining system release content, etc.content, etc.
Issue 4: Assessment, Issue 4: Assessment, Evaluation & Testing Evaluation & Testing ProcessProcess
How and when do you assess, evaluate, How and when do you assess, evaluate, and test:and test:
COTS productsCOTS products COTS-based systemsCOTS-based systems““When” includes timing (e.g., multiple points When” includes timing (e.g., multiple points
in the process) and system-independent in the process) and system-independent certification of COTS products. certification of COTS products.
““How” includes process, techniques, tools, How” includes process, techniques, tools, roles, and metrics.roles, and metrics.
Related to determination of CBS requirements.Related to determination of CBS requirements.
Issue 5: COTS Market and Issue 5: COTS Market and IdentificationIdentification
What are the processes forWhat are the processes for Acquiring and maintaining market knowledgeAcquiring and maintaining market knowledge Analyzing the forces in a market segmentAnalyzing the forces in a market segment Forecasting trends in both market segments and Forecasting trends in both market segments and
specific productsspecific products Obtaining & disseminating COTS product information Obtaining & disseminating COTS product information
and product-specific experiencesand product-specific experiences Establishing and maintaining vendor relationshipsEstablishing and maintaining vendor relationships
Both the active (e.g., influencing) and passive aspects Both the active (e.g., influencing) and passive aspects of these should be considered.of these should be considered.
Issue 6: Surprise!Issue 6: Surprise!
How does a CBS process How does a CBS process accommodate/address large discontinuities accommodate/address large discontinuities stemming from COTS product surprises? stemming from COTS product surprises? Surprises result from releases or patches:Surprises result from releases or patches:
that are not backward compatiblethat are not backward compatible that fail to performthat fail to perform whose fundamental properties changewhose fundamental properties change
Exacerbated by inadequate documentation.Exacerbated by inadequate documentation. [-> architecture issue concerning product [-> architecture issue concerning product
characterization]characterization]
Issue 7: Organizational Issue 7: Organizational Assessment CapabilityAssessment Capability
How does an organization assess its How does an organization assess its capability to be successful with CBSs?capability to be successful with CBSs?
How does an organization improve its CBS How does an organization improve its CBS capability? This includes such issues as:capability? This includes such issues as:
relationship between CMMI and CBS processesrelationship between CMMI and CBS processes metrics for CBS process maturitymetrics for CBS process maturity
Applies to capabilities of CBS acquirers, Applies to capabilities of CBS acquirers, developers, and maintainers. developers, and maintainers.
Other IssuesOther Issues Dealing with extra featuresDealing with extra features Managing customer expectations (post-Managing customer expectations (post-
selection)selection) World-class SE organizations becoming COTS World-class SE organizations becoming COTS
integratorsintegrators LicensingLicensing Systematic way of deriving system challengesSystematic way of deriving system challenges In-place transition of baselines for In-place transition of baselines for
uninterruptible systemsuninterruptible systems Metrics for degree of product & system riskMetrics for degree of product & system risk
COTS Process Issue Priorities –1aCOTS Process Issue Priorities –1aImportance to Practitioner
0123456789
0.0
0.5
1.0
1.5
2.0
2.5
3.0R
ele
ase
A,E
,&T
Po
st-D
ep
loy
Su
rpri
ses
CO
TS
Mkt
&Id
CO
TS
Su
pvs
. Drv
n
Ve
nd
or
De
ficie
nci
es
Ca
pa
bili
tyA
sse
ss
Co
st D
rive
rs
Importance to Practitioner Scores
COTS Process Issue Priorities –1bCOTS Process Issue Priorities –1b
COTS Process Issue Priorities –2aCOTS Process Issue Priorities –2aImportance to be Researched Tallies
02468
1012
Cost D
river
s
A,E,&
T
COTS Sup
vs. D
rvn
Post-D
eploy
Releas
e
Capab
ility A
sses
s
Surpr
ises
Vendo
r Def
icien
cies
COTS Mkt
& Id
COTS Process Issue Priorities –2bCOTS Process Issue Priorities –2b
0.0
0.5
1.0
1.5
2.0
2.5
3.0C
ost
Dri
vers
A,E
,&T
CO
TS
Su
pvs
. Drv
n
Po
st-D
ep
loy
Re
lea
se
Ca
pa
bili
tyA
sse
ss
Su
rpri
ses
Ve
nd
or
De
ficie
nci
es
CO
TS
Mkt
&Id
Importance to be Researched Scores
COTS Process Issue Priorities –3aCOTS Process Issue Priorities –3aDifficulty Tallies
012345678
Post-D
eploy
Releas
e
Cost D
river
s
Surpr
ises
COTS Sup
vs. D
rvn
Capab
ility A
sses
s
A,E,&
T
Vendo
r Def
icien
cies
COTS Mkt
& Id
COTS Process Issue Priorities –3bCOTS Process Issue Priorities –3b
0.0
0.5
1.0
1.5
2.0
2.5
3.0P
ost
-De
plo
y
Re
lea
se
Co
st D
rive
rs
Su
rpri
ses
CO
TS
Su
pvs
. Drv
n
Ca
pa
bili
tyA
sse
ss
A,E
,&T
Ve
nd
or
De
ficie
nci
es
CO
TS
Mkt
&Id
Difficulty Scores
Release, Surprise VotesRelease, Surprise Votes
Post-deployment, AE&T Post-deployment, AE&T VotesVotes
Assessment, COTS Id. Assessment, COTS Id. Votes Votes
COTS-supp/–driven VotesCOTS-supp/–driven Votes
Priorities (Diff. = Imp.)Priorities (Diff. = Imp.)
8
Priorities (Diff/Imp = 1)Priorities (Diff/Imp = 1)
8
Prospective CeBASE Top Prospective CeBASE Top 1010
General reactions:General reactions: Use of list as guidance Use of list as guidance Good (only?) as top 10 list of Good (only?) as top 10 list of
things CeBASE will researchthings CeBASE will research Use of unfamiliar and imprecise Use of unfamiliar and imprecise
terminologyterminology Is it valid to have hypotheses on Is it valid to have hypotheses on
this list?this list?
Specifics -1Specifics -1
1. Valid within a limited scope only.1. Valid within a limited scope only.
2. What else is new? So does everything 2. What else is new? So does everything else!else! Add: “…. Just like traditional” Add: “…. Just like traditional” Or “Cost & schedule overruns are as Or “Cost & schedule overruns are as
common with COTS as with custom common with COTS as with custom development and they (CBSs) can cost as development and they (CBSs) can cost as much.”much.”
Specifics -2Specifics -2
3. What is “CBS type”?3. What is “CBS type”? Domain?Domain? COTS-driven vs COTS-supported?COTS-driven vs COTS-supported? COTS product type?COTS product type? ……..?..?
4. Should be studied, but we don’t 4. Should be studied, but we don’t believe the hypothesis as posed:believe the hypothesis as posed: How can a parametric model account for the How can a parametric model account for the
situation in which the selected products can’t be situation in which the selected products can’t be integrated?integrated?
Specifics -3Specifics -3
5. False, this is one of many 5. False, this is one of many parameters that influence the parameters that influence the estimate.estimate. Prefer: ” Architectural mismatch will Prefer: ” Architectural mismatch will
affect CBS cost.”affect CBS cost.”
6. “ …just as with traditional systems; 6. “ …just as with traditional systems; exaggerated with COTS products.”exaggerated with COTS products.” Change to “CBS post-deployment costs Change to “CBS post-deployment costs
dominate …”dominate …”
Specifics -4Specifics -4
7. Intuitively agree; defects will be in 7. Intuitively agree; defects will be in mismatches between products, not mismatches between products, not something visible by inspection of something visible by inspection of line of code.line of code.
A (ADDITION): A (ADDITION): Writing glue code Writing glue code requires greater skill than requires greater skill than traditional coding. Often need to traditional coding. Often need to discover lots of things.discover lots of things.
Specifics -5Specifics -5
8. Agree with “frequently far worse than linear”, 8. Agree with “frequently far worse than linear”, question use of square; heavily dependent on question use of square; heavily dependent on other factors (e.g.; use of standards, quality of other factors (e.g.; use of standards, quality of products, degree of integration)products, degree of integration)
9.9. Question “twice”. Question “twice”. Replace with“Risk specific to CBSs must be Replace with“Risk specific to CBSs must be
managed; CBS risks (and their mitigations) are managed; CBS risks (and their mitigations) are different; risk doesn’t disappear.” different; risk doesn’t disappear.”
Current statement is not a useful hypothesis.Current statement is not a useful hypothesis.
10. Disagree with percentage; agree vaporware 10. Disagree with percentage; agree vaporware is one of is one of manymany factors. factors.
ConclusionsConclusions
Process is foundational for CBSs.Process is foundational for CBSs.
Risk management is still key - and Risk management is still key - and will be different for CBSs.will be different for CBSs.
It was cold, but we had fun anyway!It was cold, but we had fun anyway!