ment apability
ineering
Slide -1
Software Process AssessUsing SEI’s Software CMaturity Model
Neal S. CoulterCollege of Computing Sciences and EngUniversity of North Florida
ractices
om 1987-1990 vings.
ngineering pared with 89% , 25% of total y the customer; rted.
Slide -2
Can Software Development PImprove Dramatically? - 1
•Hughes Aircraft spent $445,000 frand realized $2,000,000 annual sa
•Schlumberger completed 94% of eprojects on schedule in 1992, comin 1991 and 51% in 1990. In 1989product defects were discovered bin 1991, 10% were customer repo
ractices
dollar invested ivity and hired eet the demand
e in SLOC per fect density livered code to costs dropped ost overruns for
8 to 3% in 1991.
Slide -3
Can Software Development PImprove Dramatically? - 2
•Raytheon realized $7.70 for everyand a two-fold increase in product100 additional staff members to mfor new business. •Raytheon realized a 190% increasperson-month from 1988-1996; dedropped from 17.2 per KLOC of de4.0 per KLOC. In two years reworkfrom 41% of project cost to 20%. Cprojects dropped form 40% in 198
ments?
rger and oth- and modify-
ing the I) Software
the improve-
Slide -4
What Caused these Improve
Hughes, Raytheon, Motorola, Schlumbeers reported such changes after assessinging its software development process usSoftware Engineering Institute’s (SECapability Maturity Model (CMM).
Hence, CMM could be the reason for ments!
?
ue of Level 2 are
3”
Slide -5
What’s In It for the Developers
Less stress“The only ones questioning the valthose who have not achieved it”“A coherent culture exists at Level
Better morale
Less turnover
Less overtime
Job security
stitute
f Defense (DoD)
iversity advancing the ering to improve on software
Slide -6
The Software Engineering In
•Operated by the US Department o•Opened in 1984 in Pittsburgh•Affiliated with Carnegie Mellon Un•Mission is to provide leadership instate of practice of software enginethe quality of systems that depend
re Pro-
tracts found missed by 20 billion A12 e to software
urse.
Slide -7
Motivation for an SEI Softwacess Model - 1
A review of 17 major DoD software conthat the average 28-month schedule wasmonths; no project was on time. The $58aircraft program was canceled mainly duproblems.
Many similar stories are common, of co
re Pro-
rge a gap verage current
h military soft-problems but tanding of soft-lopment as a pro-
g complexity and
Slide -8
Motivation for an SEI Softwacess Model - 2
DoD concluded “few fields have so labetween current best practices and apractice...today’s major problems witware development are not technical management problems... the undersware as a product and software devecess is not keeping pace with growinsoftware complexity...”
developing a brief descrip-mary archi-
Slide -9
History of CMM - 1
In November 1986, SEI and Mitre beganprocess maturity framework. In 1987, a tion of the model was released by its pritect, Watts Humphrey.
Methods were developed forSoftware process assessments (SPA)Software capability evaluation (SCE)
s improve-
alified to per-
in evaluating ftware.
Slide -10
History of CMM - 2
SPAs are intended for an internal procesment program
SCEs are used to appraise contractors quform work
A questionnaire was developed to assistorganizations’ abilities to develop so
work into the CMM, now
ce
.cmu.edu)
Slide -11
History of CMM - 3
In 1991, SEI evolved the maturity frameCapability Maturity Model for SoftwareVersion 1.1.
The CMM• Is based on actual practices• Reflects the best state of the practi• Reflects the needs of individuals• Is documented• Is publicly available (http://www.sei
-Based
eds of profes-
total agreement
Slide -12
Benefits and Risks of ModelImprovement - 1
Benefits•Provides a common language•Supports measurement•Based on collaboration with hundrsionals•Represents a consensus, but not
-Based
ls are useful” -
hes on some le and technol-
ains, advocate ow to hire,
Slide -13
Benefits and Risks of ModelImprovement - 1
Risks•All models are wrong; some modeGeorge Box•Not comprehensive; it barely toucnonprocess factors, such as peopogy•Does not address application domspecific technologies, or suggest htrain, and retain people.
ent (TQM) e goals of
M does not delighted) at the soft-satisfy the the requirements of the total sys-re development nship.
Slide -14
CMM and TQM
CMM is built on Total Quality Managemprinciples. The goal of TQM is to meet ththe customer, now and in the future. CMstate the customer should be satisfied (orwith the software product. It does state thware supplier should build products that customer’s needs as documented in allocated to the software component tem or product being supplied. Softwais usually on part of a business relatio
ING
MATURITY
LEVELS
Slide -15
The Basic CMM ModelOPTIMIZ
5
MANAGED4
DEFINED3
REPEATABLE2
INITIAL1
Continuouslyimproving
process
PredictableProcess
Standard,consistent
process
Disciplinedprocess
for a given
ethods, prac-ploy to .
xpected rocess.
ual results s.
Slide -16
Some Basic Definitions - 1
Process - A sequence of steps performedpurpose. What you do.
Software process - A set of activities, mtices and transformations that people emdevelop and maintain software products
Software process capability - Range of eresults by following a certain software p
Software process performance - The actachieved by following a software proces
o which a ntrolled and r growth in s of an orga-cy of its applica-
Slide -17
Some Basic Definitions - 2
Software process maturity - The extent tspecific process is defined, measured, coeffective. Maturity implies a potential focapability and indicates both the richnesnization’s process and the consistention across projects.
nary plateau cess. Each goals that, rt of the soft-e maturity nent in the
e in the pro-
Slide -18
Some Basic Definitions - 3
Maturity level - A well-defined evolutiotoward achieving a mature software promaturity level comprises a set of processwhen satisfied, stabilize an important paware process. Achieving each level of thframework establishes a different composoftware process, resulting in an increascess capability of the organization.
nd Risk RESULT
ty
Risk
Slide -19
CMM, Productivity, Quality, a
5Optimizing
4Managed
3Defined
2Repeatable
1Initial
LEVEL
Productiviand
Quality
cterized as ad processes are al efforts and
nt processes nd functional-n place to similar appli-
Slide -20
CMM Level Summaries - 1
1. Initial - The software process is charahoc, and occasionally even chaotic. Fewdefined and success depends on individuheroics.
2. Repeatable - Basic project managemeare established to track cost, schedule, aity. The necessary process discipline is irepeat earlier successes on projects withcations.
h manage-ented stan-oftware e organization’s
Slide -21
CMM Level Summaries - 2
3. Defined - The software process for botment and engineering activities is documdardized, and integrated into a standard sprocess for the organization. Called thstandard software process.
oftware pro-d used for
rovement is he process echnologies.
Slide -22
CMM Level Summaries - 3
4. Managed - Detailed measures of the scess and product quality are collected ananalysis and control.
5. Optimizing - Continuous process impenabled by quantitative feedback from tand from piloting innovative ideas and t
l 5,
f the 379 organi-anced enough to
s in place and ments, 73% don’t
Slide -23
So, Everyone but Us is LeveRight?
According to SEI’s reportable data, ozations at 99 companies that are advhave process improvement programhave conducted SEI maturity assessrate higher than Level 1.
The others have not applied.
vels - 1EL 4
sses are ta-under-and zed
es of ual ms are stood imi-
LEVEL 5
Processes are continuously and systemat-ically improved
Common sources of problems are understood and elimi-nated
Slide -24
Advancing Through CMM Le
PROCESS
LEVEL 1
Few stable processes exist or are used
“Just do it”
LEVEL 2
Documented and stable estimating, planning, and commitment processes at the project level
Problems are recognized and corrected as they occur
LEVEL 3
Integrated management and engineer-ing processes are used across the organization
Problems are anticipated and pre-vented, or their impacts are mini-mized
LEV
Procequantitively stood stabili
Sourcindividprobleunderand elnated
vels - 2EL 4
sense mwork within roject
LEVEL 5
Strong sense of teamwork exits across the organiza-tion
Everyone is involved in process improvement
Slide -25
Advancing Through CMM Le
PEOPLE
LEVEL 1
Success depends on individual heroics
“Fire fighting” is a way of life
Relationships between disci-plines are uncoordi-nated, perhaps even adversary
LEVEL 2
Success depends on individuals; management system sup-ports
Commitments are understood and managed
People are trained
LEVEL 3
Project groups work together, per-haps as an integrated product team
Training is planned and provided according to roles
LEV
Strongof teaexistseach p
vels - 3EL 4
chnol-re ed on itative
LEVEL 5
New technol-ogies are pro-actively pursued and deployed
Slide -26
Advancing Through CMM Le
TECHNOLOGY
LEVEL 1
Introduction of new tech-nology is risky
LEVEL 2
Technology supports established, stable pro-cesses
LEVEL 3
New technol-ogies are evaluated on a qualitative basis
LEV
New teogies aevaluata quantbasis
vels - 4L 4
fini- col-
are dized he ation
e used rstand ess -
nd sta-
LEVEL 5
Data are used to evaluate and select pro-cess improve-ments
Slide -27
Advancing Through CMM Le
MEASUREMENT
LEVEL 1
Data collec-tion and anal-ysis is ad hoc
LEVEL 2
Planning and management data used by individual projects
LEVEL 3
Data are col-lected and used in all defined pro-cesses
Data are sys-tematically shared across projects
LEVE
Data detion andlection standaracross torganiz
Data arto undethe procquantitatively abilize it
productive tion for the undation fail
under stress--ent.
t is character- they can not
er-level orga-ul in moving
Slide -28
Skipping Levels
Trying to skip maturity levels is counterbecause each level is a necessary foundanext level. Processes without a proper foat the same time they are needed most--and provide no basis for future improvem
Many developers have collected data thaistic of Level 4 organizations only to finduse it. However, some attributes of highnizations, such as an SEPG, can be helpfup levels.
ity Level
en repeating roduce
Target
Time/$/...
evel 3
Slide -29
Process Capability by Matur
A process is under statistical control whthe work in roughly the same way will proughly the same results.
Level 1 Target
Time/$/...
Probability
Level 5Target
Time/$/...
Probability
Probability
L
Levels
.
y practices
e
Slide -30
Internal Structure of Maturity
Each maturity level has constituent parts
Maturity levels
Key process areas
Common features
Ke
Processcapability
Goals
Implementation orinstitutionalization
Activities orinfrastructure
Indicate
Achieve
Address
Describ
ents - 1
elated activi-ieve a set of
process capa- processes for atisfied and
lture and ctices, poli-ngoing way e underlying
Slide -31
Some Maturity Level Compon
A key process area (KPA) is a cluster of rties that, when performed collectively, achgoals considered important for enhancingbility. To achieve a maturity level, the keythat level (and the lower levels) must be sthe processes must be institutionalized.
Institutionalization is the building of a cuinfrastructure to support the methods, pracies, and procedures so that they are the oof doing business. The infrastructure is thframework of an organization or system.
ents - 2
practices and ganization has als signify the A.
actices. Key structure that ntation and practices not mandate nted.
Slide -32
Some Maturity Level Compon
The goals of each KPA summarize its keycan be used in determining whether an oreffectively implemented the KPA. The goscope, boundaries, and intent of each KP
Each KPA is described in terms of key prpractices describe the activities and infracontribute most to the effective implemeinstitutionalization of the KPAs. The keydescribe “what” is to be done, but do “how” the process should be impleme
s
organized by lementation
ective, repeat-res are:
, sponsorship...ining...dures, track-
ple measure-
s, audits...
Slide -33
Components of Key Practice
The practices that describe the KPAs arecommon features that indicate if the impand institutionalization of a KPA are effable, and lasting. The five common featu
•Commitment to perform - Policies•Ability to perform - Resources, tra•Activities performed - Plans, proce
ing...•Measurement and analysis - Sam
ments...•Verifying implementation - Review
nge managementanagement
Slide -34
KPAs by Maturity Level - 1
Initial
Organization process focusOrganization process definitionTraining programIntegrated software managementSoftware product engineeringIntergroup communicationPeer reviews
Quantitative process managementSoftware quality management
Managed
Defined
Repeatable
Requirements managementSoftware project planningSoftware tracking and oversightSoftware subcontract managementSoftware quality assuranceSoftware configuration management
Defect preventioTechnology chanProcess change m
Optimizing
Sl
K
L
L project-man-a
L anizational is infrastructure th ngineering a ojects.
ide -35
PAs by Maturity Level - 2
evel 1 - No KPAs
evel 2 - Focuses on establishing basic gement controls.
evel 3 - Addresses both project and orgsues, as an organization establishes an at institutionalizes effective software e
nd management processes across all pr
titative ess and the
anizations continuous
Slide -36
KPAs by Maturity Level - 3
Level 4 - Focuses on establishing a quanunderstanding of both the software procsoftware work products being built.
Level 5 - Focuses on issues that both organd projects must address to implement and measurable process improvement.
me>
X> is
mmary>cess X as
Slide -37
KPA Template - 1
<Key Process Area X>
a key process for level n: <Level na
The purpose of <Key Process Area <statement>
<Key Process Area X> involves <su<Additional elaboration on Key Pro appropriate>
as goal>
written for <X>
Slide -38
KPA Template - 2
Goals
Goal 1 <Process summary statement...
Commitment to Perform
Commitment 1 The project follows aorganizational policy
...
nsible for
in Key Pro-ng to a docu-
Slide -39
KPA Template - 3
Ability to Perform
Ability 1 A group that is respo<X> exists
...
Activities PerformedActivity 1 <Activity performed
cess Area X> accordimented procedure
...
ade to deter- activities
> are manage-asis
Slide -40
KPA Template - 4
Measurement and Analysis
Measurement 1 Measurements are mmine the status of thefor <X>
...Verifying Implementation
Verification 1 The activities for <Xreviewed with seniorment on a periodic b
ble
Management of products e project’s soft-
gement
Slide -41
...SCM Template - 1
A key process area for Level 2: Repeata
The purpose of Software Configuration is to establish and maintain the integrityof the software project throughout thware life cycle....
GoalsGoal 1 Software configuration mana
activities are planned
written for imple-
ity for man-e exists or is
Slide -42
...SCM Template - 2
Commitment to PerformCommitment 1 The project follows a
organizational policymenting SCM
...
Ability to PerformAbility 1 A board having the author
aging the project’s baselinestablished
...
r each soft-a docu-
ade to deter- SCM activi-
Slide -43
SCM Template - 3
Activities PerformedActivity 1 A SCM plan is prepared fo
ware project according to mented procedure
...
Measure and AnalysisMeasurement 1 Measurements are m
mine the status of theties
...
re reviewed ent on a
Slide -44
SCM Template - 4
Verifying ImplementationVerification 1 The SCM activities a
with senior managemperiodic basis
...
1
hat requires sts of the
re
hs and weak-
of the organiza-
Slide -45
CMM-Based Assessments -
Complete assessment is a detailed task torganization wide participation. It consiphases:•Select a team•Complete the maturity questionnai•Analyze the responses•Conduct a detailed site visit•Produce a list of findings of strengtnesses•Prepare a key process area profiletion
2
3
ResponseAnalysis
6
KPAProfile
Slide -46
CMM-Based Assessments -
1
TeamSelection
2
MaturityQuestionnaire
samplesthe
CMM
5
Findings
based on the CMM
4
On-SiteVisit
Interviews and documentreviews
all key practices
ther
n’t Know
Slide -47
Maturity Questionnaire
•Based on CMM Version 1.1•Organized by CMM KPAs•Covers all 18 KPAs•Addresses each KPA goal but not •6 to 8 questions per KPA•Can be completed in one hour•No standard scoring scheme•Identifies issues to be explored fur
Each question can be answered: Yes, No, Does not Apply, Do
Also place for comments
oject?
and made ough the use
nizational anagement
Slide -48
SCM Maturity Questionnaire
Has eight questions:
1. Are SCM activities planned for the pr
2. Has the project identified, controlled,available the software work products throf configuration management?...5. Does the project follow a written orgapolicy for implementing configuration mactivities?...