Lessons learned about multiple model appraisals (CMMI for Services and CMMI for Development)

How to get costs reduction

Ramiro CarballoCertified SCAMPI Lead AppraiserCaelum, Information & Quality Technology

Madrid, 13-11-2013

2© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.Lessons learned about multiple model appraisals How to get costs reduction

Abstract

Most of the companies that had an appraisal experience using CMMI-DEV as reference model in the past, considered out of the scope an important part of the company, related with service delivery. During a multiple model (or multi-model) appraisal, consultancy services, software maintenance or IT management areas can be included in the same organizational scope as software development areas, reducing the effort and cost of multiple appraisals.

SCAMPI appraisal:

An examination of one or more processes by a

trained team of professionals using an appraisal

reference model as the basis for determining, at a

minimum, strengths and weaknesses, using the

SCAMPI MDD.

3© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.Lessons learned about multiple model appraisals How to get costs reduction

Agenda

– CMMI DEV vs. CMMI SVC– How to define the organizational scope– How to select appropriate objective evidence– Practices interpretation from services point of view– Appraisal participants– Appraisal team members– The evidence database– Quantifying savings

4© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

CMMI DEV vs. CMMI SVC

The constellations CMMI for Development and CMMI for Services have a common set of 17 process areas. The 5 engineering process areas are exclusive from development activities, so they are only included in CMMI-DEV.

In the same way, typical services process areas are focused on continuity (SCON), capacity and availability (CAM), incidents (IRP), transition (SST), strategy (STSM), delivery (SD) and development (SSD).

Constellation:

A collection of CMMI components that are used

to construct models, training materials, and

appraisal related documents for an area of interest

(e.g., acquisition, development, services).

5© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

Identifying services

It is easy to identify which company activities could be improved applying the benefits of these process areas, because most of the companies implement some kind of services model as part of their business model.

6© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

Organizational Scope

It is important to identify which services should be included in the organizational scope of a multiple model appraisal, because they should be representative of the organizational improvement.

The information about selected services is detailed in the public record of the CMMI Institute, and, usually, marketing departments are very interested in increasing the visibility of the principal services the company offers to its customers.

Organizational Scope:

The set of projects or

services that provides

objective evidence of

practices implementation. Organizational Unit:

A piece of the

organization where the

appraisal results can be

generalized.

7© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

Effort related with the number of selected services

One consideration about this services selection is that each kind of service is usually considered a subgroup. So, one or more basic units of each kind of service will be sampled, and the appraisal effort will increase as the number of samples does.

Consultancy services, software maintenance or testing services are usually included inside the organizational scope, together with the software development projects, reducing the effort of multiple SCAMPI appraisals.

Subgroup:

A set of similar projects or

services

8© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

Appropriate Objective evidence

Objective evidence must be provided for each sampled basic unit, considering that they have been collected to provide evidence on the instantiation of a service practice or a development practice.

Each objective evidence must be analyzed according to the meaning of the model practice we are evaluating.

9© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

How to discover the services point of view

It is a failure when, after passing a CMMI DEV appraisal, someone comments: now, we are going to implement CMMI SVC maturity level 2, because, it is only one more process area (SD, service delivery), and we have all others fully implemented, recently.

The right interpretation is that SD is the unique new process area that is only applicable to services basic units.

But the rest of maturity level 2 process areas must be objectively evidenced for services basic units and also for development basic units. Twice !

10© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

Discovering basic units that are service and project at the same time

Some synergies of service management and project management could help the organization to identify a set of projects that are directly related with a set of services, trying to reduce the number of artifacts in the appraisal.

But the use of these artifacts must be evaluated twice: from the services perspective and from the development one.

Basic Unit:

A Project or a Service

11© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

Multi-purpose appraisal participants

Using the same point of view, the selection of the appraisal participants should consider the role of each person in each basic unit, from the service and the development perspective.

For example, a project management activity can solve a capacity requirement, a project monitoring activity can solve an incident management issue, and a risk management activity can be considered part of a service continuity plan.

12© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

Practices Interpretation issues

Depending on the experience of the process team, the appraisal team could discuss some practices interpretation issues during the appraisal: how to estimate software versus how to estimate services, differences between software configuration management and service configuration management, and the relationship with a CMDB (Configuration Management DataBase), and others...

13© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

Experience of appraisal team members

Because of that, it is required that the appraisal team members have enough training and experience in development and in services, so they can evaluate the correct implementation of both kind of practices in the appropriate environment.

Appraisal Team Members Experience:

Constraints: Individual team members, each of

whom have at least two years of

experience performing the type of

work addressed in each appraisal

reference model included. This

means each team member has field

experience with each included model.

• An average of at least

six years of field

experience excluding

the experience of the

appraisal team leader,

relating to the content of

each of the reference

models. This ensures

that team members with

relatively limited

experience are

balanced by others who

have a greater level of

experience.

.

• An aggregate of 25

years of field

experience relating to

the content of each of

the reference models

excluding the

experience of the

appraisal team leader.

This ensures that the

team as a whole has a

substantial experience

base to support their

judgments

The team (as a group)

must have a total of at

least ten years of

management

experience, and at least

one team member must

have at least six years

of experience as a

manager – excluding

the appraisal team

leader.

14© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

Sharing the PIID with both models

The CMMI Institute provides a structured spreadsheet that is the usual way to implement the evidence database, named PIID. (Practice Implementation Indicator Description)

Each page contains the set of evidence of an individual process area. The appraisal team can use the same page for both process areas with the same name in both models.

Configuration Management

Status Key Practice / Notes Source of OE Document (s) Comments

ORGEstadisticas

Artifact

Affirmation Information

Needed

Rating (FI, LI, PI,

NI, NY)

SG1 Baselines of identified work products are established.

SP1.1Identify configuration items, components, and related work products to be placed under configuration management. FIAppraisal Considerations: - Be sure to consider configuration items representative of all disciplines and processes within the assessment scope and context. In a sense, this SP specifies the constraints under which the remaining SPs should be considered and assessed- See model for definition and description of configuration item and its work product components- See model for typical examples of work products that may be part of a configuration item (e.g. process Artifact Examples: - Identified configuration items- Configuration management lifecycle for controlled items (e.g., owner, point at which placed under control, degree of control, change approval.)- Configuration management plan

1Elementos bajo gestión de la configuración del proyecto: Plan de Proyecto, apartado datos y configuración. Items con sus respectivas lineas bases Plan de Proyecto x x

2 SCRUM SCRUM x X

PIID: http://cmmiinstitute.com/assets/SVC_PIIDs_20110608.xlsx

15© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

Work vs. Project

The same criterion can be used for WMC (Work Monitoring and Control) versus PMC (Project Monitoring and Control) and WP (Work Planning) versus PP (Project Planning), but special consideration must be taken for this last case, because WP has one more specific practice that PP.

WP SG1 : Establish Estimates

SP 1.1 Establish the Service Strategy

SP 1.2 Estimate the Scope of the Work

SP 1.3 Establish Estimates of Work

Product and Task Attributes

SP 1.4 Define Lifecycle Phases

SP 1.5 Estimate Effort and Cost

Service Strategy :The strategy

includes consideration of the following

factors at an appropriate level of

abstraction: The objectives and

constraints for the service. Possible

approaches to meeting those

objectives and constraints. The

resources (e.g., skills, environment,

tools, new technologies) that will be

needed. Risks associated with these

factors and how they are addressed

16© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

Summarizing Process Areas (ML3 example)

So, the evidence database will manage 25 process areas, where 13 are specific for services or development, and 12 are common for both models, but they have to be evaluated from both points of view.

SAM (optional)

CMMI for Development

CMM for Services

SSD (addition)

17© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

Quantifying costs reduction

Some numbers about experiences: Saved effort if your organization goes to a multiple model SCAMPI instead of two separated single model appraisals.

SAVED EFFORT AT MULTIPLE MODEL CMMI SCAMPI CLASS A

Saving Multiple Model Single Model 1 Single Model 242,86% DEV SVC ML2 DEV ML2 SVC ML234,62% DEV SVC ML3 DEV ML3 SVC ML335,00% DEV ML3 SVC ML2 DEV ML3 SVC ML2

INCREASED EFFORT FROM SINGLE TO MULTIPLE MODEL SCAMPI

Incr. Multiple Model Single Model 114,29% DEV SVC ML2 DEV ML230,77% DEV SVC ML3 DEV ML37,69% DEV ML3 SVC ML2 DEV ML3

Additional effort if your organization decides to include CMMI SVC in the next CMMI DEV SCAMPI

18© 2

013

CA

ELU

M –

All

Rig

hts

rese

rved

.

Ramiro Carballo Gutiérrez Certified SCAMPI v.1.3 Lead Appraiser( ID# 1201018-03 )CMMI for DevelopmentCMMI for Servicesramiro_carballo@caelum.esMóv.: 639078817 www.linkedin.com/in/ramirocarballo

Caelum Information & Quality Technologies, S. L.Párroco Eusebio Cuenca 50.28045 Madridwww.caelum.eswww.CalidaddelSoftware.comTlf.: 918312029