22/03/2002SPICE: Current and Future Directions - 1 -

The Future for SPICE – the Evolution of ISO 15504

SPICE 2002 Conference, Venice

Terry Rout, Software Quality Institute

Alec Dorling, InterSPICE

Inter SPICE

AgendaBackground

SPICE HistoryISO 15504 – evolving

ArchitectureProcess ModelsMeasurement ScaleCurrent status

SPICE – the communityProcess Reference ModelsSPICE ResearchSPARC Benchmarking

22/03/2002SPICE: Current and Future Directions - 2 -

Inter SPICE

What is ISO/IEC TR 15504?International standard for assessing software processesPurpose:

Continuous process improvementCapability determination

Scope:Processes include acquisition, supply, development, operation, maintenance and support

ISO/IEC TR 15504 is an emerging standardcurrently in transition to full International Standard status

Inter SPICE

The need for a standard

The increasing number of assessment approachesavailable.The increasing use of the technique in commercially-sensitive areas.Improve-IT study funded by UK Defence Research Agency.Study group report for JTC1/SC7:

the international community should pool its resources to develop a standard for software process assessment, incorporating the best features of existing software assessment methods.

22/03/2002SPICE: Current and Future Directions - 3 -

Inter SPICE

RequirementsProcess assessment as the basis for both process improvement and capability determination.Flexibility: applications, sectors, size, projects and organisations.Coverage: process, people and technology.Output: as profiles.Supportive of existing standards: ISO 9001, ISO 12207.

Reliable and consistentassessment.Simple to use and understand.Objective, and quantitative wherever possible.Not presumptive of specific:

organisational structures or management philosophies;lifecycle models, technologies or development methods.

Inter SPICE

Software Process Assessment - History1987 Humphrey / Sweet report from SEI.1989 SEI issues its first questionnaire.

Watts Humphrey’s book published.1991 CMM V1.0 released.

ISO requests a study on process assessment.1993 ISO accepts new work item for standard.

CMM V1.1 released.1995 ISO 15504 (SPICE) first draft released.

22/03/2002SPICE: Current and Future Directions - 4 -

Inter SPICE

Software Process Assessment - History

1998 ISO 15504 published as TR2.Revision authorised

2001 CMMI V1.0 released2002 CMMI V1.1 released

ISO 12207 Amd, ISO 15288, ISO TR 15504-2 Amd, ISO 15504-2 scheduled for publication.

Inter SPICE

Framework for Process Assessment

AssessmentInputs

Assessment Activities• Planning• Data Collection• Data Validation• Process Rating• Reporting

AssessmentOutputs

Assessment Model• Requirements for compliance of processreference model• Requirements for compatibility of assessmentmodel

Roles and Responsibilities• Sponsor• Responsible Assessor

ASSESSMENTPROCESS

MeasurementFramework• Capability Levels• Process Attributes• Rating Scale

22/03/2002SPICE: Current and Future Directions - 5 -

Inter SPICE

The Assessment Framework

Two-dimensional model for processes and process capability

Process Dimension

Process Categories

Processes (P1, …, Pn)

Capability Dimension

Capability Levels (CL1, …, CL5)

Process Capability Attributes

CL5CL4CL3CL2CL1CL0CL0

CUS.1 CUS.2...ORG.6

CL1CL2CL3CL4CL5

Each process receives a capability level rating

This is referred to as a Continuous Model

Inter SPICE

TR 15504: Processes

EngineeringENG.1 Development

ENG.1.1 System Requirements Analysis & DesignENG.1.2 Software Requirements Analysis ENG.1.3 Software DesignENG.1.4 Software ConstructionENG.1.5 Software IntegrationENG.1.6 Software TestingENG.1.7 System Integration & Testing

ENG.2 System & Software Maintenance

Customer-SupplierCUS.1 Acquisition

CUS.1.1 Acquisition Preparation CUS.1.2 Supplier Selection CUS.1.3 Supplier Monitoring CUS.1.4 Customer Acceptance

CUS.2 SupplyCUS.3 Requirements ElicitationCUS.4 Operation

CUS.4.1 Operational Use CUS.4.2 Customer Support

ManagementMAN.1 ManagementMAN.2 Project ManangementMAN.3 Quality ManagementMAN.4 Risk Management

SupportSUP.1 DocumentationSUP.2 Configuration ManagementSUP.3 Quality AssuranceSUP.4 VerificationSUP.5 ValidationSUP.6 Joint ReviewsSUP.7 AuditSUP.8 Problem Resolution

OrganisationORG.1 Organisational AlignmentORG.2 Improvement

ORG.2.1 Process EstablishmentORG.2.2 Process AssessmentORG.2.3 Process Improvement

ORG.3 Human Resource ManagementORG.4 InfrastructureORG.5 MeasurementORG.6 Reuse

22/03/2002SPICE: Current and Future Directions - 6 -

Inter SPICE

TR 15504-2 AmendmentCustomer-Supplier

CUS.1 AcquisitionCUS.1.1 Acquisition Preparation CUS.1.2 Supplier Selection CUS.1.3 Supplier Monitoring CUS.1.4 Customer Acceptance

AcquirerACQ.1 Acquisition PolicyACQ.2 Acquisition StrategyACQ.3 Benefits AnalysisACQ.4 Technical requirementsACQ.5 Legal & Administrative RequirementsACQ.6 Financial RequirementsACQ.7 Project RequirementsACQ.8 Request for ProposalsACQ.9 Supplier QualificationACQ.10 Proposal EvaluationACQ.11 Contract EstablsihmentACQ.12 Supplier MonitoringACQ.13 AcceptanceACQ.14 Contract ClosureACQ.15 Supplier RealtionshipsACQ.16 User Realtionships ACQ.17 Financial Management

Substitute Processes

ISO/IEC TR 15504-2 Amendment – Publication Q1 2002Original Source: Pulse/Probe EC projects

Other processes read in context of acquisition

Software lifecycle >> acquisition lifecycle

Software work product >> acquisition work product

Inter SPICE

ISO/IEC TR 15504 Software Process Assessment

Part 9Vocabulary

Part 3Performing an assessment

Part 4Guide to performing assessments

Part 5An assessment model and

indicator guidance

Part 2A reference model for

processes and process capability

Part 7Guide for use in

Part 8Guide for use in determiningsupplier process capability

Part 6Guide to competency

of assessors

Part 1Concepts and introductory guide

process improvement

22/03/2002SPICE: Current and Future Directions - 7 -

Inter SPICE

ISO/IEC 15504 Process Assessment

Concepts and Vocabulary

Part 1

Part 3Part 4

Part 5

Guidance onPerforming Assessments

Guidance on using Assessment Results

Exemplar Assessment Model

Part 2Performing anAssessment

Part 1 & 9 >> Part 1

Part 2 & 3 >> Part 2

Part 7 & 8 >> Part 4

Part 4 & 6 >> Part 3

Part 5 >> Part 5

Compliant Process Reference Model

Inter SPICE

The Changes

Name change from Software Process Assessment to Process AssessmentRestructuring from 9 parts to 5 partsProcess dimension removed to ISO 12207 AMD1Introduction of Process Reference Model conceptAlignment of Capability Dimension with ISO9001: 2000 (under consideration)

22/03/2002SPICE: Current and Future Directions - 8 -

Inter SPICE

The Assessment Model

Assessment Model

ProcessReference

Model

Requirements for Compliance

Requirementsfor Conformity (Compatibility)

determine suitability of

determine applicability of

ISO 15504-2 (rev)

Measurement Framework

Inter SPICE

Measurement Framework Changes

Level 1 PerformedPA.1.1 Process Performance

Level 2 ManagedPA.2.1 Performance ManagementPA.2.2 Work Product Management

Level 3 EstablishedPA.3.1 Process DefinitionPA.3.2 Process Deployment

Level 4 PredictablePA.4.1 Process MeasurementPA.4.2 Process Control

Level 5 OptimizingPA.5.1 Process InnovationPA.5.2 Process Optimisation

Level 0 Incomplete IncompleteThe process is not implemented or fails to achieve its purpose

PerformedThe process is implemented and achieves its process purpose

ManagedThe process is managed and work products are established, controlled and maintained.

PredictableThe process is enacted consistently within defined limits

OptimisingThe process is continuously improved to meet relevant current and projected business goals

EstablishedA defined process is used based on a standard process.

L5 - Significant changes in the names and descriptions of the attributes, although the capability issues addressed not affected

L1-L3 Redrafting to remove ambiguity and increase consistency with ISO 9001:2000

L4 -Definitions clarified to make the concept of “quantitative understanding” clearer

L3 - Restructuring - Definition and Deployment as the two aspects of capability - “Process Resource” issues are re-distributed

22/03/2002SPICE: Current and Future Directions - 9 -

Inter SPICE

High Capability Processes

The reworking of the high levels of capability clarifies the relationship between “Process Capability” as defined in ISO 15504, and conventional concepts of Statistical Process Control

Level 4 (the Predictable Process) is concerned with the ability to identify and address special causes of process variationLevel 5 (the Optimising Process) is concerned with the ability to identify and address common causes of process variation

Inter SPICE

Current StatusPart 2 has completed “Final Committee Draft” ballot

Comments will be addressed at the meeting in Pisa, 18 MarchA second FCD ballot will be required

Part 3 has completed “Committee Draft” ballotComments will be addressed at the meeting in Pisa, 18 MarchA further FCD ballot will be required

Part 4 is currently in “Committee Draft” ballotComments will be addressed in Busan, 13 May

Part 5 has been developed as a Working DraftFor rework next week, and subsequent Registration Ballot

Part 1 is pending more detailed work on the other parts

22/03/2002SPICE: Current and Future Directions - 10 -

Inter SPICE

Timeline

Part 2 (normative) expected publication mid 2003Part 3 to follow two months laterRemaining parts to be published over the following 12 monthsProject completion expected December 2004

Inter SPICE

22/03/2002SPICE: Current and Future Directions - 11 -

Inter SPICE

Process Reference Models

Model Architecture

ProcessCategories

Processes

CapabilityLevels

ProcessAttributes

requirements

15504Assessment

Model

Process

Reference

Model

15504Requirements

REQUIREMENTSPerforming an assessmentProcess Reference ModelsProcess Assessment ModelsConformity assessment

12207 based

OOSPICE AutomotiveSPICE

ISO9001IT Service Management

Inter SPICE

Process Reference Models

IT Infratructure Management

15288 System Lifecycle Process

12207 Software Lifecycle Process

ISO/IEC 15288– publication Q3/Q4 2002

ISO/IEC 12207 Amendment– publication Q1 2002

SPICE User Group– starting Q4 2002

18529 Human Centered Lifecycle Process

ISO/IEC 18529– published 1999

9001 Quality Management

Component Based Development

European Space Agency– under development – May 2002

OOSPICE project– under development – Q2 2003

Automotive Embedded Software Procurement Forum, Automotive SIG– under development – Q3 2002

22/03/2002SPICE: Current and Future Directions - 12 -

Inter SPICE

CBD PRMEngineering

ENG.1 Modeling ENG.1.1 Domain EngineeringENG.1.2 Business ModelingENG.1.3 Requirement Engineering ENG.1.4 Behaviour Specification ArchitectureENG.1.5 Provisioning StrategyENG.1.6 Use Interface Specification

ENG.2 Application AssemblyENG.2.1 Application DesignENG.2.2 Component SelectionENG.2.3 Component AssemblingENG.2.4 Application TestingENG.2.5 Application Delivery

ENG.2.6 Legacy Mining

ENG.3 Component ProvisioningENG.3.1 Comonent Internal DesignENG.3.2 Component ImplementationENG.3.3 Component TestingENG.3.4 Component Delivery

OrganisationORG.2 Improvement

ORG.2.1 Process EstablishmentORG.2.2 Process AssessmentORG.2.3 Process Improvement

ORG.3 Human Resource ORG.4 InfrastructureORG.5 MeasurementORG.6 ReuseORG.7 UsabilityORG.8 Asset ManagementORG.9 Component Management

Customer-SupplierCUS.1 Acquisition

CUS.1.1 Acquisition Preparation CUS.1.2 Supplier Selection CUS.1.3 Supplier Monitoring CUS.1.4 Customer Acceptance

CUS.2 SupplyCUS.4 Operation

CUS.4.1 Operational Use CUS.4.2 Customer Support

SupportSUP.1 DocumentationSUP.2 Configuration ManagementSUP.4 VerificationSUP.6 Joint ReviewsSUP.8 Problem Resolution

ManagementMAN.1 ManagementMAN.2 Project ManangementMAN.3 Quality ManagementMAN.4 Risk ManagementMAN.5 Programme Management

Source: OOSPICE project www.oospice.com- under development – completion end 2002

Inter SPICE

IT Infrastructure PRMIT Infrastructure ManagementComputer OperationsNetwork ManagementService Level ManagementAvailability ManagementContingency PlanningCapacity ManagementCost ManagementSoftware Control and DistributionComputer Installation and AcceptanceHelp DeskProblem ManagementChange ManagementConfiguration ManagementVendor ManagementCustomer and User LiaisonFacilities ManagementAsset Management

Reference: CCTA IT Infrastructure Library

22/03/2002SPICE: Current and Future Directions - 13 -

Inter SPICE

Automotive PRM

Error, the brake system doesn't answer.

Please stop as soon as possible!

Abort Ignore Retry

Error ?

Inter SPICE

Automotive SPICE

Manufacturers:DaimlerChrysler, VW, Audi, BMW, Porsche, OpelVolvo, Saab, Fiat, PSA, Renault, Landrover, Jaguar

Represenatative OrganisationsVDA, MISRA, INCOSE

GoalCommon approach for assessing and evaluating suppliers based on 15504 Automotive SPICE PRM

22/03/2002SPICE: Current and Future Directions - 14 -

Inter SPICE

ISO9001: 2000

ISO9001 - process basedISO9004 Guidelines for Performance Improvement

contains guidelines for self-assessment with a set of associated performance maturity levels

The burning questions:What process capability levels does an ISO9001 certified

company have ?

What is the relationship (if any) between ISO15504 and ISO9001 ?

Is it possible to establish an ISO15504 assessment framework based on ISO9001 ?

Inter SPICE

ISO9001 Assessed Companies - Mean Capability Levels

0 0.5 1 1.5 2 2.5 3

CUS.2

ENG.1

ENG.2

ENG.3

ENG.4

ENG.5

ENG.6

ENG.7

SUP.1

SUP.2

SUP.4

SUP.8

MAN.1

MAN.2

MAN.3

Capability level

Source: Bootstrap InstituteFindings from SPICE trials – Jung, Hunter, Goldenson, El-Emam

22/03/2002SPICE: Current and Future Directions - 15 -

Inter SPICE

ISO9001 PRMCore Processes

COP.1 Design & Development ManagementCOP.2 Design & Development AssuranceCOP.3 Technical ReviewCOP.4 Production AssuranceCOP.5 Product Acceptance& DeliveryCOP.6 Product PreservationCOP.7 Design & Development Change ManagementCOP.8 Product IdentificationCOP.9 Risk Assessment and ControlCOP.10 Space System Operation Assurance

Purchasing AssurancePAS.1 Supplier SelectionPAS.2 Purchasing RequirementsPAS.3 Supplier MonitoringPAS.4 Incoming Inspection

Customer FocusCUF.1 Requirements DeterminationCUF.2 Requirements ReviewCUF.3 Customer CommunicationCUF.4 Customer SatisfactionCUF.5 Customer Property

Resource and Facilities ManagementRFM.1 Infrastructure ManagementRFM.2 Work Environment ManagementRFM.3 Human Resource ManagementRFM.4 Control and Monitoring of Measuring Devices

OrganisationORG.1 Management DirectionORG.2 Quality EstablishmentORG.3 Management ReviewORG.4 Document ControlORG.5 Record Control

Measurement, Analysis & ImprovementMAI.1 Internal AuditMAI.2 Process Monitoring and MeasurementMAI.3 Product Monitoring and MeasurementMAI.4 Nonconforming Product ControlMAI.5 Data AnalysisMAI.6 Continual ImprovementMAI.7 Corrective ActionMAI.8 Preventive Action

34 Processes in 6 CategoriesSource: European SPACE Agency project – under development - completion May 2002

Inter SPICE

SPICE Network

SPICE User Group

SPICE NetworkMembership Services

Annual Conference

Conformance verificationResearch

Benchmarking

Network Partner

22/03/2002SPICE: Current and Future Directions - 16 -

Inter SPICE

Research

Rational Unified ProcessExtreme programmingOpen sourceApplicable metricsSafety Integrity Levels ISO9001:2000

Inter SPICE

0

1

2

3

4

5

CUS.1 CUS.1.3 CUS.3 CUS.4.2 ENG.1.2 ENG.1.5 ENG.2 SUP.3 SUP.6 MAN.1 MAN.4 ORG.2.1 ORG.3 ORG.6

Predicting Software Quality with TR 15504

Rational Unified Process (5.5) Capability Profile

Source: Predicting Software Quality with TR 15504, Marie Jakobsson, Master thesis, Hogskolan I Boras

22/03/2002SPICE: Current and Future Directions - 17 -

Inter SPICE

BenchmarkingBenchmarking

Benc

hmar

king

Se

rver

Assessment Report

Individual Benchmark

Report

Assessment Tools Assessment rating

& context

Data collection

suite Assessment questions (7)

Extended Individual

Assessment ReportData collection

suiteAssessment rating

& context

Additional Extended Individual

Benchmark Reprorts

Periodic Global

Benchmark Reports

Web upload

Data analysis

Data collection

Data collection12345

12345

Inter SPICE

Where to get your SPICE

SPICE web site ( www.sqi.gu.edu.au/spice/ ) SPICE User Group ( www.seg.iit.nrc.ca/spice )SUGaR web site ( www.seg.iit.nrc.ca/spice )

SUGaR Listserv ( www.isospice.com )SPICE Conference ( www.isospice.com )

The SUGaR and SPICE User Group web sites will in the near future be transitioned to

www.isospice.comwww.spiceusergroup.org

Newsletter from Q3 2002Intranet from Q3 2002

22/03/2002SPICE: Current and Future Directions - 18 -

Thank you for listening