a best practice approach for software maintenance - sustaining critical capability paul r. croll...

A Best Practice Approach for Software Maintenance -

Sustaining Critical Capability

Paul R. CrollChair, IEEE Software and Systems Engineering Standards CommitteeConvener, ISO/IEC JTC1/SC7 WG9

Computer Sciences [email protected]

16th Annual Systems and Software Technology Conference – Track 6, IEEE Sponsored Track – 20 April 2004, 1355-1440Copyright 2004, Paul R. Croll, All rights reserved


Paul R. Croll - 2

Agenda

The Five Types of Software Maintenance A Best Practice Approach Maintenance in the System Life Cycle Maintenance in the Software Life Cycle The CMMI and Maintenance ISO/IEC 14764 – The International Software Maintenance

Standard IEEE 1219 – The National Software Maintenance Standard ISO/IEC 14764 / IEEE 1219 Harmonization Final Thoughts on Software Maintenance


Paul R. Croll - 3

Five Types of Software Maintenance

Corrective Maintenance Identify and remove defects Correct actual errors

Perfective Maintenance Improve performance,

dependability, maintainability Update documentation

Adaptive Maintenance Adapt to a new/upgraded environment (e.g.,

hardware, operating system, middleware) Incorporate new capability

Emergency Maintenance Unscheduled corrective

maintenance

(Risks due to reduced testing)

Preventive Maintenance Identify and detect latent faults Systems with safety concerns


Paul R. Croll - 4

Two Types of Maintenance Activities – Bug Fixing

Corrective Maintenance Identify and remove defects Correct actual errors

Emergency Maintenance Unscheduled corrective

maintenance

(Risks due to reduced testing)

Preventive Maintenance Identify and detect latent faults Systems with safety concerns


Paul R. Croll - 5

Two Types of Maintenance Activities – Development/Migration

Perfective Maintenance Improve performance,

dependability, maintainability Update documentation

Adaptive Maintenance Adapt to a new/upgraded environment (e.g.,

hardware, operating system, middleware) Incorporate new capability


Paul R. Croll - 6

A Best Practice Approach for Software Maintenance

Look to the CMMI for Process Capability

Expectations

Look to Supporting

Standards for Process Detail

Build or Refine Your Process Architecture

Look to Framework Standards for Maintenance

Process Objectives

How do we know what the objectives

and expected outcomes should be for Maintenance?

How do we know what the objectives

and expected outcomes should be for Maintenance?


Process Objectives



Paul R. Croll - 8

Maintenance in the SystemLife Cycle

SYSTEM LIFE CYCLE

PROJECT ASSESSMENT

PROJECT PLANNING

PROJECT CONTROLDECISION MAKING

RISK MANAGEMENTCONFIGURATION MANAGEMENT

INFORMATION MANAGEMENT

ENTERPRISE(5)

SYSTEM LIFE CYCLE MANAGEMENT

RESOURCE MANAGEMENT

QUALITY MANAGEMENT

ENTERPRISE ENVIRONMENT MANAGEMENT

INVESTMENT MANAGEMENT

TECHNICAL (11)

PROJECT (7)

ACQUISITION

SUPPLYAGREEMENT (2)

TRANSITIONSTAKEHOLDER REQUIREMENTS DEFINITION

REQUIREMENTS ANALYSISARCHITECTURAL DESIGN

IMPLEMENTATIONINTEGRATION

VERIFICATION

VALIDATIONOPERATION

MAINTENANCEDISPOSAL

(25)

Maintenance

ISO/IEC 15288

System Life Cycle Processes


Paul R. Croll - 9

ISO/IEC 15288 – Maintenance Process – Purpose

Clause 5.5.11.1 Purpose of the Maintenance Process The purpose of the Maintenance Process is to sustain the

capability of the system to provide a service. This process monitors the system’s capability to deliver

services, records problems for analysis, takes corrective, adaptive, perfective and preventive actions and confirms restored capability.

Source: ISO/IEC 15288:2002, © ISO/IEC2002.


Paul R. Croll - 10

ISO/IEC 15288 – Maintenance Process – Purpose


Sustain the capability of the system to provide a service

Monitor and record problems Take corrective, adaptive, perfective

and preventive actions Confirm restored capability


Paul R. Croll - 11

ISO/IEC 15288 – Maintenance Process – Expected Outcomes

Clause 5.5.11.2 Maintenance Process Outcomes As a result of the successful implementation of the Maintenance

Process: a) A maintenance strategy is developed. b) Maintenance constraints are provided as inputs to requirements. c) Replacement system elements are made available. d) Services meeting stakeholder requirements are sustained. e) The need for corrective design changes is reported. f) Failure and lifetime data is recorded.



Paul R. Croll - 12

ISO/IEC 15288 – Maintenance Process – Expected Outcomes


A maintenance strategy is developed Maintenance constraints are provided Replacement system elements are made

available Services are sustained The need for changes is reported Failure and lifetime data is recorded


Paul R. Croll - 13

Maintenance in the Software Life Cycle

MAINTENANCE

SOFTWARE LIFE CYCLE

TAILORING

CONFIGURATION MANAGEMENTDOCUMENTATION

QUALITY ASSURANCEVERIFICATION

VALIDATIONJOINT REVIEW

AUDITPROBLEM RESOLUTION

PRIMARY (5)

DEVELOPMENT

OPERATION

ACQUISITION

SUPPLY

ORGANIZATIONAL (4)MANAGEMENT

INFRASTRUCTUREIMPROVEMENT

TRAINING

SUPPORTING (8)

Source: Singh97

(17+1)

Maintenance

IEEE/EIA 12207

Software Life Cycle Processes


Paul R. Croll - 14

IEEE/EIA 12207 – Maintenance Process – Purpose

Clause 5.5 Maintenance process This process is activated when the software product undergoes

modifications to code and associated documentation due to a problem or the need for improvement or adaptation

The objective is to modify the existing software product while preserving its integrity

This process includes the migration and retirement of the software product

The process ends with the retirement of the software product The activities provided in this clause are specific to the

Maintenance Process; however, the process may utilize other processes in this International Standard. If the Development Process (5.3) is utilized, the term developer there is interpreted as maintainer

Source: IEEE/EIA 12207.0-1996, © IEEE.1998


Paul R. Croll - 15

IEEE/EIA 12207 – Maintenance Process – Purpose


Activated when the software product undergoes post-delivery modifications

Modifies the existing software product while preserving its integrity

Includes the migration and retirement of the software product


Paul R. Croll - 16

IEEE/EIA 12207 – Maintenance Process – Expected Outcomes

Appendix G, Clause G.9 Maintenance process The use of the Maintenance process should achieve the

following objectives: a) Define the impact of organization, operations, and interfaces on

the existing system in operation; b) Identify and update appropriate life cycle data; c) Develop modified system components with associated

documentation and tests that demonstrate that the system requirements are not compromised;

d) Migrate system and software upgrades to the user's environment; e) Ensure fielding of new systems or versions does not adversely affect

ongoing operations; f) Maintain the capability to resume processing with prior versions.



Paul R. Croll - 17

IEEE/EIA 12207 – Maintenance Process – Expected Outcomes


Defined impact on the existing system in operation

Updated life cycle data Modified system components Demonstration that the system requirements are not

compromised Migration of upgrades to the user's environment No adverse affect ongoing operations The capability to restore prior versions

How do we know what the process capability expectations are for

Maintenance?

How do we know what the process capability expectations are for

Maintenance?


Expectations



Paul R. Croll - 19

Bug Fixing vs. Post-Delivery Development/Migration

VER VAL TS REQM PI

Bug Fixing

CARPP CM PMC RSKM PPQA

All Maintenance

Post-Delivery

Development/

Migration RD REQM TS PI VER VAL

ISM SAM


Paul R. Croll - 20

How does the CMMI Treat Maintenance?

Development The word “development,” when used in the CMMI

Product Suite, implies not only development activities, but also maintenance activities.

Projects that benefit from the best practices of CMMI can focus on maintenance, development, or both.

Source: CMMI -SE/SW/IPPD/SS, V1.1, Continuous Representation, © CMU SEI,2002.

The term Maintenance appears in the CMMI 70 times The term Maintenance appears in the CMMI 70 times

How does the CMMI map to the consensus-based objectives and expected outcomes for Maintenance?

How does the CMMI map to the consensus-based objectives and expected outcomes for Maintenance?



Paul R. Croll - 22

The CMMI and System Maintenance – Objectives

ISO/IEC 15288

Maintenance Objectives

CMMI

Process Areas

Monitor the system’s capability to deliver services

Record problems for analysis

Take corrective, adaptive, perfective and preventive actions

Confirm restored capability

CM

RSKM

RSKMVER VAL CAR

RSKM

VER VAL

PMC

PMC

RD TS PIREQM

PPQA

PPQA

CM CAR


Paul R. Croll - 23

The CMMI and System Maintenance – Outcomes

ISO/IEC 15288

Maintenance Outcomes

CMMI

Process Areas

A maintenance strategy is developed

Maintenance constraints are provided as inputs to requirements

Replacement system elements are made available

Services meeting stakeholder requirements are sustained

The need for corrective design changes is reported

Failure and lifetime data is recorded

PP TS

RD

PI

PMC SAM

RDRSKM

ISM

PPQA CM

RSKMCMPPQA PMC


Paul R. Croll - 24

The CMMI and Software Maintenance – Objectives

IEEE/EIA 12207

Maintenance Objectives

CMMI

Process Areas

Define the impact of organization, operations, and interfaces on the existing system in operation

Identify and update life cycle data

Develop modified system components with associated documentation and tests that demonstrate that the system requirements are not compromised

Migrate system and software upgrades to the user's environment

Ensure fielding of new systems or versions does not adversely affect ongoing operations

Maintain the capability to resume processing with prior versions

TS RSKM

RSKM

CM PPQA

RD REQM

TS PI VER VAL

RSKM

PI

VER VAL RSKM

TS

CM PPQA

CM

Where do we find detailed information to help us build or

refine processes for Maintenance?

Where do we find detailed information to help us build or

refine processes for Maintenance?

Look to Supporting Standards for Process Detail



Paul R. Croll - 26

ISO/IEC 14764 – Maintenance Processes

8 Maintenance Processes Inputs Tasks Controls Support Outputs


Source:ISO/IEC 14764:1999, © ISO/IEC1999.

ProcessImplementation

8.1Problem andModification

Analysis8.2

Maintenance Review/

Acceptance8.4

ModificationImplementation

8.3

Migration8.5

Retirement8.6

Clause number


Paul R. Croll - 27

ISO/IEC 14764 –Process Implementation






Analysis8.2

Maintenance Review/

Acceptance8.4


8.3

Migration8.5

Retirement8.6


Paul R. Croll - 28

ISO/IEC 14764 –Process Implementation






Analysis8.2

Maintenance Review/

Acceptance8.4


8.3

Migration8.5

Retirement8.6

8.1.2.1 Maintenance plans and proceduresThe maintainer shall develop, document, and execute plans and procedures for conducting the activities and tasks of the Maintenance Process.

The Maintenance Plan should document the strategy to be used to maintain the system, while the Maintenance Procedures should provide a more detailed approach on how to actually accomplish the maintenance. In order to develop effective Maintenance Plans and Procedures, the maintainer should perform the following task-steps:a) Assist the acquirer in developing the maintenance concept.b) Assist the acquirer in determining the scope of maintenance.c) Assist the acquirer in analyzing maintenance organization alternatives.d) Ensure written designation as the maintainer for the software product.e) Conduct resource analyses.f) Estimate maintenance costs.g) Perform a maintainability assessment of the system.h) Determine transition requirements.i) Determine transition milestones.j) Identify the maintenance process which will be used.k) Document the maintenance process in the form of operating procedures.


Paul R. Croll - 29

ISO/IEC 14764 – Problem and Modification Analysis






Analysis8.2

Maintenance Review/

Acceptance8.4


8.3

Migration8.5

Retirement8.6


Paul R. Croll - 30

ISO/IEC 14764 – Problem and Modification Analysis






Analysis8.2

Maintenance Review/

Acceptance8.4


8.3

Migration8.5

Retirement8.6

8.2.2.1 MR/PR analysisThe maintainer shall analyze the problem report or modification request for its impact on the organization, the existing system, and the interfacing systems for the following:a) Type; for example, corrective, improvement, preventive, or adaptive to new environment;b) Scope; for example, size of modification, cost involved, time to modify;c) Criticality; for example, impact on performance, safety, or security.

In order to ensure that the requested MR/PR is feasible, the maintainer should perform the following task-steps:d) Determine if the maintainer is adequately staffed to implement the proposed change.e) Determine if the program is adequately budgeted to implement the proposed change.f) Determine if sufficient resources are available and whether this modification will affect ongoing or projectedprojects (may not be necessary for PRs).g) Determine the operational issues to be considered. For example, what are the anticipated changes to systeminterface requirements, the expected useful life of the system, the operational priorities, safety, and security,security impacts, if it is not implemented? (may not be necessary for PRs).h) Determine safety and security implications (may not be necessary for PRs).i) Determine short-term and long-term costs (may not be necessary for PRs).j) Determine the value of the benefit of making the modification.k) Determine the impact on existing schedules.l) Determine the level of test and evaluation required.m) Determine the estimated management cost to implement the change (may not be necessary for PRs).


Paul R. Croll - 31

ISO/IEC 14764 –Modification Implementation






Analysis8.2

Maintenance Review/

Acceptance8.4


8.3

Migration8.5

Retirement8.6


Paul R. Croll - 32

ISO/IEC 14764 –Modification Implementation






Analysis8.2

Maintenance Review/

Acceptance8.4


8.3

Migration8.5

Retirement8.6

8.3.2.1 AnalysisOnce the MR/PR has been approved, the maintainer shall conduct analysis and determine which documentation, software units, and versions thereof need to be modified. These shall be documented. The results of this additional analysis should be documented in the software development folders (SDFs). This effort includes the following task-steps:a) Identify the elements to be modified in the existing system.b) Identify the interface elements affected by the modification.c) Identify the documentation to be updated.d) Update the SDFs.


Paul R. Croll - 33

ISO/IEC 14764 – Maintenance Review/Acceptance






Analysis8.2

Maintenance Review/

Acceptance8.4


8.3

Migration8.5

Retirement8.6


Paul R. Croll - 34

ISO/IEC 14764 – Maintenance Review/Acceptance






Analysis8.2

Maintenance Review/

Acceptance8.4


8.3

Migration8.5

Retirement8.6

8.4.2.1 ReviewsThe maintainer shall conduct review(s) with the organization authorizing the modification to determine the integrity of the modified system. The following task-steps should be performed:a) Trace the MR/PR from requirements, to design, to code.b) Verify testability of the code.c) Verify that coding standards were complied with.d) Verify that only necessary software components were modified.e) Verify that the new software components were integrated properly.f) Check documentation to ensure that it was updated.g) Perform testing.h) Develop test report.


Paul R. Croll - 35

ISO/IEC 14764 – Migration






Analysis8.2

Maintenance Review/

Acceptance8.4


8.3

Migration8.5

Retirement8.6


Paul R. Croll - 36

ISO/IEC 14764 – Migration






Analysis8.2

Maintenance Review/

Acceptance8.4


8.3

Migration8.5

Retirement8.6

8.5.2.2 Migration planIn order to adequately control the migration of a system, a migration plan shall be developed, documented and executed. The planning activities shall include users. Items included in the plan shall include the following:a) Requirements analysis and definition of migration;b) Development of migration tools;c) Conversion of software product and data;d) Migration execution;e) Migration verification;f) Support for the old environment in the future. The development of the Migration Plan should include input from the users. As part of this task, the maintainer should perform the following task-steps:a) Analyze the migration requirements.b) Determine the impact of migrating the software product.c) Establish a schedule for performing the migration.d) Identify data collection requirements for post-operation review.e) Define and document the migration effort.f) Determine and mitigate risks.g) Identify needed migration tools.h) Identify support for the old environment.i) Develop and/or acquire migration tools.j) Incrementally decompose software products and data for conversion.k) Prioritize conversion of software products and data.l) Convert software products and data.m) Migrate software products and data to new environment.n) Run parallel operations.o) Verify migration through testing.p) Provide support for old environment.


Paul R. Croll - 37

ISO/IEC 14764 – Retirement






Analysis8.2

Maintenance Review/

Acceptance8.4


8.3

Migration8.5

Retirement8.6


Paul R. Croll - 38

ISO/IEC 14764 – Retirement






Analysis8.2

Maintenance Review/

Acceptance8.4


8.3

Migration8.5

Retirement8.6

8.6.2.1 Retirement planA retirement plan to remove active support by the operation and maintenance organizations shall be developed and documented. The planning activities shallinclude users. The plan shall address the items listed below. The plan shallbe executed.a) Cessation of full or partial support after a certain period of time;b) Archiving of the software product and its associated documentation;c) Responsibility for any future residual support issues;d) Transition to the new software product, if applicable;e) Accessibility of archive copies of data.

As part of this task, the maintainer should perform the following task-steps:a) Analyze the retirement requirements.b) Determine the impact of retiring the software product.c) Identify replacement software product, if any.d) Establish a schedule for retiring the software product.e) Identify the responsibility for any future residual support.f) Define and document the retirement effort.


Paul R. Croll - 39

IEEE 1219 – Maintenance Processes

ProcessProcess

Associated Processes

OutputInput

Control

4.3 Design4.4 Implementation4.5 Regression/system

testing4.6 Acceptance testing4.7 Delivery

4.1 Problem/modification identification, classification, and prioritization

4.2 Analysis

Source: IEEE 1219-1998, © IEEE, 1998.

Clause number


Paul R. Croll - 46

IEEE 1219 – Annexes

Annex A (informative) Maintenance guidelines Annex B (informative) Supporting Maintenance Technology

Re-engineering Reverse engineering Holistic reusing (i.e., cloning of a “new” system from a parent

system) Software tools

Annex C (normative) Maintenance Plan Guidelines Annex D (informative) Guidelines For Compliance With

IEEE/EIA 12207.1-1997Source: IEEE 1219-1998, © IEEE, 1998.

How are these Standards being

harmonized to be consistent and

complimentary?

How are these Standards being

harmonized to be consistent and

complimentary?



Paul R. Croll - 48

ISO/IEC14764 / IEEE 1219 Harmonization - Differences

ISO/IEC 14764

1. Process Implementation

2. Problem and modification analysis

3. Modification implementation

4. Maintenance review/acceptance

5. Migration

6. Software retirement

IEEE STD 1219

1. Problem identification

2. Analysis

3. Design

4. Implementation

5. System test

6. Acceptance test

7. Delivery

The Standards Have Different Process ModelsThe Standards Have Different Process Models


Paul R. Croll - 49

ISO/IEC14764 / IEEE 1219 Harmonization Approach

Assumptions Use the ISO/IEC 14764 process model as the basic structure Fold IEEE Std 1219 into the 14764 structure

Approach Include relevant IEEE Std 1219 normative material in the document

body Include relevant IEEE Std 1219 informative material in annexes to

the document

Create Two Standards With Identical Normative ContentCreate Two Standards With Identical Normative Content

Some Final Thoughts on

Software Maintenance

Some Final Thoughts on

Software Maintenance

Build or Refine Your Process

Architecture



Paul R. Croll - 51

Software Maintenance is Different Than Software Development

Establish the maintenance process Develop maintenance plans and procedures Establish MR/PR procedures Implement configuration management

Perform problem/modification analysis Analyse MRs/PRs Replicate or verify the problem Develop options for implementing the modification Document the MR/PR, the results, and implementation options Obtain approval for the selected modification option

Develop and test the modification Verify and validate the modification



Paul R. Croll - 52

A Best Practice Approach for Software Maintenance


Expectations

Look to Supporting

Standards for Process Detail

Build or Refine Your Process Architecture


Process Objectives


Paul R. Croll - 53

For More Information . . .

Paul R. CrollComputer Sciences Corporation5166 Potomac DriveKing George, VA 22485-5824

Phone: +1 540.644.6224Fax: +1 540.663.0276e-mail: [email protected]

For IEEE Standards:http://computer.org/standards/sesc/http://computer.org/cspress/CATALOG/st01110.htm

For ISO/IEC Standards:http://saturne.info.uqam.ca/Labo_Recherche/Lrgl/sc7/


Paul R. Croll - 54

References

CMMI -SE/SW/IPPD/SS, V1.1, CMMI for Systems Engineering/Software Engineering/Integrated Product and Process Development, and Supplier Sourcing Version 1.1, CMMI -SE/SW/IPPD/SS, V1.1, Continuous Representation. CMU/SEI-CMU/SEI-2002-TR-011, ESC-TR-2002-011, Carnegie Mellon University, Software Engineering Institute, Pittsburgh, PA, March 2002.

IEEE Standard 1219-1998, IEEE Standard for Software Maintenance, Institute of Electrical and Electronics Engineers, Inc. New York, NY, 1998.

IEEE/EIA Standard 12207.0-1996, Industry Implementation of International Standard ISO/IEC12207:1995 — (ISO/IEC 12207) Standard for Information Technology —Software life cycle processes, Institute of Electrical and Electronics Engineers, Inc. New York, NY, 1998


Paul R. Croll - 55

References - 2

Introduction to CMMI – Continuous V 1.1, Carnegie Mellon University , 2002-2003

ISO/IEC 14764-1999, Software Engineering — Software Maintenance, ISO/IEC JTC1/SC7, 1999.

ISO/IEC 15288:2002, Systems Engineering — System Life Cycle Processes, ISO/IEC JTC1/SC7, 2002.

[Singh97] Raghu Singh, An Introduction to International Standards ISO/IEC 12207, Software Life Cycle Processes, 1997.

a best practice approach for software maintenance - sustaining critical capability paul r. croll...

Documents