software configuration management

1 Blanko ‘98 22-23 Oct. 1998 © Tichy 1998

Software Configuration Management

Walter F. TichyUniversity of Karlsruhe

Blanko ‘98 22-23 Oct. 1998 2 © Tichy 1998

Definition

Software Configuration Management (SCM) is the discipline of controlling the evolution of software systems.

SCM is indispensable for the development and maintenance of large, long-lived software.

Blanko ‘98 22-23 Oct. 1998 3 © Tichy 1998

Why is it needed?

Software changes constantly. Change causes chaos. SCM helps prevent change chaos:

Tracks development history, Prevents or reconciles conflicting changes, Tracks configurations and baselines, Tracks error reports and fixes, Automates configuration selection, keeps software in well-defined state.

All this in a distributed environment

Blanko ‘98 22-23 Oct. 1998 4 © Tichy 1998

Symptoms of the Change Chaos

Problems of identification and tracking: “This program worked yesterday. What

happened?” “I fixed this error last week. Why is it back?” “Where are all my changes from last week?” “This seems like an obvious fix. Has it been

tried before?” “Who is responsible for this change?”

Blanko ‘98 22-23 Oct. 1998 5 © Tichy 1998

Change Chaos Continued

Problems of version selection “Has everything been compiled? Tested?” “How do I configure for test, with my updates

and no others?” “How do I exclude this incomplete/faulty

change?” “I can’t reproduce the error in this copy!” “Exactly which fixes went into this

configuration?” “Oh shoot. I need to merge 250 files!”

Blanko ‘98 22-23 Oct. 1998 6 © Tichy 1998

More Chaos ….

Software Delivery Problems “Which configuration does this customer have?” “Did we deliver a consistent configuration?” “Did the customer modify the code?” “The customer skipped the previous two

releases. What happens if we send him the new one?”

These problems sound familiar to anyone who has worked in software development.

Blanko ‘98 22-23 Oct. 1998 7 © Tichy 1998

History of SCM

Configuration management was first developed for the aerospace industry in the 1950’s. It was used among others in the Apollo Space Program for tracking thousands of changes.

SCM began in the 1970’s with SCCS and MAKE. SCM is now a successful part of the software

tools market, with at least 50 tools available. SCM tools deliver recognized benefits to users. SCM is a technically challenging area.

Blanko ‘98 22-23 Oct. 1998 8 © Tichy 1998

Literature about SCM

International Workshop on Software Configuration Management 6th to 8th meeting in 1996, 1997, 1998. Conference proceedings by Springer Verlag.

Various books. “Applying RCS and SCCS”, Bolinger & Bronson,

O’Reilly, 1995. “Configuration Management”, Tichy, Wiley, 1994.

Survey of many SCM systems: www.cs.colorado.edu/ ~andre/configuration_management.html

Blanko ‘98 22-23 Oct. 1998 9 © Tichy 1998

Contents

1. Introduction 2. Requirements Space of SCM Systems3. Versions4. Distribution5. Configurations and Building6. Change Tracking7. Process8. Summary

Blanko ‘98 22-23 Oct. 1998 10 © Tichy 1998

2. SCM Requirements Space

Version Management

Building

Change Request Tracking

Process SupportDistribution (LAN, W

AN)

Blanko ‘98 22-23 Oct. 1998 11 © Tichy 1998

Requirements Space

Version Management Traceable version history branch & merge stable workspaces for developers and

integrators control over incorporation of concurrent

changes

Blanko ‘98 22-23 Oct. 1998 12 © Tichy 1998

Requirements Space Cont.

Configurations build anywhere, anytime, with correct components several builds running in parallel each build can run itself in parallel

Change Request Tracking link between change requests, solutions, releases state changes (tested, accepted, released, etc.) accurate status reports

Blanko ‘98 22-23 Oct. 1998 13 © Tichy 1998


Process Support process support for

submit/pickup/build&test/accept parallel work is the rule change set tracking dependency tracking

Blanko ‘98 22-23 Oct. 1998 14 © Tichy 1998


Distribution distributed teams need to be

coordinated. in case of time zone differences: difficult

to do SCM informally. databases must be connected and up-

to-date.

Blanko ‘98 22-23 Oct. 1998 15 © Tichy 1998

3. Version Management

What is covered by Version Management?Software object/item: Any identifiable,

machine-readable document.Examples:requirements documents, design documents, specifications, code, documentation, test programs, test data, test results, user manuals, CAD drawings, etc.

Integration with CAD/CAM and engineering data management is becoming possible today.

Blanko ‘98 22-23 Oct. 1998 16 © Tichy 1998

Software Objects

A software object has a unique name and a persistent body (contents).

A software object is immutable:every change creates a new object with new

name; it cannot be changed once creation process is

finished (freezing prevents misidentification).

Classification according to creation: source or derivedstructure: atomic or configuration

Blanko ‘98 22-23 Oct. 1998 17 © Tichy 1998

Software Objects

Source Object: A manually created software object. Normally “sacred”, i.e., cannot be deleted.

Derived Object: a software object generated fully automatically. Can be deleted to make room.

Has a history attribute recording exactly how it was generated (input+generator).

Blanko ‘98 22-23 Oct. 1998 18 © Tichy 1998

Structure of Software Objects

The body of a software object is eitheratomic (there are no parts that can

be varied independently, without changing the object as a whole).

a configuration (there are several independently variable or replaceable parts, which are either atomic or themselves configurations).

Blanko ‘98 22-23 Oct. 1998 19 © Tichy 1998

Example Configurations

Table top TT1

Bolts B1

Legs L1

Instructions:Attach Legs L1 with Bolts B1 to TT1.

Which one is a derivedconfiguration?

Blanko ‘98 22-23 Oct. 1998 20 © Tichy 1998

Two Types of Versions:

Revision: A software object that was created by modifying an existing one.

Variant: Two software objects that have an important property in common (e.g., the functional specification, interface), but differ in others (e.g. user interface, platform, algorithms, data structures, user groups).

Blanko ‘98 22-23 Oct. 1998 21 © Tichy 1998

More on Versions

Both atoms and configurations have versions.

Source objects always have versions (except perhaps bug reports).

Version group: a set of software objects related via the relations revision-of and variant-of.

Blanko ‘98 22-23 Oct. 1998 22 © Tichy 1998

Organization of Version Groups: Reservation Model

Data base (vault) Workspace (sandbox)

1.31.21.1 2.1 2.2

Fix.1

Par.1 Par.2

Change cycle: 1. checkout (create copy in workspace, reserve template in vault)2. modify copy in workspace3. checkin (move workspace copy into template)

Revision-of

Variant-of

2.32.3

checkout

2.3

checkin

Blanko ‘98 22-23 Oct. 1998 23 © Tichy 1998

Optimistic Model

Data base (vault) Workspace 1

1.31.21.1 2.1 2.2

Fix.1

Par.1 Par.2

Revision-of

Variant-of

2.3

checkout

2.3

checkin

Workspace 2

2.3’

Blanko ‘98 22-23 Oct. 1998 24 © Tichy 1998

The Branch&Merge Models

M.1

R.2

R.1

L.1

L.2

M.2

M.3

M.1

R.2

R.1

L.1

M.2

R.3

M.3

with reservation without reservation

Blanko ‘98 22-23 Oct. 1998 25 © Tichy 1998

Distributed Version ControlGUI.0 GUI.0

Kanji.1

Kanji.2

Kanji.3Kanji.3

GUI.3

GUI.1

GUI.2

GUI.4

Boston Tokyo

GUI.3

Kanji.5

Kanji.6

GUI.5

Kanji.4

Kanji.6

Blanko ‘98 22-23 Oct. 1998 26 © Tichy 1998

Using deltas to reduce space

Data base (vault) Workspace (sandbox)

2.21.1 1.2 1.3 2.1

Fix.1

Par.1 Par.2

A delta is a difference between two revisions. Can be computed in forward or backward direction on checkin. Used to regenerate one revision from another.

2.32.3

checkout

2.3

checkin

2.2

Blanko ‘98 22-23 Oct. 1998 27 © Tichy 1998

Recent improvements to versioning

a. Versioning of arbitrary objects, not just ASCII text binaries text processor documents spreadsheets CAD files

Improvements in delta generators make this possible (also with factor 5 better compression ratio than Unix diff.)

Blanko ‘98 22-23 Oct. 1998 28 © Tichy 1998

Recent improvements to versioning

b. Direct vault access, without checkout, for various tools (compile, print, etc.)

Possible by intercepting file system calls or with special DLLs. No checkout for tools.

c. Hyperlinks to other documents (design, documentation, change requests)

d. Graphical user interfaces.e. Version control built into tools (editors).

Blanko ‘98 22-23 Oct. 1998 29 © Tichy 1998

4. Distributed Version Control

Boston:new GUI

London:main program

Tokyo:Kanji locale

Blanko ‘98 22-23 Oct. 1998 30 © Tichy 1998

Distributed Version Control

Geographically separated, cooperative work requires: remote access to vaults, or replication of vaults and update

propagation. branch mastership by distributed

groups. merging of parallel branches.

Blanko ‘98 22-23 Oct. 1998 31 © Tichy 1998

Solutions to Distributed Version Control

1. Cross-mounted files systems (LAN);2. Remote checkout/checkin

(client/server), central builds (central model);

3. Remote access plus local caching, local builds (decentralized pull model)

4. Periodic database replication, local builds

(decentralized push model);

Blanko ‘98 22-23 Oct. 1998 32 © Tichy 1998

Merging is essential for distributed development

Current merge tools are line based and produce too many false conflicts.

Full semantic merging not practical.Structural merges (smart merges) in

the research stage. (Example: developer A changes identifiers, developer B changes code using the old identifiers. Now what?)

Blanko ‘98 22-23 Oct. 1998 33 © Tichy 1998

5. Configurations

Three major models:a. Composition model (sets of software

objects)b. Change set model (bundles of

changes)c. Long Transaction Model (all changes

are isolated into transactions)

Blanko ‘98 22-23 Oct. 1998 34 © Tichy 1998

a. Composition Model (Parts List)

Configuration is a set of software objects (either atoms or other configurations) or version groups.

When version groups appear in a configuration, then version selection is needed. Best done with rules.

Blanko ‘98 22-23 Oct. 1998 35 © Tichy 1998

Version Selection

Compiler

Back EndLexical Analysis

Syntactic Analysis

SemanticAnalysis

Optimization Code Generation

System Model Version Groups Baseline

Blanko ‘98 22-23 Oct. 1998 36 © Tichy 1998

Version selection by rules

1. Baseline: no versions, no selection needed

2. Developer: select all version checked out by me, + the newest revision on main branch for

others.

Blanko ‘98 22-23 Oct. 1998 37 © Tichy 1998


3. Cautious Developer:select last baseline+ all version checked out by me+ all versions checked in by me.

Blanko ‘98 22-23 Oct. 1998 38 © Tichy 1998


4. Maintainer:select latest released baseline+ all versions checked out by me+ newest versions on corrective branches

5. Cautious Maintainer:select latest released baseline+ all versions checked out by me+ all versions checked in by me

Blanko ‘98 22-23 Oct. 1998 39 © Tichy 1998


6. Reconfiguration:select according to 1 through 5+ variants by attribute (e.g.

platform=Unix)

7. Time machine:select according to 1 through 6+ ignore everything after a cutoff date.

Blanko ‘98 22-23 Oct. 1998 40 © Tichy 1998


8. New Release:select according to 1 through 7+ the newest, stable versions that are

associated with a given configuration of change requests.

9. Automatic configuration:select according to 1 through 8, but search

for objects whose interfaces fit those of a small set of key software objects.

Blanko ‘98 22-23 Oct. 1998 41 © Tichy 1998

b. Change Set Model

A change set is a set of deltas relative to a baseline.

A new configuration is selected by specifying a baseline and a change set to be applied.

Arbitrary combination of changes is not normally possible, because of change interaction.

Blanko ‘98 22-23 Oct. 1998 42 © Tichy 1998

Change Set Model

A

B

C

enh

enh

fix

fix

enh

A.enh

C.enh

B

fix

A.fix

B.fix

C

fix/enh

B.fix

C.enh

A.f/emerge

Blanko ‘98 22-23 Oct. 1998 43 © Tichy 1998

c. Long Transaction Model

Each change happens in a long-running transaction, starting with a configuration.

Subtransactions are started for explorative work.

Versions are not visible except when committing transactions.

Transactions generate work spaces that are completely isolated.

Blanko ‘98 22-23 Oct. 1998 45 © Tichy 1998

Long Transaction Model

Concurrency control schemes coordinate concurrent transactions. Locking Optimistic concurrency control

Long transactions represent workspaces and development paths.

Disadvantage: Concurrency control limits change propagation in the repository.

Blanko ‘98 22-23 Oct. 1998 46 © Tichy 1998

Recent improvements

A. Configuration management on the Web World-wide accessibility Rich infrastructure Platform independence (Anytime, anywhere configuration

mgmt.)B. Automatic configuration, starting with

a small set of software objects.

Blanko ‘98 22-23 Oct. 1998 47 © Tichy 1998

6. Change Tracking

Maintain a database of change requests;

Trace their states (submitted, accepted, in progress, in test, accepted, etc.);

Collect change configurations;Provide a multitude of reports;Link change requests to solutions.

Blanko ‘98 22-23 Oct. 1998 48 © Tichy 1998

Traceability Links

Requirements:This system shall….……………….………. … ..

Design Part 1:

C1

C2

Design Part 2:

D1

D2

class C1 { method c11(){ } method c12 (){ }}

Bug report27Bug report26

Bug report25

Blanko ‘98 22-23 Oct. 1998 49 © Tichy 1998

Change Tracking Solutions

Use a database (relational or special purpose),

Are typically not distributed and permit little customization.

Next generation change trackers use standardized database interface (SQL), so customer can choose the database, even a distributed one.

Blanko ‘98 22-23 Oct. 1998 50 © Tichy 1998

7. Process Support

Checkout/checkin with reservation model is too primitive for large teams.

Some tool vendors provide a process engine which the customer can program. This is difficult, time consuming and error prone for most production teams.

Ready-made parallel process models are usable, but still somewhat simple-minded.

Blanko ‘98 22-23 Oct. 1998 51 © Tichy 1998

A Parallel Development Process

Each developer or small team works on ist own branch.

Accepted changes are on a separate branch.Periodically, developers start an update process

that merges newly accepted changes into their development branches.

After completion, developer submits change set to integrator.

Integrator/tester runs additional merge; if none or simple to resolve, tests and accepts.

Blanko ‘98 22-23 Oct. 1998 52 © Tichy 1998

Parallel Development Model

Rel.1

Team2.2

Rel.2

pickup/accept

Team2.3

update

Rel.3

pickup/accept

Team1.1

collectchange set

Team2.1

collectchange set

Blanko ‘98 22-23 Oct. 1998 53 © Tichy 1998

Summary

Software objects, version groups, configurations;

Checkin/checkout model;Composition model with version selection;Change set model, transaction model;Distributed SCM;Change tracking;Process support.

Blanko ‘98 22-23 Oct. 1998 54 © Tichy 1998

6. Outlook

What is well understood? Storing and tracking changes. Reliable, repeatable version selection. Automatic configuration processing

(Make and Make-oids)

Linking change requests with changes.

Blanko ‘98 22-23 Oct. 1998 55 © Tichy 1998

Outlook cont’d

Where is further development necessary? Distributed SCM; Software distribution, installation, and

update in networks; Automatic configuration inference; Parallel development process support; Process support for distributed

development; Versioning of webs.

Blanko ‘98 22-23 Oct. 1998 56 © Tichy 1998

7. Conclusions

SCM is an established discipline of recognized, practical use.

Useful tools of various sophistication are available.

There are a number of new requirements for SCM.

SCM tools are often technically advanced.

software configuration management

Documents