The common Software The common Software Repository for SL and STRepository for SL and ST

(The SCaM Project)(The SCaM Project)

A. Bragg (SL/CO), E. Hatziangeli (SL/CO), J. Patino (ST/MO)

PlanPlan The project The solution Razor The elements of SCM The Functionality

The system The development environment

The Work Documentation and Training The Benefits

Provide the means for users to develop and exchange software, using a common repository, based on available industrial products

Why the SCaM project?Why the SCaM project?

Replace and extend the functionality of SL/CO software management system (SLAPS)

Give control to the users over their software development process Parallel development Quick fixes by Operators without the need of administrator Allow version and release management for in-house software, as well as external

software integration Improve the maintenance of the Operational Software Unify all diverse versioning systems efforts (SLAPS, SL/BI cvs, ST/MO

sccs, PS/CO rcs and sccs, ...) Create a software management system that is easy to extend and maintain

The people involvedThe people involved

Advisors SL/BI Jannes de Vries ST/MO Roberto Bartolome LHC/VAC Isabelle Laugier PSSL Convergence Project Alessandro Risso

Primary Team SL/CO Andrew Bragg & Eugenia Hatziangeli [pr. leader] ST/MO Jose Patino

Project Sponsors SL/CO/AP Pierre Charrue ST/MO Pierre Ninin

Observers PS/CO Alessandro Risso IT/IPT Arash Khodabandeh (LHC experiments: SPIDER SCM sub-project)

Major Project MilestonesMajor Project Milestones Identify problem and create project SCaM [SL-Note 97-44 (CO), Jun. 97]

Collect User Requirements [SL-Note 97-59 (CO), Oct. 97]

Market survey for industrial and public domain tools=> Evaluation report and recommendations for tool and procedures to SL-CO and ST-MC management [SL-Note 98-022(CO) Mar 98]

Re-commit SL-CO & ST-MC management to solution chosen [SL-Note 98-34 (CO), Jun. 98]

Purchase and installation of the chosen tool (Nov. 98)

Software management system implementation (end Aug. 99)

The solution packageThe solution package

1) Tool => Razor

2) Software Management Procedures => User and administrator

3) Administration and User support for the:Software Management tool and the Repository

Software Management Procedures

New User Training

The team will ensure that the Software Management activities for Operational Software development are carried out as planned.

Why Razor?Why Razor? It demonstrates a good all around functionality

good management of software versions, supports baseline and release management provides problem reporting management

Very adaptable, with GUI and command line interface

It is simple to learn, to use and to maintain

Good level of technical support and constant product evolution

It is commercial and well priced

It is based on floating or site license scheme 14 floating licenses available (28 concurrent developers)

More on Razor ...More on Razor ... Architecture

License Manager

Repository Server

Licenses

Repository Files

Archive Engine

Razor User

Interface

Platforms

• HPUX• Linux• Windows NT/95• AIX (IBM)

• SCO• Solaris 2.4 (SunOS 5.4) or higher• Solaris x86• SunOS 4.1.x

• IRIX (SGI)• OSF (DEC)

UNIX UNIX Win95/NT

Win95/NT UNIX

Win95/NT

Win95/NTUNIX

LAN Connection

WA

N C

onne

c tio

n

License Manager Repository Server

Network

Software Configuration Software Configuration Management ??Management ??

Version Management

Problem Management

Release Management

Software Repository

It is about “what changes” in our development environment, while we are working and programming

Control and track “what changes” during development

Trilogy of RazorTrilogy of Razor

Versions

Issues

Threads

Archives changes and coordinates efforts amongst developers

Manages all files related to

a released product

Records and coordinates things yet to be accomplished

Release Management

Version Management

Repository

Problem Management

Version ManagementVersion Management“….what the hell, let’s call him Ramses, too….”

Ramses the 1st, inventor of the versioning, at the birth of his son, 1784 B.C.

File: hello.c Version 1.0#include <stdio.h>main(void){

printf(“hello world\n”);}

File: hello.c Version 1.1#include <stdio.h>main(void){

printf(“Hello World\n”);}

File: hello.c Version 1.2#include <stdio.h>main(void){ printf(“Hello New World\n”);}

Version 1.0 Version 1.1 Version 1.2

Version 1.2.1.1

Version 1.3

Version 1.2.1.2

Version 1.4

Version 1.5

Release ManagementRelease Management

1.3 1.4 1.5 1.6 1.7

1.2 1.3 1.4 1.5

2.4 2.5 2.6 2.7 2.8

calories.c

meal.data

food.h

get_thin 1.2 get_thin 1.3

• Grouping (threading) software as a collection of specific versions of files is done for:

• release or/and build management

• try out certain combination of files for testing or prototyping

• The exact state of each file, as they relate to a specific named release, can be reproduced

• Each release is given a unique name and evolves over time.

• The meaning and the utility of a release is up to the user

Problem ManagementProblem Management

File: hello.c

Version 1.0

#include <stdio.h>main(void){ printf(“hello world\n”);}

File: hello.c

Version 1.1

#include <stdio.h>main(void){ printf(“Hello World\n”);}

File: hello.c

Version 1.2

#include <stdio.h>main(void){ printf(“Hello New World\n”);}

File VersionsRelease Versions

calories.c 1.4

Release: get_thin

meal.data 2.4

food.h 1.2

Version: 1.2

calories.c 1.6

Release: get_thin

meal.data 2.7

food.h 1.5

Version: 1.3

Change Requests

Bug Reports Software changes, arising from Bug Fixes or Change Request are managed through the software lifecycle

Track the changes by relating the versions of the files/releases with the Problem Reports or the Change Requests

RepositoryRepository“…. you know, I preferred the way you had the pyramid yesterday….”

Ramses the 5th, inventor of the vault, shortly before his entombment, 1637 B.C.

All versions of files

Software Releases

Software Change Requests,Problem Reports

RCS SCCS

All previous development investment in RCS or SCCS can be injected in the new repository

We are independent from the tool

Located on REPSRV

• HPUX 10.20

• 8GB of space

• on regular backup schedule

• 365 days/year available

• on power fail safe

Repository ContentsRepository Contents

• Source files

• Header files

• Data files

• Scripts

• Documentation files

• Help files

• Icons

• Bitmaps

• Other graphics

• Configuration files

• Spreadsheets

• Off the self software

All versions of files

Reproducible code (binaries, output data files,…)

ASCII Files RCS : The last

version is kept along with backward deltas for the older versions

SCCS: The first version is kept along with forward deltas for the newer versions

Binary Files

Stored in compressed format

The set up of the repositoriesThe set up of the repositories

ST-MO projects

SL-CO projects

SL-OP projects

SL-BI projects

SPS2001

PS-SL-conv

…. . . .

Alarm Software LHC/VAC Software

Minimize user inconvenience

User should not need to traverse an enormous tree to have access to his software

Possibility to customize according to the local development philosophy

Repository Login Interface

User Several repositories with software related by a

common theme, set up using standard defaults

ST LHCALARMSSL …...

SL

The set up of each repositoryThe set up of each repository

A software group is a “narrower” collection, contained within a repository

SL-COSL-BI

SL-OP

SPS2001 PSSL-conv

groups

folders

The folders are the simplest collection. They contain files and are a way of grouping related files

The plan…The plan…

The philosophy

Access Control

PEOPLE files

The structure of a project in the repository

File name/extension, Symbolic-Link Control Accessing the right Repository and starting

the GUI’s

The philosophyThe philosophy

Underlying philosophy: Who do I want to have write access to my software?

(individual files and projects)

Myself and my project team Others in my group in case of my absence

(holidays, sickness etc etc…) There is a complete history of repository

activity linking actions to users

“Do unto others as you would have them do unto you” Someone a long time ago

Access ControlAccess Control Applies to both (Unix) Command Line and GUI

PC Windows95/NT Client Transparent to user - No additional passwords Possible to configure, per Repository

per Project

Any action which can affect a file or the attributes associated with a file are under access control

Two levels of validation 1. PEOPLE file1. PEOPLE file 2. User’s CERN Division/Group2. User’s CERN Division/Group

PEOPLE file?PEOPLE file?

Reflects the project team / collaborations

A PEOPLE file is a list of valid UNIX user id’s

Separated by ‘white space’

Allow write access to the structure below

Used to obtain e-mail information for group

validation

Maintained by the project team

Scenario.3.

Scenario.2.

Scenario.1.

The file PEOPLE contains the group of people that they have write access under that structure. The lowest level PEOPLE file has precedence, over the upper one, for the directory structure below.

Any connection to an existing or proposed Division/Project is entirely co-incidental.The names have been changed to protect the innocent.

SPS2001ST-MO

cryo tds

PEOPLE inc lib src

PEOPLE inc lib src

PEOPLE

SL-CO

APWS

Repository

bctmeaslib

PEOPLE inc lib src

File/Extension, File/Extension, Symbolic-Link ControlSymbolic-Link Control

Only valid file extensions may be introduced, e.g. *.c, *.java, *.doc

Certain file’s without extensions may be introduced, e.g. PEOPLE, Makefile, readme

Acceptable extensions are defined per software group Why? - Avoid unwanted files e.g. core

New file(s) / extension(s) are entered by the developers

Symbolic links are NOT allowed!

Accessing the right Repository Accessing the right Repository and starting the GUI’sand starting the GUI’s

From a Unix environment - GUI’s

Logging into the repository

Starting the Razor Suit

Accessing the right Repository Accessing the right Repository and starting the GUI’sand starting the GUI’s

(Continued…)(Continued…) From a PC windows95/NT client environment

26

Problem ReportingProblem Reporting(Issues)(Issues)

PlanPlan

What is the ISSUES program

Operations on a Problem Report

Problem Report lifecycle

Attributes of a Problem Report

What is the ISSUES What is the ISSUES program?program?

Main list: Displays each problem report or change request made within a software domain

The main display:

It is the Problem Tracking System part of the Razor trilogy

Operations on a Problem Operations on a Problem Report Report

View/modify problems reports Delete problem reports Print problem reports Relate problems reports to an activity in either

the file version control or release management aspect of Razor

Make filtered selections on all the archived reports

Generate reports

GUIEmail templatesWeb

Create new problems reports

Problem Report lifecycleProblem Report lifecycle

The possible STATES and TRANSITIONS of an issue are:

(e-mail notifications on state changes)

Submitted

Closed

Assigned

Feedback Suspended

RejectedAccepted

Attributes of a Problem Attributes of a Problem Report (I)Report (I)

Common Attributes

Attributes of a Problem Attributes of a Problem Report (II)Report (II)

Category

Assigned person

State

Attributes of a Problem Attributes of a Problem Report (III)Report (III)

Two text panes Problem Description (Originator) Actions Taken (Assigned Person)

User Development Area

Che

ck o

ut

Ch

eck

in

Cre

ate/

build

/test

a r

elea

se

Repository

Public Software Area

(For compilation and linking of user software)Libraries: SL_EQUIP, SL_RPC, SL_MEAS,SL_UMMI,

…

Includes: nc.h, Mequip.h, sl_measlib.h,

binaries:

Configuration:.Xdefaults, etc.

Installation procedure

Operational Software Area

(Used to operate the machines)Products:tz_drive, logbook, shiftlog,

…

Complete releases with sources

Inst

alla

tion

pro

ced

ure

Operational Software InstallationOperational Software Installation

Install software release

Install public software

Public SW Area

lib include bin

Build Support EnvironmentBuild Support Environment

Support for csh environment Support the existing SLAPS environment

$SL_INCLUDE, $SL_LIB, $SL_BIN,... $SL_RPC, $SL_EQUIP,…

Support for sh like environment

General Makefile (new projects)

Nex

t

Cur

rent

Pre

viou

s

……

…..

Previous versions of public softwarePrevious versions of public software

Current (default) public softwareCurrent (default) public software

Newly released public softwareNewly released public software

The Operational SoftwareThe Operational Software Operational software will be released in and run from a designated

distribution area

Each release will be complete with sources to allow quick bug fixes

The Operator will be able to quickly revert to previous working version of a product (global for all consoles)

The Operator will be able to access any available version (previous or future) of a product (local to a console)

/user/ops/software/releases/product1 product2 product3 ….. productx

Under each product:

/user/ops/software/releases/product1/v1.0 v1.1 v1.2 PREVIOUS (Previous working) v2.0 v2.2 CURRENT (In operation)

v2.3 NEXT

Roles and CapabilitiesRoles and Capabilities

Developers

introduce new files in his software project

check out their software

check in their software

create/modify releases of their software projects

extract releases of software products from repository

build releases of their software projects

install software releases in the Operational Software Area

receive/create/modify Problem Reports and Change Requests

Librarians

install public software (libraries, includes,…) in Public Software Area

Administrator

manage the tool, the repositories, the software management procedures and new user training

Defined per software group according to group requirementsDefined per software group according to group requirements

Some of the Policies and Procedures

PEOPLE file (Access control to a product in the repository)

Control over the type of files in the repository (File Extension control)

Public software installation via an Automatic Installation Procedure

Operational software installation via an Automatic Installation Procedure

Policies and RecommendationsPolicies and Recommendations

“Write your code for somebody who does not understand it, because by the time you come back to it, that somebody else will be you….”

Some of Recommendations

Files self-identification (RCS keywords)

Relation of file/release versions to PRs or SCRs

Naming scheme of an Operational release

Who does the work?Who does the work? Project team

Tool installation Set up of the software repositories Set up of user development environment Set up of Operation Software distribution area Implementation of automatic installation procedures User training and documentation

Project team + Developers Porting of software project (SLAPS, SL/BI, ST/MO, LHC/VAC, ALARM, …) inside

the repository

Administrator(s) Administration of the tool and management of software repository User support User training

DocumentationDocumentation User Documentation

Administration Documentation

SCaM project web page: http://venice.cern.ch/~slaps/hot/SCAM_II/index.html

Razor documentation available: http://venice.cern.ch/~slaps/hot/razorDocs/rz_web/razortoc.htm

The official Razor internet site:http://www.tower.com

TrainingTraining

We will provide training Users Librarians

Small groups of users - Targeting their specific needs

Lasting no more the 1½ to 2 hours maximum

There will be several training sessions

Future training sessions will be arranged as required

The situation todayThe situation todayOperational Software unable to identify the versions of the majority of the operational software unable to identify/control all elements that influence the operational software unable to control changes of the operational software and identify the initiators not easy to run another version of the operational software

Development Environment public software change often and without warning difficult to debug, since reproduction of the exact version of the software

concerned is often not possible non unique software and hardware configuration of the development platforms

External Software software written by external contractors cannot be properly synchronised with

software written in CERN, because rules to integrate software are not well specified

What will it change?What will it change?

All software from different groups will reside in a single repository

Good working practices and procedures Version management (at least) for existing project in

maintenance phase Software improvements

knowledge of where each software component is and what state is in stability, because changes affecting operational software will be under control useful reports available about the evolution and changes of the software

Centralization of repository system management

Free some resources

Benefits for the Operation teamBenefits for the Operation team

Delivery of consistent operational software

Being able to trace and identify any component of an operational system

Being able to incorporate quick bug fixes in the operational environment

Minimisation of uncontrolled changes of the operational software

Able to run other (future or past) versions of the operational software

Able to “revert to previous working” version of the operational software

Platform independent repository access (HPUX, Win95/NT, AIX, Linux,..)

Benefits for developersBenefits for developers

Version management

Software is kept safely in the repository

“Care free” parallel development

Manage the process used to develop your software

Project lifecycle management

Projects in maintenance New Projects

All Projects

Guidelines and procedures concerning the integration and installation of operational software and the introduction of software modifications

A single software configuration management environment

Transparency of software and documentation

Important for subcontracted software development

Why this presentation?Why this presentation?

Make you aware of this project Receive feedback on our proposed solution Encourage you to try the new system

We would like to produce a Software Management system that works

Iterative process

Help and feedback from the software developers

Thank you for comingThank you for coming

Any questions?