http://alienbuild.cern.ch:8888 1 / 22 aliroot and alien build integration and testing system

http://alienbuild.cern.ch:8888 http://alirootbuild.cern.ch:88891 / 22

AliRootAliRoot and and AliEnAliEnBBuild uild IIntegration and ntegration and TTesting esting SSystem (BITS)ystem (BITS)

Alina [email protected] Vladimirovna [email protected]


BITS purpose

Having a current, up to date, installable binary image of AliEn and AliRoot for different architectures

Automate the process of building AliEn and AliRoot components

Introduce a better understanding of the dependencies between packages

Perform unit and functional tests during build time, deploying error alerts immediately

Provide a web interface that allows monitoring of the current build status and remote access to logs


Supported Platforms

AliEn and AliRoot are supported for the following platforms: SLC5 32 and 64 bits SLC5 Itanium IntelMac 32 and 64 bits Leopard IntelMac 32 and 64 bits Snow Leopard (to be added) Ubuntu 9.10 64 bits (testing purpose)

AliEnBITS Web Interface Main Menu


Build environment overview

Based on the Konstruct framework used to build KDE and dependencies which in itself is based on BSD ports system

Each package is defined as a directory in a hierarchy For each package there is a Makefile containing a set of

variables: Name, version, author, web page License information Build and runtime dependencies Master sites for downloading the package source

Additional patches needed for AliEn/AliRoot environment Checksums for verifying the integrity of the downloaded

sources


Build process[1]

Get the available Releases Select the one to be (re)built

Fetch the latest changes for this release Do a cvs update

Determine the build order Analise the dependencies between packages

Select the packages that have to be rebuilt Based on modifications received on cvs update Based on last build status Based on package dependencies


Build process [2]

For each package Clean its environment Fetch source archive from master sites Build, install and run tests Create the binary image for binary installations

Generate package web page Build / SLOC / Graph dependencies Cleanup environment Send notification email when status changes


AliEn: packages graph


AliEn: available builds

The current build system maintains the following versions of AliEn across all supported platforms:

The build also provides the following AliEn distributions (the sizes are for v2-18 SLC5 64 bits):

Distribution Packed size (MB) Unpacked size (MB)

user 18.7 71.1

workernode 44.8 156.4

client 62.2 279.7

vobox 126.0 476.1

all 160.1 660.6

http://alienbuild.cern.ch:8888


AliEn: Release Web Page


Package Web Page

AliEn Package Details- general information about the package- links to the Makefile + the applied patches

AliEn Package Dependencies- list of packages that have to be build before building this package

AliEn Packages Depending on it- the list of packages that can't be build before building this package


AliEn: user distribution

Latest installation instructions for the user distribution can be found at http://alien2.cern.ch

User distribution contains the necessary libraries for compiling ROOT with AliEn Grid support. The detailed instructions can be found on the same page as above.

wget http://alien.cern.cern/alien-installer

chmod +x alien-installer

./alien-installer

[pcepalice10] /home/alienmaster > ./alien-installer ###################### Automated AliEn user installer ################ Installing in the default directory: /home/alienmaster/alien If you wish to install in a specific directory do: ./alien-installer -install-dir Platform set as: x86_64-unknown-linux-gnu Waiting 10 seconds. Press 'Ctrl+c' to exit the installation........Starting installation Downloading user distribution...................................Done Extracting the files........................................Done Relocating the user installation..............................Done Installation finished!


AliEn Tests

– AliEn Tests are performed after each build

• Test the installation procedure

• Perform different functional tests

• Save the log files for the failed tests

• Save the log files of all AliEn services during the test

• Publish relevant logs


Release Test Web Page

General information

Main test steps and their results

Results per test types

Logs:- AliEn services logs- failed tests logs

Global log of the build process


AliRoot: build specifics

Same build process as AliEn– Checks for new releases or changes in the current

development releases– After each build performs quality and functional

tests

http://alirootbuild.cern.ch:8889


AliRoot: web interfacehttp://alirootbuild.cern.ch:8889


Adding a new AliRoot release new releases can be added using a web interface For each AliRoot release we need to specify the dependencies

– AliEn version– Geant3 version– Root version

Release administrationhttp://alirootbuild.cern.ch:8889

Available version list extractedfrom source repositories


AliRoot benchmarks

AliRoot benchmarks are triggered after each build

The results are published on a web page

The logs are available from any remote location

History charts to compare with previous builds / releases


AliRoot: on the GRID

After a complete build and in case of no errors during the tests, AliRoot packages can be registered into AliEn using a web interface from MonAlisa


AliRoot: Geant4

Geant4-v2010xxxxGeant4-v2010xxxx

CLHEPCLHEP

Geant4_vmcGeant4_vmc

Geant4Geant4Data FilesData Files

Build dependency

Installation pointROOTROOT

The Grid

Geant4 unlike its predecessor is written in C++ and has a completely new installation procedure

AliRoot BITS for Geant4 packages was adjusted in order to facilitate automatic compilation and installation procedures

The final Geant4 package installed on the GRID (see left diagram) contains a number of dependent packages, libraries and data files.


Perl upgrade from 5.8.8 to 5.10.1Always use the latest packagesEnabling thread support for improving AliEn performanceIssues

Some packages are really old and they are not supported anymore

Rebuilding a distribution takes ~a couple of hours thus each iteration is slow

Problems are really difficult to spot Packages that don't have a clear building and installation script Because of the big number of packages and dependencies one

change in a package can trigger a whole chain of problems Some packages are not supported at all for some platforms,

usually for Mac

AliEn: current work [1]


Adding a gLite test environment, without installing gLite with AliEn

Issues: installing gLite on all build servers independent of the build and testing process, certificates etc

Synchronizing the build number between different platforms

You can have AliEn 2-18.23 for 32bits and AliEn 2-18.30 for 64bits

Major cleaning of AliEn packages (somewhere in the future)

AliEn: future plans


Usual work: Building and registering to AliEn different tags of AliRoot

Current work: Implementing an automatic email alert system

If AliRoot fails, check where it failed and email the responsible persons about it

This applies only to the trunk, that is build every night We are in test phase

Future plans : Automatically register packages to AliEn, now it is done

using an intermediate MonAlisa webpage

AliRoot: present and future

http://alienbuild.cern.ch:8888 1 / 22 aliroot and alien build integration and testing system

Documents