applications area plan - cernlcgapp.cern.ch/project/mgmt/appplanv2.doc · web viewlcg...

LCG Applications Area Plan

Version 2 – 21 August 2003


Editing History

Draft 0.1, October 7, 2002, T. Wenaus: First draft assembled from (mainly) existing planning material, drawing also on material developed in the context of the blueprint RTAG.

Draft 0.2, November 26, 2002, T. Wenaus: Project update, intro update. Renamed as version 1.0

Version 2.0, August 22, 2003, T. Wenaus: Update for the SEAL, PI, and simulation projects. Incorporate material from Pere’s SEAL project plan that is applicable to the full applications area. Incorporate risks material as appendix.

– 2 –


Table of Contents

1 Executive Summary 4

2 Scope 5

3 Requirements 6

4 Software Architecture 8

4.1 Software structure 8

4.2 Component model 9

4.3 Software reuse 9

4.4 Distributed operation 9

5 Software Domains 10

5.1 Software process and infrastructure 10

5.2 Foundation and utility libraries, core services 10

5.3 Persistency and data management 10

5.4 Detector simulation, geometry 11

5.5 Analysis 11

5.6 Grid middleware interfaces 11

5.7 Reconstruction, event model, calibration 11

5.8 Trigger/DAQ 11

6 The Applications Area Projects 12

6.1 Software Process and Infrastructure (SPI) 12

6.2 Persistency Framework (POOL) 12

6.3 Core Libraries and Services (SEAL) 12

6.4 Physicist Interface (PI) 12

6.5 Simulation 12

7 Project Management and Communication 13

7.1 Overall organization and management 13

7.2 Architects Forum 14

– 3 –


7.3 Focus on Experiment Need 14

7.4 Coherence and Communication 14

7.5 Project tracking 15

7.6 Risk management 15

7.7 Reporting and reviews 16

8 WBS, Milestones and Deliverables 16

9 Manpower Resources 17

10 External participation and collaboration 19

11 Technical process 19

11.1 Process Model 19

11.2 Iterative development 20

11.3 Methods, Tools and Techniques 21

11.4 Infrastructure 21

11.5 Configuration Management 22

11.6 Verification and Validation 23

11.7 Documentation and training 24

11.8 Quality Assurance 24

12 Applications Area Activities in LCG Phase 1 25

Phase 1 of the LCG Project runs through 2005. Applications area activities in this phase are: 25

Prepare the LHC computing environment 25

o Provide the common tools and infrastructure for the physics application software 25

o Grid interfaces to application software 25

o Integrate applications software into progressively more complex grid prototypes to validate the computing model 25

o Seek opportunities for re-use of project results 25

13 References 25

– 4 –


1 Executive SummaryThis is an internal, working document of the LCG Applications Area to document the status of planning in the applications area. Material from this document can be adapted and exported for use in ‘official’ LCG PEB and other planning documents.

2 ScopeThe Applications Area (AA) is one of four activity areas managed by the Project Execution Board (PEB) of the LCG. The AA is concerned with developing, deploying and maintaining that part of the physics applications software and associated supporting infrastructure software that is common among the LHC experiments, with the experiments determining via the SC2 what software projects will be undertaken in common. The expected scope includes common applications software infrastructure, frameworks, libraries, and tools; common applications such as simulation and analysis toolkits; and assisting and supporting the integration and adaptation of physics applications software in the Grid environment. Applications area activities anticipated at the project’s inception are grouped in four general topics:

1) Application software infrastructure: Basic environment for physics software development, documentation, distribution and support. General-purpose scientific libraries, C++ foundation libraries, and other standard libraries. Software development tools, documentation tools, quality control and other tools and services integrated into a well-defined software process.

2) Common software frameworks: Common frameworks, toolkits and applications supporting simulation, reconstruction and analysis in the LHC experiments. Adaptation to integrate external frameworks and toolkits provided by projects of broader scope than LHC. Examples are GEANT4 and ROOT.

3) Support for physics applications: Integration and deployment of common software tools and frameworks required by the LHC experiments, particularly in the distributed environment of LHC computing. ‘Portal’ environments required to mask the complexity of the Grid from researchers while providing fully distributed functionality. Direct assistance to the experiments at the interface between core software and the grid. Support for adaptation of physics applications to the grid environment.

4) Physics data management: Tools for storing, managing and accessing data handled by physics applications, including calibration data, metadata describing events, event data, and analysis objects. Database management systems supporting both relational and object-based data management, including support for persistent storage of objects. Persistency support for common applications. Provide data management services meeting the scalability requirements of the LHC experiments, including integration with large-scale storage management and the Grid.

For those projects defined by the SC2 which fall within this scope, the applications area is responsible for building a project team among participants and collaborators; developing an approved work plan; designing and developing software that meets experiment requirements, adheres to a coherent overall architecture, and performs within the distributed LHC computing

– 5 –


grid; assisting in integrating the software within the experiments; and providing support and maintenance.

Guidance and oversight from the experiments on applications area activities comes from the SC2 committee, from the experiment delegates on the PEB, and vitally from the direct participation of experiment developers and architects in common project activities and in the Architects Forum. The SC2 sets requirements and approves the strategy and workplan developed by the PEB.

The PEB and its management team

acts on SC2 recommendations for common projects

develops and gets agreement on strategy and workplan

o goals, milestones, deliverables, schedule, resource allocation, prioritization

o assembly of and buy-in from implementation teams

o SC2 approval of strategy and workplan

manages and tracks the progress and direction of the project

ensures conformance with SC2 recommendations

identifies areas for study or resolution by SC2

The work of the applications area is conducted within projects. As of August 2003 there are five active projects: software process and infrastructure (SPI), persistency framework (POOL), core libraries and services (SEAL), physicist interface (PI), and simulation.

3 RequirementsThese are the important high-level requirements for LCG applications software.

3.1.1 LifetimeLCG software design should take account of the >10 year lifetime of the LHC. Software environments and optimal technology choices will evolve over time. The LCG software itself must be able to evolve smoothly with it. This requirement implies others on language evolution, modularity of components, use of interfaces, maintainability and documentation. At any given time the LCG should provide a functional set of software with implementation based on products that are the current best choice.

3.1.2 Languages

The standard language for physics applications software in all four LHC experiments is C++. The language choice may change in the future, and some experiments support multi-language environments today. LCG software should serve C++ environments well, and also support multi-language environments and the evolution of language choices.

– 6 –


3.1.3 Distributed applicationsLCG software must operate seamlessly in a highly distributed environment, with distributed operation enabled and controlled by components employing Grid middleware. All LCG software must take account of distributed operation in its design and must use the agreed standard services for distributed operation when the software uses distributed services directly.

3.1.4 TGV and airplane workWhile the software must operate seamlessly in a distributed environment, it must also be functional and easily usable in ‘disconnected’ local environments.

3.1.5 Modularity of componentsLCG software should be constructed in a modular way based on components, where a software component provides a specific function via a well-defined public interface. Components interact with other components through their interfaces. It should be possible to replace a component with a different implementation respecting the same interfaces without perturbing the rest of the system.

3.1.6 Use of interfacesThe interaction of users and other software components with a given component is entirely through its public interface. Private communication between components would preclude later replacement of the implementation of a component with another one. They should be designed for stability and for minimal perturbation of external software when they are required to change.

3.1.7 Interchangeability of implementationsThe component architecture and interface design should be such that different implementations of a given component can be easily interchanged provided that they respect the established interfaces. Component and interface designs should not, in general, make assumptions about implementation technologies; they should be as implementation-neutral as possible.

3.1.8 IntegrationA principal requirement of LCG software components is that they integrate well in a coherent software framework, and integrate well with experiment software and other tools. LCG software should include components and employ designs that facilitate this integration. Integration of the best of existing solutions as component implementations should be supported, in order to profit from existing tools and avoid duplication.

3.1.9 Design for end-usersWhere design decisions involve choices between making life easier for the developer (ease of implementation) vs. making life easier for the user (ease of use), the latter should be given precedence, particularly where the targeted users are non-specialists.

– 7 –


3.1.10 Re-use existing implementations

Already existing implementations which provide the required functionality for a given component should be evaluated and the best of them used if possible. Use of existing software should be consistent with the LCG architecture.

3.1.11 Software qualityFrom design through implementation, LCG software should be at least as good and preferably better in terms of quality than the internal software of any of the LHC experiments.

3.1.12 PlatformsLCG software should be written in conformance to the language standard. Platform and OS dependencies should be confined to low level system utilities. The current main development platform is GNU/Linux on Intel. Support for the Intel compiler under Linux, Solaris environment on Sun and Microsoft environment on Intel should also be provided. Production software, including external software, will be provided on all these four environments. Development tools and prototype software may be deployed and supported on only a limited number of platforms.

3.1.13 Trigger/DAQ environmentAlthough the Trigger and DAQ software applications are not be part of the LCG scope, it is very likely that such applications will re-use some of the core LCG components. Therefore, the LCG software must be able to operate in a real-time environment and it must be designed and developed accordingly, e.g. incorporating online requirements for time budgets and memory leak intolerance.

4 Software ArchitectureSee the Architecture Blueprint RTAG report for the details of the high level LCG software architecture. The main points are highlighted here.

4.1 Software structure

The overall software structure consists of

At the lowest level: the foundation libraries, utilities and services employed in building the basic framework. Foundation libraries are fairly independent class libraries (e.g. STL, or a library providing a LorentzVector). Utilities and services include higher level, possibly optional software such as grid middleware and utility libraries. This level is covered by the SEAL project.

The basic framework: A coherent, integrated set of core infrastructure and core services supporting the development of higher level framework components and specializations. For example, the object “whiteboard” and object dictionary by which all parts of the system have knowledge of and can access to the objects of the system. Components and services for this level are provided by the SEAL project. Experiment frameworks themselves are the in-house responsibility of the experiments.

– 8 –


Specialized frameworks: Frameworks for simulation, reconstruction, interactive analysis, persistency, etc. They employ and extend the services and infrastructure of the basic framework. The POOL, PI and Simulation projects are active at this level.

Experiment applications: The applications built on top of the specialized frameworks. They are generally specific to the experiment and not in LCG scope.

Figure 1: Software structure

4.2 Component model

LCG software is modular, with the unit of modularity being the software component. A component internally can consist of a number of collaborating classes. Its public interface expresses how the component is seen and used externally. The granularity of the component breakdown should be driven by that granularity at which replacement of individual components (e.g. with a new implementation) is foreseen over time.

4.3 Software reuse

The design and implementation of AA software emphasise the reuse of existing software wherever practical. Almost all of the work to date leverages existing software. Some external packages are used for key elements of the AA software, such as the use of ROOT for POOL’s object persistency and for its analysis environment. Software taken or derived from experiment software, HEP community software, and open source software are important contributors.

4.4 Distributed operation

The architecture should enable but not require the use of distributed resources via the Grid.

– 9 –

Basic Framework

Foundation Libraries

Simulation Framewor

k

Reconstruction

Framework

Visualizatio

n Framework

Applications

. . .

Optional Libraries

OtherFrameworks


The configuration and control of Grid-based operation should be encapsulated in components and services intended for these purposes. Outside of these, grid-based operation should be largely transparent to other components and services, application software, and users.

Grid middleware constitutes optional libraries at the foundation level of the software structure. Services at the basic framework level encapsulate and employ middleware to offer distributed capability to service users while insulating them from the underlying middleware.

5 Software DomainsThe principal physics applications software domains for the LHC experiments are illustrated here. Software support services (management, packaging, distribution etc.) are not included. The blue areas are those in which common project activity is expected to take place. All are now covered by active AA projects with the exception of grid services (pending the outcome of the ARDA RTAG).

EventGeneration

Core Services

Dictionary

Whiteboard

Foundation and Utility Libraries

DetectorSimulation

Engine

Persistency

StoreMgr

Reconstruction

Algorithms

Geometry Event Model

GridServices

I nteractiveServices

Modeler

GUIAnalysisEvtGen

Calibration

Scheduler

Fitter

PluginMgrMonitor

NTupleScripting

FileCatalog

ROOT GEANT4 DataGrid Python Qt

Monitor

. . .MySQLFLUKA

EventGeneration

Core Services

Dictionary

Whiteboard

Foundation and Utility Libraries

DetectorSimulation

Engine

Persistency

StoreMgr

Reconstruction

Algorithms

Geometry Event Model

GridServices

I nteractiveServices

Modeler

GUIAnalysisEvtGen

Calibration

Scheduler

Fitter

PluginMgrMonitor

NTupleScripting

FileCatalog

ROOT GEANT4 DataGrid Python Qt

Monitor

. . .MySQLFLUKA

Figure 2: Physics applications software domains. Grey domains are not within LCG applications area scope, though they may be users of project software. The indicated

products and components are examples and are not a complete list.

5.1 Software process and infrastructure

Software development processes, infrastructure and tools supporting the development, documentation, testing, distribution, use and maintenance of software. Covered by the SPI project.

– 10 –


5.2 Foundation and utility libraries, core services

See the software architecture section. Covered by the SEAL project. Math and statistics libraries are also covered.

5.3 Persistency and data management

This domain covers object persistency, metadata cataloging, event data storage and management; conditions data storage management. Covered by the persistency framework project developing POOL and the common conditions database software.

5.4 Detector simulation, geometry

The common aspects of these domains covering physics event generators, physics simulation of the detectors, and the detector geometry required by simulation and reconstruction, are covered by the simulation project. Event generators themselves are outside the scope of the LCG. Ancillary services surrounding event generators (e.g. standard event and particle data formats, persistency, configuration service), and support and distribution of event generator software, are covered by this project. Support and LHC-driven development of simulation toolkits such as Geant4 and Fluka and ancillary services and infrastructure surrounding them are covered by the project. Application of these toolkits in experiment-specific simulation software is not within scope, but the physics validation of the simulation toolkits and infrastructure using experiment simulations is covered by the project.

5.5 AnalysisHistogramming, ntuples, fitting, statistical analysis, data presentation tools. Should be well integrated with the experiment framework: access to experiment data objects; integrated with event display; allowing configurable interactive simulation and reconstruction. Concurrent availability of ‘event’ oriented capability and ‘distribution’ oriented capability. Also covers interactivity and visualization in the analysis environment. Command line environments and GUI toolkits for interactive and batch (script) access to the functionality of the system. Detector and event displays in 2D and 3D. And not least, providing all this in the distributed analysis environment of the LHC grid. Those aspects of this domain agreed as common work are pursued by the PI project.

5.6 Grid middleware interfaces

Encompasses both the grid services domain and aspects of the analysis domain. The components, services and tools by which physics applications software and physicists interface to Grid middleware and services. Utilization of Grid middleware and infrastructure to support job configuration, submission and control; distributed data management; Grid-enabled analysis; etc. Entirely amenable to common projects and a major LCG focus. The role of the applications area with respect to the Grid is expected to be in this area, where the physicist and physics application software meet the Grid. Any AA role in this area awaits the ARDA RTAG outcome.

– 11 –


5.7 Reconstruction, event model, calibration

These domains involve experiment-specific software and are not within LCG scope. These areas within the experiments are however expected to be customers of certain AA software deliverables. Examples include geometry services (used by reconstruction), object whiteboard (used by the event model) and the conditions database tool (used by calibration).

5.8 Trigger/DAQ

Not shown in the diagram and not currently foreseen to be addressed by the LCG. However, high level trigger applications and DAQ monitoring programs are expected to be implemented on top of experiment data processing frameworks, in order to exploit re-use of reconstruction algorithms and services directly in such applications. In this way Trigger/DAQ could be customers of AA software, and our requirements (see section 3.1.13) reflect this.

6 The Applications Area ProjectsThe applications area projects are outlined here. For further information see the project-level workplans and other materials.

6.1 Software Process and Infrastructure (SPI)

Responsible for the software development process, infrastructure, quality assurance, and training that supports the development, documentation, testing, usage, distribution and maintenance of AA software. Led by Alberto Aimar, CERN IT/API.

6.2 Persistency Framework (POOL)

Responsible for developing POOL, a hybrid data store for event and other physics data, consisting of an ‘object streaming’ component for the data itself and a relational component for associated metadata. Also responsible for developing the common conditions database software. Led by Dirk Duellmann, CERN IT/DB.

6.3 Core Libraries and Services (SEAL)

Responsible for foundation libraries, utilities, basic framework services, object dictionary, object whiteboard, system services, interactive services and grid enabled services. Also responsible for math and statistics libraries common among the experiments and used in analysis, reconstruction, simulation. Led by Pere Mato, CERN EP/SFT, LHCb.

6.4 Physicist Interface (PI)

This project covers the interfaces and tools by which physicists will directly use the software. It covers interactivity (the "physicist's desktop"), analysis tools, visualization, distributed analysis, and grid portals. Not all this activity is presently within scope. Led by Vincenzo Innocente, CERN EP/SFT, CMS.

– 12 –


6.5 Simulation

This project covers a range of simulation activities in which common work is taking place. The project hosts the CERN LHC contribution to Geant4 which is directed at meeting the LHC experiments’ Geant4 requirements. It is developing a ‘generic simulation framework’ allowing easy use of multiple simulation engines (Geant4, FLUKA, etc.), building upon the extensive work already done in this area by ALICE. It has the participation of FLUKA, for integration with the generic framework and simulation physics validation. It coordinates a major effort among all the experiments and the LCG to validate the physics of the simulation, and the simulation infrastructure. And it covers generator services, providing generator code management (librarian), event data catalog, and support and selective development of ancillary generator tools such as HepMC. The overall project is led by Torre Wenaus (BNL, CERN). The subprojects are led by Andrea Dell’Acqua (generic simulation framework), John Apostolakis (Geant4), Alfredo Ferrari (FLUKA integration), Fabiola Gianotti (physics validation), and Paolo Bartalini (generator services).

Sof

twar

e P

roce

ss &

Infra

stru

ctur

e (S

PI)

Core Libraries & Services (SEAL)

Persistency(POOL)

PhysicistsInterface

(PI)

MathLibraries…

LCG Applications Area

Other LCG Projects in other Areas

LHC

Exp

erim

ents

Oth

er E

xter

nal P

roje

cts

ArchitectsForum

Sof

twar

e P

roce

ss &

Infra

stru

ctur

e (S

PI)

Core Libraries & Services (SEAL)

Persistency(POOL)

PhysicistsInterface

(PI)

MathLibraries…

LCG Applications Area

Other LCG Projects in other Areas

LHC

Exp

erim

ents

Oth

er E

xter

nal P

roje

cts

ArchitectsForum

Figure 3 Organisational chart depicting the projects of the LCG Applications Area and their interfaces (arrows)

7 Project Management and Communication

7.1 Overall organization and management

Applications Area projects are each led by a Project Leader with overall responsibility for the management, design, development and execution of the work of the project. The Applications Area Leader has overall responsibility for the work of the Applications Area. Consultation and major decision making on management, planning and architectural oversight take place in the Architects Forum (AF).

– 13 –


Figure 4: Applications Area organization

7.2 Architects Forum

The AF is chaired by the applications area leader and has the experiment architects and AA project leaders as members, together with additional invitees at the discretion of the membership. The experiment computing coordinators are also invited to the AF. The AF provides for the formal participation of the experiments in the planning, decision making and architectural and technical direction of applications area activities. Architects represent the interests of their experiment and contribute their expertise.

The AF decides the difficult issues that cannot be resolved in open forums such as the applications area meeting. Most of the time, the AF will converge on a decision (to date, it has never failed to converge, and it rarely has difficulty doing so). If convergence is not possible, the applications area manager takes a decision. Such decisions can be accepted or challenged by the experiments. Challenged decisions go to the full PEB, then if necessary to the SC2 and we all abide happily by an SC2 decision.

The AF meets every 2 weeks or so. Meetings are minuted and the minutes are made public, usually within 1-2 weeks after the meeting.

7.3 Focus on Experiment Need

The applications area (and LCG in general) is structured and managed to ensure a focus on real experiment needs. This begins with the SC2/RTAG process to identify, define (need-driven requirements), initiate and monitor common project activities in a way guided by the experiments themselves. The Architects Forum involves the experiment architects in day to day project management and execution. Open information flow and decision making promotes a good connection between projects and experiment communities, as does direct participation by experiment developers in project activities. The AA relies on tight iterative feedback loops to gather user feedback from frequent software releases. Early deployment and evaluation of LCG software in experiment contexts is encouraged in order to get early feedback. And finally the success of AA projects is defined by the extent to which they achieve experiment adoption and production deployment.

– 14 –


7.4 Coherence and Communication

Work in the projects must be consistent and coherent with the architecture, infrastructure, processes, support and documentation functions that are agreed application area-wide. Establishing, supporting and monitoring this coherence is covered by the SPI project and its quality assurance activities. A high quality, current web presence is cultivated to make the web a major and effective communication and working medium.

Standard services, tools,infrastructure and interface glue,

code mgmt and support,QA, …

Project

Project

Project

Project

Web basedproject info,documentation,browsers, buildand release access,bug tracking, …

Coherentoverallarchitecture

Figure 5: Coherence across Application Area activities is emphasized

Redundancy and duplication is avoided in AA software development; focused development in an agreed direction is emphasized. An approach and design is chosen, in some cases after short explorations of the options, and set as the basis for work. Efforts that are divergent, redundant or under consideration for future adoption are isolated in contrib areas.

General applications area meetings, open to all, are held weekly (nominally; typically there are 3 meetings a month). These meetings involve a rich program of presentations and discussion on status and plans of the projects, technical design and implementation reports, experiment feedback from integration and validation work, non-project activities of interest, etc. Meetings typically have 25-35 attendees; sometimes up to 50.

All applications area meetings make provision for external participation (both in terms of employing conferencing facilities and in when meetings are held).

7.5 Project tracking

Principal mechanisms for project tracking are milestones for deliverable completion including project software releases, bug tracking via the Savannah system, QA metrics monitored with each

– 15 –


software release, regular reports assessing QA progress and status, and integration/validation milestones from the experiments.

7.6 Risk management

A risk analysis of the applications area was performed in spring 2003 and is periodically reviewed. It analyses project risks in terms of their likelihood, impact, overall risk, and management strategy. It is available at http://lcgapp.cern.ch/project/mgmt/apps-risks.html and, in a form integrated with the overall PEB risk analysis, from the PEB web pages.

7.7 Reporting and reviews

Oversight and review mechanisms are

A monthly PEB-wide report to SC2

Quarterly reports covering applications area activities, planning, and resources

Regular Applications Area status reports to the PEB

Applications Area status report to SC2 three times per year

Internal Applications Area technical reviews (the first one scheduled for Oct 2003)

Ongoing external (LHCC) reviews of the LCG project

8 WBS, Milestones and DeliverablesThe applications area work breakdown structure, milestones and deliverables for all aspects of the project are documented at http://atlassw1.phy.bnl.gov/Planning/lcgPlanning.html. The work breakdown structure maps directly onto the project breakdown of the AA. The schedule of milestones for the completion of deliverables is similarly organized. Milestones are organized at three levels:

Level 1: the highest level. A small, select number of very important milestones are at this level. These milestones are monitored at the LHCC level.

Level 2: the ‘official milestones’ level. Milestones at this level chart the progress of applications area activities in a comprehensive way. Each project has a small number of milestones per quarter at this level. These milestones are monitored at the LCG project level (PEB, SC2).

Level 3: internal milestones level. Milestones at this level are used for finer grained charting of progress for internal applications area purposes. These milestones are monitored at the AA level.

– 16 –

http://atlassw1.phy.bnl.gov/Planning/lcgPlanning.html

http://lcgapp.cern.ch/project/mgmt/apps-risks.html


Milestones include integration and validation milestones from the experiments to track the take-up of AA software in the experiments. Since this is the real measure of success, these are among our most important milestones for monitoring progress, even though they are not under AA control.

9 Manpower ResourcesThe figures below summarize the steadily evolving manpower situation as of August 2003. Personnel resource information is managed in a joint database used by the applications area, the EP/SFT group, and the overall LCG project. It covers all participation in the project, at CERN and outside CERN.

People FTEs

LCG applications area personnel 21 20.25

Working directly for apps area projects 14 13.85

ROOT 2 2

Grid integration work with experiments 3 2.8

Distributed analysis 2 1.6

Contributions from

IT 4 3.30

EP/SFT not experiment specific 21 16.10

EP/SFT experiment specific 7 4.35

Experiments outside EP/SFT 29 12.55

Total - direct project contributions 52 30.50

Total - indirect contributions (ROOT, ALICE VMC) 9 5.80

Total directly working on apps area projects 66 44.35

Overall total 82 56.55

Table 1: Manpower summary

Table 1 summarizes manpower resources and their sources. Most, but not all, LCG hires are working directly in applications area projects, but not all. A few do work supportive of the project in other environments, in particular in the ROOT project and in the grid integration efforts of the experiments. Applications area personnel are provided roughly equally by the LCG project,

– 17 –


CERN staff (without an experiment affiliation), and the experiments (including CERN staff with an experiment affiliation). The table indicates that, apart from experiment people, many of whom contribute only a modest fraction of their time to the LCG because of experiment commitments, applications area participants devote the great majority of their time to the project.

Figure 6: Applications Area FTEs by project

Figure 5 shows the distribution of applications area effort among the various activities. Note that simulation includes off-project ALICE effort developing their Virtual Monte Carlo (~2 FTEs), because this effort is leveraged directly by the simulation project. The ROOT level also includes the off-project efforts of the core ROOT team members in EP/SFT.

POOL SPI SEAL PI SimuLCG funded personnel 4.10 3.35 1.10 0.50 3.80IT personnel 2.30 1.00 0.00 0.00 0.00EP/SFT non-expt specific 0.00 1.20 2.00 0.90 9.20EP/SFT experiment specific 0.50 0.00 0.40 0.70 1.15Experiments outside EP/SFT 4.90 1.20 2.35 0.00 2.50Total 11.80 6.75 5.85 2.10 16.65Comparing the plan from the blueprint RTAG:Blueprint plan - present scope 11 7 7.5 4 15Blueprint plan - anticipated scope 11 7 9.5 12 17 i.e. adding grid grid, det desc

– 18 –

LCG AA : FTEs by project

POOL, 11.80, 21%

SPI, 6.75, 12%

SEAL, 5.85, 10%

PI, 2.10, 4%

Simu, 17.65, 31%

Mgmt, 1.80, 3%

ROOT, 6.20, 11%

Grid, 4.40, 8%


vis

Table 2: Effort distribution among projects (as of Oct 1 2003)

Table 2 is in two parts. The first part reproduces the information of the pie chart, and also gives the breakdown in terms of manpower sources for each project. The second (blue) part shows the estimates made in the blueprint RTAG in October 2002 for the effort levels required for each project. The blueprint plan number for the present scope of the active projects can be compared directly with the actual manpower levels above, showing that actual manpower levels are very close to the estimated requirements. The ‘anticipated scope’ number includes also activities anticipated in the blueprint but not yet within scope: grid services and grid-enabled analysis, visualization, and detector description. These activities add 12 FTEs to the estimate, most of it (~8 FTEs) in the grid area.

10 External participation and collaborationExternal (away from CERN) participation in AA projects is vital for their success. External participation is cultivated by measures to smooth the way for the always difficult task of participating in a project for which the center of gravity is far away. Applications area meetings make provision for external participation both in terms of employing conferencing facilities and in when meetings are held. Steady pressure to improve conferencing facilities is applied, and is sometimes successful. The inevitable inefficiencies and communication/control problems of remote collaboration are tolerated and worked around whenever possible. The POOL, SEAL, SPI and simulation projects all have important external contributions.

Many collaborations with other projects exist and are cultivated. In addition to the obvious (the experiments, ROOT), collaborations exist with all the other LCG activity areas (GDA: requirements from AA for LCG-1, Savannah; GTA: requirements on grid file access software, Savannah; Fabrics: POOL and SPI facilities, Savannah), the grid projects (RLS requirements and integration, EDG testbed participation, software packaging), and HEP community projects (Geant4, FLUKA, CLHEP).

11 Technical process

11.1 Process ModelThe software development process adopted by the project follows the general guidelines described in Thoughts on Software Process Error: Reference source not found. These guidelines are a mixture of the useful elements meeting the needs of the LCG software taken from two well known and established software processes: Extreme Programming and Rational Unified Process.

Summary of the good practices:

Planning

− Release planning creates the schedule − Make frequent releases with small functionality increments − Divide the project into iterations

Designing

– 19 –


− Design a Component Architecture− Design as simple as possible, but no simpler − No functionality is added early − Refactor whenever and wherever possible

Coding

− The customer is always available − Code must be written to agreed standards − Integrate often − Use collective code ownership − Leave optimization until last

Testing

− All code must have unit tests − All code must pass all unit tests before it can be released − When a bug is found, tests are created − Acceptance tests are run often and the score is published

11.2 Iterative developmentThe project works in tight, iterative development cycles: release early and release often. The diagram below tries to be explicit about what the planning, development and feedback cycle roughly should look like.

– 20 –


Figure 7 Development Cycle proposed for LCG Application Area software projects

11.3 Methods, Tools and TechniquesTable 1 contains the list of principal methods and tools used for the development of this software project. These tools are provided by the Software Process and Infrastructure project (SPI). In addition to the tools themselves, SPI is providing a number of guidelines and templates to support the software process: coding guidelines, repository structure guidelines, configuration management guidelines, testing guidelines, documentation templates, web page templates, project plan templates, etc.

11.4 InfrastructureThe infrastructure needed by the project in terms of hardware (code repository servers, build servers, test servers, web servers, software distribution area, etc.), operating system, network and software licences are provided by SPI project, or by CERN IT by arrangement with SPI.

– 21 –


The facilities required to conduct the project, which includes desktop workstations, local area networks, desks, office space and others, are provided by line management and CERN.

Design tools Together

Code Management CVS

Conf. Management SCRAM

C++ Compilers(ISO standard)

Deployment: gcc on Linux, VC++ on WindowsTesting: above and icc, ecc on Linux, CC on Solaris

Code Editors Linux: emacs

Debuggers Windows: DevStudioLinux: gvd (gdb)

Testing CppUnit, Oval, MCtest

Memory Leaks Valgrind

Mark-up Language XML, Xerces-C

Scripting Language Python

Documentation Manuals: WordSource Code: Doxygen, lxr

Bug Tracking Savannah

Table 3 List of Tools going to be used during development

11.5 Configuration ManagementConfiguration Management deals with defining configurations, building and labelling, and collecting versioned products into consistent sets and maintaining traceability between these versions. We will use CVS and SCRAM tools to fulfil all configuration management needs of the project. In the following we describe the principal elements of configuration management such as the product structure and identification of the constituent configuration items and some procedures. The example of the SEAL project is used; the other projects are analogous.

The product structure is depicted in Figure . The central configuration entity is a Package, which is the minimal entity that makes sense to release and associate a version (CVS tag). All the packages (delivery products) of the SEAL project are organized in a two level structure.

– 22 –

http://wwwinfo.cern.ch/asd/lhc++/STD/C++Standard.pdf


SEALProject

SubSystemB

SubSystemC

SubSystemA

PackageBa

PackageBb

File1File1File1File1File1File1 Versioned by

CVS

Versioned byCVS + SCRAM

Versioned bySCRAM

SEALProject

SubSystemB

SubSystemC

SubSystemA

PackageBa

PackageBb

File1File1File1File1File1File1 Versioned by

CVS

Versioned byCVS + SCRAM

Versioned bySCRAM

Figure 8 Product structure of the project with respect configuration management

Package versions are identified with a string of the format “PackageName_Major_Minor_Patch”. Where “Major”, “Minor”, “Patch” are version numbers with a specific semantics. The “Major” number changes for major releases of the software, which may require a major re-coding of customer software because the interface or the use model is changed. The “Minor” version number indicates new or changed functionality but the existing customer software is compatible at source level with the previous version. The last version number, “Patch”, indicates bug fixes, no new functionality being added.

The release management is done in various steps:

− The developer of a package can apply an internal tag as a declaration on the part of the developer that the package is tested and working.

− The package coordinator has the right to apply a release tag indicating that the package is tested and ready for release integration.

− The release manager will collect all the release tags for all the packages constituting the project, and will integrate them to produce a project release.

Change requests are handled using the bug tracking tool. Requests will always refer to a given version of the software. The changes in the software will be explained and identified by the change request identified in the ChangeLog file. This file exists in each package and therefore is tagged with the rest of the files constituting the package.

11.6 Verification and ValidationTests are written together with the code. Unit tests should validate expected functionality at the level of individual classes or small groups of collaborating classes. Regression test cases at levels higher than unit tests, tests involving package and component interactions and system level tests, should be written based on problems found in the pre-release QA process and user reported problems. System level tests should be based on requirements test cases that have failed in the past. These should be collected cumulatively to assemble a comprehensive regression test suite. Regression testing involves applying a standard test suite to a candidate release to find bugs

– 23 –


introduced since the last release (i.e. to spot regression), including bugs seemingly unrelated to the changes introduced. They test changes in integration behavior: they are typically large-grained, and test for unexpected consequences of integrating software components.

In addition to validation based on the different types of tests described, we perform:

Validation of coding standards compliance Dependency analysis Memory leak checking

All levels of testing (unit tests, component level tests, component interaction tests, system level tests) are to be run as part of automated nightly (and pre-release) builds.

11.7 Documentation and training

The SPI project develops and manages standard documentation tools, requirements and processes. Documentation of AA software is the responsibility of developers in the AA projects (manpower has not been found to provide dedicated technical writers to produce documentation). Documentation requirements are tied into release pre-requisite requirements to an increasing degree as one means of driving the provision of required documentation. SPI also develops and manages a training program covering AA and related software. The applications area supports a document registry at http://lcgapp.cern.ch/project/mgmt/doc.html.

Standard pieces of documentation for the project are:

Installation guide User manual with code examples (tested as part of release acceptance) Design documentation Auto-generated web-based documentation and code browsing Release notes Chapters in the LCG Workbook

11.8 Quality AssuranceQA elements are covered in various sections of this document. Here is quick summary:

Documentation. Documents for the project deliverables: design, installation guide, user manual, generated reference from code and release notes.

Standards and Guidelines. Set of guidelines and templates are provided by SPI. This includes the coding guidelines and the workbook.

Metrics. Metrics are being developed by SPI. Some examples are: number of implemented tests, test results, validation results, nightly build performance (failure rate), bug counts in released code (from bug reports), time to bug resolution, milestone performance, amount of invested effort, coding standards compliance, compile performance (warnings, etc.), number of dependencies, kLOC, commit count (frequency), date of last commit, developer count (cvs committers)

Review and Audit Plan. Quarterly project status reports. Design, requirements and code reviews every major release.

– 24 –

http://lcgapp.cern.ch/project/mgmt/doc.html


Evaluation and Tests. Writing tests is part of the software coding. Various kinds of tests are used: unit tests, regression tests, system level tests. In addition we check for memory leaks, package dependencies and conformance with coding rules.

Problem Resolution and Corrective Action. The bug tracking tool is used to report and track problems up to their resolution.

Tools, Techniques and Methods. A number of widely used tools are used all the way through the development process. Many are listed above.

Configuration Management. The combination of CVS and SCRAM provides us with the necessary tools to support the configuration management activity.

Training. Training is part of the deliverables of the project in form of documentation and tutorials. SPI organizes training programs. The relevant project is responsible for content.

Risk Management. See the risk assessment referenced above.

12 Applications Area Activities in LCG Phase 1

Phase 1 of the LCG Project runs through 2005. Applications area activities in this phase are:

Prepare the LHC computing environment

o Provide the common tools and infrastructure for the physics application software

o Grid interfaces to application software

o Integrate applications software into progressively more complex grid prototypes to validate the computing model

o Seek opportunities for re-use of project results

Participate in and support experiment data challenges

o Deploy and evaluate tools, infrastructure and grid interfaces in data challenge production and analysis environments

Participate in writing the LCG Technical Design Report

o Will describe the full LHC computing grid to be built in phase 2

13 ReferencesReferenced applications area documents can be found on the AA documents page at http://lcgapp.cern.ch/project/mgmt/doc.html

– 25 –

http://lcgapp.cern.ch/project/mgmt/doc.html


1. Thoughts on software process, T. Wenaus , CERN-LCGAPP-2002-03

2. POOL 2002 project plan, D. Duellmann et al , CERN-LCGAPP-2002-05

3. Architecture Blueprint RTAG report , CERN-LCGAPP-2002-09

4. SEAL project plan, P. Mato et al , CERN-LCGAPP-2003-01

5. PI project proposal to SC2, V. Innocente , CERN-LCGAPP-2003-03

6. POOL 2003 workplan, D. Duellmann , CERN-LCGAPP-2003-04

7. Simulation Project overview and plan, T. Wenaus et al , CERN-LCGAPP-2003-05

8. Applications area risk analysis, T. Wenaus , CERN-LCGAPP-2003-07

– 26 –

http://lcgapp.cern.ch/project/mgmt/apps-risks-v1.html

http://lcgapp.cern.ch/project/simu/SimuProjectOverview200303.pdf

http://pool.cern.ch/talks/ReleasePlan03.ppt

http://lcgapp.cern.ch/project/pi/talks/pireportsc2_jan03.ppt

http://seal.web.cern.ch/seal/documents/SEAL_Project_Plan_1_0.pdf

http://lcgapp.cern.ch/project/blueprint/BlueprintReport-final.doc

http://pool.cern.ch/talks/pool2002workplan.pdf

http://lcgapp.cern.ch/project/processV1.html