Software Overviewand
LCG Project Status & Plans
Torre WenausTorre Wenaus
BNL/CERNBNL/CERN
DOE/NSF Review of US LHC Software and ComputingDOE/NSF Review of US LHC Software and Computing
NSF, ArlingtonNSF, ArlingtonJune 20, 2002 June 20, 2002
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 2
Outline
Overview, organization and planningOverview, organization and planning
Comments on activity areasComments on activity areas
PersonnelPersonnel
ConclusionsConclusions
LCG Project Status and PlansLCG Project Status and Plans
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 3
U.S. ATLAS Software Project Overview
Control framework and architectureControl framework and architecture Chief Architect, principal development role. Software agreement in place
Databases and data management Database Leader, primary ATLAS expertise on ROOT/relational baseline
Software support for development and analysisSoftware support for development and analysis Software librarian, quality control, software development tools, training… Automated build/testing system adopted by and (partly) transferred to
Int’l ATLAS
Subsystem software roles complementing hardware responsibilitiesSubsystem software roles complementing hardware responsibilities Muon system software coordinator
Scope commensurate with U.S. in ATLAS: ~20% of overall effortScope commensurate with U.S. in ATLAS: ~20% of overall effort
Commensurate representation on steering group
Strong role and participation in LCG common effortStrong role and participation in LCG common effort
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 4
U.S. ATLAS Software Organization
William WillisProject Manager
John HuthAssociate Project Manager,
Computing and Physics WBS 2
James ShankDeputy
External Advisory Group
Ian HinchliffeManager, Physics
WBS 2.1
Torre WenausManager, Software
WBS 2.2
Bruce GibbardManager, Facilities
WBS 2.3
C. TullControl/Framework
2.2.1.1,2.2.1.2
David MalonData Management
2.2.1.3
S. RajagopalanEvent Model
2.2.1.4
J. ShankDetector Specific
2.2.2, 2.2.2.1
F. MerrittTraining
2.2.5L. Vacavant
Pixel/SCT2.2.2.2
F. LuehringTRT
2.2.2.3
S. RajagopalanLiquid Argon Calorimeter
2.2.2.4
T. LeCompteTilecal2.2.2.5
S. GoldfarbMuons2.2.2.6
S. GonzalezTrigger/DAQ
2.2.2.7
Subsystems
Core Software
R. BakerTier 1 Facility
R. GardnerDistributed ITInfrastructure
Facilities
TBNCollaborative
Tools2.2.3
R. BakerDeputy
T. WenausSoftware Support
Coordinator2.2.4
A. UndrusSoftware Librarian
2.2.4.1
Computing CoordinationBoard
Physics Manager, IB Convener, co-chairs
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 5
U.S. ATLAS - ATLAS Coordination
William WillisU.S. ATLAS Project
Manager
Peter JenniATLAS
Spokesperson
John HuthAssociate PM
NormanMcCubbinSoftware
Coordiinator
I. HinchliffePhysics SM
F. GianottiPhysics
Coordinator
T. WenausSoftware SM
B. GibbardFacilities SM
C. TullFramework
D. MalonDatabase
SubsystemSoftware
T. WenausPlanning Officer
D. QuarrieChief Archectect
D.MalonDatabase
A. PutzerNCB
L. PeriniATLAS GRID
R. GardnerDistributed Computing
SubsystemSoftware
US roles in Int’l ATLAS software:US roles in Int’l ATLAS software:
D. Quarrie (LBNL), Chief ArchitectD. Quarrie (LBNL), Chief Architect
D. Malon (ANL), Database CoordinatorD. Malon (ANL), Database Coordinator
P. Nevski (BNL), Geant3 Simulation P. Nevski (BNL), Geant3 Simulation CoordinatorCoordinator
H. Ma (BNL), Raw Data CoordinatorH. Ma (BNL), Raw Data Coordinator
C. Tull (LBNL), Eurogrid WP8 LiaisonC. Tull (LBNL), Eurogrid WP8 Liaison
T. Wenaus (BNL), Planning OfficerT. Wenaus (BNL), Planning Officer
US International
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 6
ATLAS Subsystem/Task Matrix
Offline Offline
CoordinatorCoordinator
ReconstructionReconstruction SimulationSimulation DatabaseDatabase
ChairChair N. McCubbinN. McCubbin D. RousseauD. Rousseau A. Dell’AcquaA. Dell’Acqua D. MalonD. Malon
Inner DetectorInner Detector D. BarberisD. Barberis D. RousseauD. Rousseau F. LuehringF. Luehring S. Bentvelsen /S. Bentvelsen /
D. CalvetD. Calvet
Liquid ArgonLiquid Argon J. CollotJ. Collot S. RajagopalanS. Rajagopalan M. LeltchoukM. Leltchouk H. MaH. Ma
Tile CalorimeterTile Calorimeter A. SolodkovA. Solodkov F. MerrittF. Merritt V.TsulayaV.Tsulaya T. LeCompteT. LeCompte
MuonMuon J.ShankJ.Shank J.F. LaporteJ.F. Laporte A. RimoldiA. Rimoldi S. GoldfarbS. Goldfarb
LVL 2 Trigger/ LVL 2 Trigger/
Trigger DAQTrigger DAQ
S. GeorgeS. George S. TapproggeS. Tapprogge M. WeilersM. Weilers A. Amorim /A. Amorim /
F. TouchardF. Touchard
Event FilterEvent Filter V. VercesiV. Vercesi F. TouchardF. Touchard
Computing Steering Group members/attendees: 4 of 19 from USComputing Steering Group members/attendees: 4 of 19 from US
(Malon, Quarrie, Shank, Wenaus)(Malon, Quarrie, Shank, Wenaus)
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 7
Project Planning Status
U.S./Int’l ATLAS WBS/PBS and schedule fully unifiedU.S./Int’l ATLAS WBS/PBS and schedule fully unified ‘Projected’ out of common sources (XProject); mostly the same
US/Int’l software planning covered by the same person US/Int’l software planning covered by the same person Synergies outweigh the added burden of the ATLAS Planning Officer role
No ‘coordination layer’ between US and Int’l ATLAS planning: direct ‘official’ interaction with Int’l ATLAS computing managers. Much more efficent
No more ‘out of the loop’ problems on planning (CSG attendance)
True because of how the ATLAS Planning Officer role is currently scoped As pointed out by an informal ATLAS computing review in March, ATLAS would
benefit from a full FTE devoted to the Planning Officer function I have a standing offer to the Computing Coordinator: to willingly step aside
if/when a capable person with more time is found Until then, I scope the job to what I have time for and what is highest priority ATLAS management sought to impose a different planning regime on computing
(PPT) which would have destroyed US/Int’l planning commonality; we reached an accommodation which will make my time more rather than less effective, so I remained in the job
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 8
ATLAS Computing Planning
US led a comprehensive review and update of ATLAS computing schedule in US led a comprehensive review and update of ATLAS computing schedule in Jan-MarJan-Mar Milestone count increased by 50% to 600; many others updated Milestones and planning coordinated around DC schedule Reasonably comprehensive and detailed through 2002
Things are better, but still not greatThings are better, but still not great Schedule still lacks detail beyond end 2002 Data Challenge schedules and objectives unstable Weak decision making (still a major problem) translates to weak planning
Strong recommendation of the March review to fix this; no observable change
Use of the new reporting tool PPT (standard in ATLAS construction project) Use of the new reporting tool PPT (standard in ATLAS construction project) may help improve overall planningmay help improve overall planning Systematic, regular reporting coerced by automated nagging Being introduced so as to integrate with and complement XProject-based
planning materials. XProject adapted; waiting on PPT adaptations.
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 9
Short ATLAS planning horizon
Schedule items
0
50
100
150
200
250
1999
2000
2001
2002
2003
2004
2005
As of 3/02
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 10
Summary Software Milestones
0 1 2 3 4 5 6 1 Tbyte database prototype (Done)Release of Athena pre-alpha version (Done)Athena alpha release (Done)Geant3 digi data available (Done)Athena beta release (Done)Athena accepted (ARC concluded) (Done)Athena Lund release (Done)Event store architecture design document (Done)DC0 production releaseDecide on database product (Done)DC0 Completed - continuity testFull validation of Geant4 physics DelayDC1 CompletedComputing TDR Finished DelayDC2 Completed (followed by annual DCs)Physics readiness report completed Full software chain in real environmentFull DB infrastructure available
Green: Done Gray: Original date Blue: Current date
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 11
Data Challenge 1
DC1 phase 1 (simu production for HLT TDR) is ready to startDC1 phase 1 (simu production for HLT TDR) is ready to start
Software ready and tested, much developed in the USSoftware ready and tested, much developed in the US Baseline core software, VDC, Magda, production scripts
2M events generated and available for filtering and simulation2M events generated and available for filtering and simulation US is providing the first 50K of filtered, fully simulated events for QA
Results will be reviewed by QA group before the green light is given for full Results will be reviewed by QA group before the green light is given for full scale production in about a weekscale production in about a week
During the summer we expect to process 500k fully simulated events at the During the summer we expect to process 500k fully simulated events at the BNL Tier 1BNL Tier 1
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 12
Brief Comments on Activity Areas
Control Framework and ArchitectureControl Framework and Architecture
DatabaseDatabase
Software Support and Quality ControlSoftware Support and Quality Control
Grid SoftwareGrid Software
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 13
Control Framework and Architecture
US leadership and principal development rolesUS leadership and principal development roles
David Quarrie recently offered and accepted a further 2 year term as Chief ArchitectDavid Quarrie recently offered and accepted a further 2 year term as Chief Architect
Athena role in ATLAS appears well consolidatedAthena role in ATLAS appears well consolidated Basis of post-simulation Data Challenge processing Actively used by end users, with feedback commensurate with Athena’s stage of development LHCb collaboration working well FADS/Goofy (simulation framework) issue resolved satisfactorily Regular, well attended tutorials
Other areas still have to prove themselvesOther areas still have to prove themselves ATLAS data definition language
Being deployed now in a form capable of describing the ATLAS event model
Interactive scripting in Athena Strongly impacted by funding cutbacks New scripting service emerging now
Tremendous expansion in ATLAS attention to event modelTremendous expansion in ATLAS attention to event model About time! A very positive development Broad, (US) coordinated effort across the subsystems to develop a coherent ATLAS event model Built around the US-developed StoreGate infrastructure
Core infrastructure effort receiving useful feedback from the expanded activity
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 14
Database
US leadership and key technical expertiseUS leadership and key technical expertise
The ROOT/relational hybrid store in which the US has unique expertise in ATLAS is
now the baseline, and is in active development
The early US effort in ROOT and relational approaches (in the face of ‘dilution of
effort’ criticisms) was a good investment for the long term as well as the short term
Event data storage and management now fully aligned with the LCG effortEvent data storage and management now fully aligned with the LCG effort
~1 FTE each at ANL and BNL identified to participate and now becoming active
Work packages and US roles now being laid out
ATLAS and US ATLAS have to be proactive and assertive in the common project for the ATLAS and US ATLAS have to be proactive and assertive in the common project for the
interests of interests of ATLASATLAS (I don’t have my LCG hat on here!), and I am pushing this hard (I don’t have my LCG hat on here!), and I am pushing this hard
Delivering a data management infrastructure that meets the needs of (US) ATLAS
and effectively uses our expertise demand it
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 15
Software Support, Quality Control
New releases are available in the US ~1 day after CERN (with some exceptions New releases are available in the US ~1 day after CERN (with some exceptions when problems arise!)when problems arise!) Provided in AFS for use throughout the US
Librarian receives help requests and queries from ~25 people in the USLibrarian receives help requests and queries from ~25 people in the US US-developed nightly build facility used throughout ATLASUS-developed nightly build facility used throughout ATLAS
Central tool in the day to day work of developers and the release process Recently expanded as framework for progressively integrating more quality
control and testing Testing at component, package and application level Code checking to be integrated
CERN support functions being transferred to new ATLAS librarian Plan to resume BNL-based nightlies
Much more stable build environment than CERN at the moment Hope to use timely, robust nightlies to attract more usage to the Tier 1
pacmanpacman (Boston U) for remote software installation (Boston U) for remote software installation Adopted by grid projects for VDT, and a central tool in US grid testbed work
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 16
Grid Software
Software development within the ATLAS complements of the grid Software development within the ATLAS complements of the grid projects is being managed as an integral part of the software effortprojects is being managed as an integral part of the software effort Objective is to integrate grid software activities tightly into ongoing
core software program, for maximal relevance and return Grid project programs consistent with this have been developed
And has been successfulAnd has been successful e.g. Distributed data manager tool (Magda) we developed was
adopted ATLAS-wide for data management in the DCs
Grid goals, schedules integrated with ATLAS (particularly DC) programGrid goals, schedules integrated with ATLAS (particularly DC) program
However However we do suffer some program distortionwe do suffer some program distortion e.g. we have to limit effort on providing ATLAS with event storage
capability in order to do work on longer-range, higher-level distributed data management services
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 17
Effort Level Changes
ANL/Chicago – loss of .5 FTE in DBANL/Chicago – loss of .5 FTE in DB Ed Frank departure; no resources to replace
BNL – cancelled 1 FTE new hire in data managementBNL – cancelled 1 FTE new hire in data management Insufficient funding in the project and the base program to sustain the bare-
bones plan Results in transfer of DB effort to grid (PPDG) effort – because the latter
pays the bills, even if it distorts our program towards lesser priorities As funding looks now, >50% of the FY03 BNL sw development effort will be
on grid!! LBNL – stable FTE count in architecture/frameworkLBNL – stable FTE count in architecture/framework
One expensive/experienced person replaced by very good postdoc It is the DB effort that is most hard-hit, but ameliorated by common projectIt is the DB effort that is most hard-hit, but ameliorated by common project
Because the work is now in the context of a broad common project, US can still sustain our major role in ATLAS DB
This is a real, material example of common effort translating into savings (even if we wouldn’t have chosen to structure the savings this way!)
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 18
Personnel Priorities for FY02, FY03
Priorities are the same as presented last time, and this is how we are Priorities are the same as presented last time, and this is how we are doing…doing… Sustain LBNL (4.5FTE) and ANL (3FTE) support
This we are doing so far.
Add FY02, FY03 1FTE increments at BNL to reach 3FTEs Failed. FY02 hire cancelled.
Restore the .5FTE lost at UC to ANL No resources
Establish sustained presence at CERN. No resources
As stated last time… we rely on labs to continue base program and As stated last time… we rely on labs to continue base program and other lab support to sustain existing complement of developersother lab support to sustain existing complement of developers And they are either failing or predicting failure soon. Lab base
programs are being hammered as well.
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 19
SW Funding Profile Comparisons
0.00
1.00
2.00
3.00
4.00
5.00
6.00
FY01 FY02 FY03 FY04 FY05 FY06
M$
2000 agency guideline
January 2000 PMP
11/2001 guideline
Compromise profile
requested in 2000
Current bare bones
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 20
Conclusions
US has consolidated the leading roles in our targeted core software areasUS has consolidated the leading roles in our targeted core software areas
Architecture/framework effort level being sustained so farArchitecture/framework effort level being sustained so far
And is delivering the baseline core software of ATLAS
Database effort reduced but so far preserving our key technical expertiseDatabase effort reduced but so far preserving our key technical expertise
Leveraging that expertise for a strong role in common project
Any further reduction will cut into our expertise base and seriously weaken the US
ATLAS role and influence in LHC database work
US has made major contributions to an effective software development and release US has made major contributions to an effective software development and release
infrastructure in ATLASinfrastructure in ATLAS
Plan to give renewed emphasis to leveraging and expanding this work to make the
US development and production environment as effective as possible
Weakening support from the project and base programs while the emphasis on grids Weakening support from the project and base programs while the emphasis on grids
grows is beginning to distort our program in a troubling waygrows is beginning to distort our program in a troubling way
LCG Project Status & Plans(with an emphasis on applications software)
Torre WenausTorre Wenaus
BNL/CERNBNL/CERN
DOE/NSF Review of US LHC Software and ComputingDOE/NSF Review of US LHC Software and Computing
NSF, ArlingtonNSF, ArlingtonJune 20, 2002 June 20, 2002
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 22
The LHC Computing Grid (LCG) Project
Developed in light of the LHC Computing Review conclusionsDeveloped in light of the LHC Computing Review conclusions Approved (3 years) by CERN Council, September 2001 Injecting substantial new facilities and personnel resources
Activity areas:Activity areas: Common software for physics applications
Tools, frameworks, analysis environment
Computing for the LHC Computing facilities (fabrics) Grid middleware Grid deployment
Global analysis environment
Foster collaboration, coherence of LHC computing centers Foster collaboration, coherence of LHC computing centers
Goal – Prepare and deploy the LHC computing environment
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 23
OtherLabs
The LHC Computing Grid Project
LHCC
Project Overview Board
TechnicalStudy
Groups
Reports
Reviews
ResourcesBoard
Resource Issues
ComputingGrid
Projects
Project execution teams
HEPGrid
Projects
Project Manager
ProjectExecution
Board
Requirements,Monitoring
Software andComputingCommittee
LCG Project Structure
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 24
Current Status
High level workplan just written (linked from this review’s web page)High level workplan just written (linked from this review’s web page) Two main threads to the work:Two main threads to the work:
Testbed development (Fabrics, Grid Technology and Grid Deployment areas)
A combination of primarily in-house CERN facilities work and working with external centers and the grid projects
Developing a first distributed testbed for data challenges by mid 2003 Applications software (Applications area)
The most active and advanced part of the project Currently three active projects in applications:
Software process and infrastructure Mathematical libraries Persistency
Pressuring the SC2 to open additional project areas ASAP – not enough current scope to put available people to work effectively (new LCG and existing IT people)
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 25
LHC Manpower needs for Core Software
20002000
Have (Have (missmiss))
20012001 20022002 20032003 20042004 20052005
ALICEALICE 12(12(55)) 17.517.5 16.516.5 1717 17.517.5 16.516.5
ATLASATLAS 23(23(88)) 3636 3535 3030 2828 2929
CMSCMS 15(15(1010)) 2727 3131 3333 3333 3333
LHCbLHCb 14(14(55)) 2525 2424 2323 2222 2121
TotalTotal 64(64(2828)) 105.5105.5 106.5106.5 103103 100.5100.5 99.599.5
Only computing professionals counted
From LHC Computing Review (FTEs)
LCG common project activity in applications software: Expected number of new LCG-funded people in applications is 23 Number hired or identified to date: 9 experienced, 3 very junior Number working today: 8 LCG (3 in the last 2 weeks), plus ~3 existing IT, plus expts
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 26
Applications Area Scope
Application Software InfrastructureApplication Software Infrastructure Scientific libraries, foundation libraries, software development
tools and infrastructure, distribution infrastructure Physics Data ManagementPhysics Data Management
Storing and managing physics data: events, calibrations, analysis objects
Common FrameworksCommon Frameworks Common frameworks and toolkits in simulation, reconstruction
and analysis (e.g. ROOT, Geant4) Support for Physics ApplicationsSupport for Physics Applications
Grid ‘portals’ and interfaces to provide distributed functionality to physics applications
Integration of physics applications with common software
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 27
Typical LHC Experiment Software Architecture… a ‘Grid-Enabled’ View
Standard LibrariesH
igh level tr
iggers
One main framework, e.g. ROOT
Various specialized frameworks: persistency (I/O), visualization, interactivity, simulation, etc.
Grid integration
Widely used utility libraries (STL, CLHEP); distributed services
Applications built on top of frameworks
Analy
sis
Sim
ula
tion
Reco
nst
ruct
ion
FrameworksToolkits
FrameworksToolkits
= Common solutions being pursued or foreseen
GridServices
GridInterfaces
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 28
Current major project: Persistency
First major common software project begun in April: First major common software project begun in April: Persistency Framework (POOL)Persistency Framework (POOL) To manage locating, reading and writing physics data
Moving data around will be handled by the grid, as will the distributed cataloging Will support either event data or non-event (e.g. conditions) data
Selected approach: a Selected approach: a hybrid storehybrid store Data objects stored by writing them to ROOT files
The bulk of the data Metadata describing the files and enabling lookup are stored in relational
databases Small in volume, but with stringent access time and search requirements, well suited to
relational databases Successful approach in current experiments, e.g. STAR (RHIC) and CDF (Tevatron)Successful approach in current experiments, e.g. STAR (RHIC) and CDF (Tevatron)
LHC implementation needs to scale to much greater data volumes, provide distributed functionality, and serve the physics data object models of four different experiments
Early prototype is scheduled for September 02 (likely to be a bit late!)Early prototype is scheduled for September 02 (likely to be a bit late!) Prototype to serve a scale of 50TB, O(100k) files, O(10) sites Early milestone driven by CMS, but would have been invented anyway: we need
to move development work from abstract discussions to iterating on written software
Commitments from all four experiments to development participationCommitments from all four experiments to development participation ~3 FTEs each from ATLAS and CMS; in ATLAS, all the participation (~2 FTEs) so
far is from the US (ANL and BNL); another ~1-2 FTE from LHCb+ALICE
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 29
Hybrid Data Store – Schematic View
EventEvent
TrackListTrackList
TrackerTracker CalorimeterCalorimeter
TrackTrackTrackTrack
TrackTrackTrackTrackTrackTrack
HitListHitList
HitHitHitHitHitHitHitHitHitHit
Experiment event model
ObjectDictionary
Service
PersistencyManager
Experimentspecific object model
descriptions
Storage:
Pass object(s)
Get ID(s)
Retrieval:
Pass ID(s)
Get object(s)
Locator ServiceDistributed
Replica Manager(Grid)Locate Files
File(s)
Storage Manager
ID – FileDB
fopen etc.
ObjectStreaming
Service
File info
File records
Data objects
Objectinfo
Object descriptions
DatasetLocator
Process dataset DatasetDB
File(s)
Name
Data File
Human Interaction
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 30
Coming Applications RTAGs
After a very long dry spell (since Jan), the SC2 has initiated the first After a very long dry spell (since Jan), the SC2 has initiated the first stage of setting up additional projects – establishing requirements stage of setting up additional projects – establishing requirements and technical advisory groups (RTAGs) with 2-3 month durationsand technical advisory groups (RTAGs) with 2-3 month durations
Detector geometry and materials description To address high degree of redundant work in this area (in the case
of ATLAS, even within the same experiment)
Applications architectural blueprint High level architecture for LCG software
Pending RTAGs in applicationsPending RTAGs in applications Physics generators (launched yesterday) A fourth attempt at a simulation RTAG in the works (politics!) Analysis tools (will follow the blueprint RTAG)
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 31
Major LCG Milestones
June 2003 – LCG global grid service available (24x7 at 10 centers)June 2003 – LCG global grid service available (24x7 at 10 centers)
June 2003 – Hybrid event store releaseJune 2003 – Hybrid event store release
Nov 2003 – Fully operational LCG-1 service and distributed production Nov 2003 – Fully operational LCG-1 service and distributed production
environment (capacity, throughput, availability sustained for 30 days)environment (capacity, throughput, availability sustained for 30 days)
May 2004 – Distributed end-user interactive analysis from Tier 3May 2004 – Distributed end-user interactive analysis from Tier 3
Dec 2004 – Fully operational LCG-3 service (all essential functionality Dec 2004 – Fully operational LCG-3 service (all essential functionality
required for the initial LHC production service)required for the initial LHC production service)
Mar 2005 – Full function release of persistency frameworkMar 2005 – Full function release of persistency framework
Jun 2005 – Completion of computing service TDRJun 2005 – Completion of computing service TDR
June 20, 2002June 20, 2002Torre Wenaus, BNL/CERN DOE/NSF ReviewTorre Wenaus, BNL/CERN DOE/NSF Review 32
LCG Assessment
In the computing fabrics (facilities) area, LCG is now the context (and In the computing fabrics (facilities) area, LCG is now the context (and funding/effort source) for CERN Tier0/1 developmentfunding/effort source) for CERN Tier0/1 development But countries have been slow to follow commitments with currency
In the grid middleware area, the project is still trying to sort out its role In the grid middleware area, the project is still trying to sort out its role as ‘not just another grid project’; not yet clear how it will achieve the as ‘not just another grid project’; not yet clear how it will achieve the principal mission of principal mission of ensuringensuring the needed middleware is available the needed middleware is available
In the deployment area (integrating the above two), testbed/DC plans In the deployment area (integrating the above two), testbed/DC plans are taking shape well with an aggressive mid 03 production deploymentare taking shape well with an aggressive mid 03 production deployment
In the applications area, the persistency project seems on track, but In the applications area, the persistency project seems on track, but politics etc. have delayed the initiation of new projectspolitics etc. have delayed the initiation of new projects The experiments do seem solidly committed to common projects
This will change rapidly if LCG hasn’t delivered in ~1 year CMS is most proactive in integrating the LCG in their plans; ATLAS
less so to date (this extends to the US program). I will continue to push (with my ‘ATLAS’ hat on!) to change this