chep04 interlaken, ch 26 september – 1 october 2004 adele rimoldi university of pavia & infn,...
TRANSCRIPT
CHEP04 Interlaken, CH 26 September – 1 October 2004
adele rimoldiUniversity of Pavia & INFN, Italy
The full detector simulation The full detector simulation for the ATLAS experiment: for the ATLAS experiment:
status and outlookstatus and outlook
A.Rimoldi, J. Boudreau, D. Costanzo A. Dell’Acqua, M. Gallas, A. Nairz, V.Tsulaia
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 2
outlook
Simulation data flow
The ATLAS Simulation Project G4ATLAS the Geant4-based simulation for ATLAS The ATLAS Simulation from the Data Challenge and Physics Validation
perspective
The ATLAS Detector in GEANT4 and its Subdetectors the Inner Detector Simulation The ATLAS Calorimeters simulation The Muon System Simulation
The ATLAS Testbeam
The Detector Digitization
The Simulation Validation Preproduction and DC2 Memory usage @runtime Timing for different event samples
The Data Challenges in ATLAS
The Physics with DC2 and CTB as a feedback for Simulation
Conclusions
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 3
ATLAS Simulation data flow
Generator McTruth(Gen)HepMC
ROD EmulationAlgorithm
L1 Digitization
Particle Filter Simulation
PileUp
McTruth(Sim)HitsROD Input
Digits
McTruth(PileUp)
DigitizationRawDataObjects
ByteStreamConversionSvc
MergedHits
L1Digits
L2Result
EFResult
L1 Emulation(inc. L1 ROD)
L1Result
ROD Emulation
(passthru)
L2 SelectionAlgorithm
EF SelectionAlgorithm
ByteStream
Use
s R
aw
Da
taO
bje
cts
ATLAS
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 4
The ATLAS Simulation Project
Present Status
GEANT3-based simulation was operational for the last 10 years, now discontinued (mid 2004)
• It provided a simulation infrastructure used for Data Challenges(DC0,DC1), heavy ions, early testbeam and design optimization, experiment commissioning
GEANT4-based simulation developed in a full OO environment since 2000• Very detailed and up-to-date in all the previous items, in most cases more
accurate and performant used for DC2 the 2004 combined testbeam, the last before the first data-taking Heavy ions production Ready for early commissioning studies
A strategy for passage from GEANT3 to GEANT4 was successfully launched and followed in ATLAS end of last Year
• ATL-SOFT-2003-013 . Strategy for the transition from Geant3 to Geant4 in ATLAS. by:Barberis, D.; Polesello, G.; Rimoldi, A ; Geneva : CERN, 13 Nov 2003
Now the Geant4-based simulation is the main simulation engine in ATLAS
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 5
G4ATLAS: the GEANT4-based simulation for ATLAS
Features Completely written in C++ Extensively usage of dynamical loading and action on demand Completely embedded in the ATLAS ATHENA framework Success story in terms of
• open software • Multi programmers facility• Results: Performance and robustness optimal after a short ramp-up
Started as standalone exercise, then embedded in the ATLAS framework, now fully operational for experiment and testbeam purposes with the same software
POOL utilized for the I/O
Functionality Most functionality is there. Interactivity is provided Python scripting replacing the old macro-files structure
Developments backward compatibility always provided Not the end of the story: improvemnet foreseen in many fields (background treatment, visualization,
more interactivity, documentation for end users, etc.)
Validation process Multi-step process through Data Challenges -> see next slide Readiness for the next Physics Workshop in 2005 with new features and upgrades
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 6
The Atlas Simulation in GEANT4 from..
the Data Challenge viewpoint DC0 (since end 2001 tests of event productions Geant3 & Geant4) DC1(Phase I ->II)
Geant3 based• Validation samples (single particle, Et scans,Higgs) 740K ev• Single-particle production 30 million ev• Minimum-bias production 1.5 million ev• QCD di-jet production 5.2 million ev• Physics events requested by HLT groups 4.4 million ev• Pile-up • Data samples requests from end-user community
DC2 and following GEANT4 based
• large scale physics analysis, tests on computing model, test calibrations and alignment procedures
12 millions fully simulated events• And a grand total of 1 job crash !And a grand total of 1 job crash !
Distributed production
• 1M Z->ee events in 10K jobs and no failures (@NorduGrid)
the Physics Validation viewpoint Used since 2001 mainly for testbeam simulations and simple setups from 2004 for physics events analyses (Z->ee, , single particle ..)
• Growing users community (the only way of shaking down bugs…)• Comparisons with real data in the testbeams, different layouts
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 7
The ATLAS detector in Geant4
Four main subdetectors Inner detector - momentum
measurement of charged particles, electron ID
• high precision silicon trackers• straw tracker with TR capability
Calorimeters - measurement of particle energies
• EM LAr calorimeters (barrel & endcap)• Hadron Lar calorimeters (endcap)• Scintillating Tile hadron calorimeter
Muon spectrometer - muon identification and measurement
• High precision Drift Tubes for tracking, RPCs and TGCs for triggering
Magnet system - bending of charged particles for momentum measurements
5.2M volumes objects(G3 27M) 110K volume types (G3 23K)
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 8
The Inner Detector
Eta
Red: InitialBlack: Final
Eta
Red: InitialBlack: Final
GeometryThree subdetectors components
• Pixel• SCT• TRT
Final / initial layout available, preliminary validation on hits content done
Still to do• Allow global movements of the Pixel• Increase the level of details
Detector responseTuned on test beam resultsHome-grown TR model
DigitizationAdapt hit reading for pile-upIntroduce the concept of time in the digitizationNoise for TRT
Digitization packages used by the Combined Test Beam Tuning with data to feed the Atlas Simulation
Eta
Pixel
SCT
Pixel+SCT
Red: InitialBlack: Final
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 9
The ATLAS Calorimeters Simulation
4 subsystems Electromagnetic barrel (EMB) Hadronic end-cap (HEC) Forward calorimeter (FCal) Hadronic calorimeter (Tile)
Heavy tests/investigations for optimizing the physics in Geant4 the geometry for reducing memory consumption and cpu
time• Parameterization studies ongoing
Main software infrastructure issues: the detector description
no more hand-coded numbers, full GeoModel version available (library of geometrical primitives for describing detector geometries)
-> V.Tsulaia talk #279, tomorrow the versioning of the database constants
• LAr has already been switched to Oracle the inclusion of calibration hits
• do a careful accounting of where the energy goes in ATLAS
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 10
The Muon System Simulation
System composed by Four main subdetectors
• 2 precision chambers tracking detectors MDT, CSC
• 2 trigger chambers detectors RPC,TGC
Interleaved with the toroids structure • Feets and rails
The outermost detector of ATLAS• Services and cables passing through
Pileup & cavern background• Functionality for handling pileup in place
digitization time window for all technologies Current DC2 production: no cavern background yet, only minimum-bias Full background as in DC1 expected soon
CombinedTestBeam (Muon side)• Robustness of sim-digi chain demonstrated through tons of events produced
Comparisons with real data for all technologies• Now the fine-tuning stage
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 11
The ATLAS Testbeam
The 2004 CTB case All the components for a
complete ATLAS sector are being tested together on a beam line in Summer 2004 (Combined TestBeam Setup)
• @different beam energies• Magnetic field (2)• Ancillary detectors• Customized beam profile at
generation• Deep comparisons with real
data sample in each subdetector prototype
• Fine tuning• Full chain Simulation-
Digitization-Reconstruction done!.
• In production mode.
• Same software as for the full experiment
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 12
The Detector Digitization
The digitization procedure is disentangled from the simulation process proper and it can be started from pregenerated hits or in a full chain
Fully functional and "leak free" since months Each subdetector loaded on demand Digitization of GEANT4 hits done Hits I/O with Pool for all subdetectors
Expect extensive Validation work from the CTB by comparing with data All assumptions (resolutions, smearing, …) to be revisited
Pile up
Pile-up stays in ~1Gbyte of memory, takes 5mn/event (1GHz) and needs ~10Mbyte/event of disk Pythia used for the min-bias pile up events
• A set of 500K min bias events with the atlas tunings is used To do
• Optimize I/O use vs. memory
• Occupancy studies need to be done
• MC truth navigation, not supported for pile-up
• Lot of work still needed. Both for Validation and new Development
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 13
The Simulation Validation Preproduction and DC2
preliminary tests started at end 2003
comparisons with Geant3 using common event samples and about the same geometry.
Hits and digits for all the ATLAS subdetectors generated
jobs run in parallel using the LSF batch facility and Castor facility (at CERN and outside)we measured at different event/run phases the local peformance and memory usageGenerated samples
Single particle vs. E SUSY events H->4 leptons, Z->2leptons(e, mu,tau) di-jet minimum bias
Initial failure rate of ~10% for single particle jobs (30% in physics events), corrected patiently (geometry problems, G4 physics problems…)
Final failure rate is approximately 0% apart from AFS or Castor problems. All jobs go straightforward to the end
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 14
Watching the application @runtime
Configuration @ Process Size @runtime (MB)
G4 initialisation (application alone) 50 MB
building ATLAS envelopes 0.9 MB
building ID Geometry + SensitiveDetector 2.2 MB + 0.3 MB
building LArCalo 83.7 MB
building Tile Geometry + SensitiveDetector 0.6 MB + 0.3 MB
building Muon Geometry + SensitiveDetector 8.0 MB + 0.2 MB
geometry optimisation 30 MB
magnetic field 12 MB
loading of full (default) Physics Lists 49 MB
External Dependencies reading events from ROOT files 27 MB
declaring POOL in jobOptions (hit persistification)* ~40 MB
LAr hit calibration ~ 100 MB
using ORACLE database (under study now) ~ 100 MB
ID GeoModel @ initialisation (overhead) ~18 MB
Inspections @runtime allow to control the memory consumption control the possible memory leaks during data production evaluate pros / cons when a new feature is implemented
They are possible everywhere in the production flow, particulary useful at Begin of Run Begin of Event Begin of Step
And at EOR, EOE, EOS
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 15
G4 Timing: single particle and full events
2191.2200 H(130) 4l, ||<5 2200394420664.0
2265.5200 Z ee, ||<5 1890407790686.5
2546.1200 SUSY/SUGRA events,||<5 1950458290771.5
Configuration Event SizeCPU Time per Event
Full Detector (DC1 Layout) [NCU-s] [SI2k-s](2.4 GHz PIV) [s] [kB]
Heavy ions (Hijing) 3 full events produced
Configuration Event SizeCPU Time per Event(2.4 GHz PIV) [s]
±,||<3.2 ±, ||<3.2e±, ||<2.5 [kB]
Full Detector (DC1 Layout) 1.33 22.7 /40.4 / 10.654.0469.21
Muon System + Toroids
Tile Calorimeter
LAr Calorimeter
Inner Detector
Full Detector, no B-Field
.17
.16
.82
.11
.92
147.9 / 74.4 /2.738.8216.45
0.7 / 0.8 / .319.1310.91
5.8 /16.6 / .6149.5769.50
14.0 /10.2 / 5.1 .930.44
22.0 / 39.3 /10.245.0160.27
pT=50 GeV
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 16
The Physics with DC2 & CTB as a feedback to Simulation
Our goals Get physics community familiar with the new software, new persistency, new analysis
tools Use large produced samples to understand performance and tune new simulation Test algorithms on CTB data:
• Understand limitations of simulation• Understand key issues for reconstruction algorithms• Tune simulation parameters
Get to the end of the year with large well-understood samples of simulated data, stable and tested software chain
• Full simulation analyses (signal + background) for initial detector setup on key physics channels
Inject additional realism into simulation studies
Waiting for feedback from our users community
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 17
Conclusions
The Simulation Geant4-based was successfully tested and it has by now replaced the Geant3-based one
We extensively measured the performance and robustness of the new simulation with great success
We can use Parameterizations for further improving the simulation performance
Geant4 is the main engine for the simulation in ATLAS
Interlaken - 29 Sep 2004 A.Rimoldi, University of Pavia & INFN, Italy Slide 18
Thanks
To all the core developersFor the robust, versatile and complete code provided
To the subdetector people For their prompt implementation of any new functionality provided
To the GEANT4 collaboration for their help in transforming this exercise into a success