Simulation in Babar

Concezio BozziINFN Ferrara

Desy/ECFA workshopCern, 15/11/2001

http://www.fe.infn.it/~bozzi/G4cern.ppt (or .pdf, .ps)

Outline

• Reminder of Babar computing

• Geant4 and Babar simulation

• Simulation production management

• References: – Babar reports at G4 workshops and reviews (D. Wright)

– Babar reports at CHEP01 conference

– Babar Simulation NIM paper (in preparation…)

Physics Framework

• Data taking since Oct 1999

• Luminosity greater than design (>4 1033 cm-2 s-1) regularly achieved

• Yearly integrated luminosity expected to grow in the future:– 25fb-1 in 2000– 40fb-1 in 2001– 225fb-1 in 2005

620fb-1 in total

Data storage• Objectivity OO Database technology• Hierarchical structure

• Disk & tape (HPSS) storage

RAW (50kB/evt)

REC (150kB/evt)

ESD (5kB/evt)

AOD (5kB/evt)

TAG (0.5kB/evt)

Navigation DBs

Data 1999+2000: 128TB (7.3 on disk)~27500 files

MC 1999+2000: 75TB (3.1 on disk)~25300 files

NOW: 350TB

Mic

roMin

i

The Babar computing model

OBJY main event store• Filtered events (“skims”)

– Pointers to real events– Little overhead– Only navigation DBs needed

• Powerful tool for hosting a large fraction of data at SLAC

• Exportation of skimmed collection very inefficient– OBJY limitation of

64kDB/federation– Large DBs (2-10GB)

– 1 collection: several hundreds of GBs data

• Introduction of a compact format based on ROOT (KanGA)– Self-contained events– Significant overhead

• In parallel:– Develop multiple federations– Smaller DBs (0.5 GB)– Self-contained physics streams

(20) for data distribution at remote sites

• Drop Kanga if new system OK

The Babar computing model (2)

• Multi-Tiers (a-la-LHC)• Tier A: main data

repository and distribution center, all data available, all kinds of analyses possible (SLAC, in2p3, ral, infn)

• Tier B: regional, data for physics analysis (micro)

• Tier C: institutes, universities, small sample of data

3121991318254Band Mb/s

38624616210166Sum

704429169Raw

232148985422Rec

4730191820Mini

3321141115Micro

4.7321.11.5Tag

20052004200320022001(TB)Expected data to be xferred to TierA

Transfer of MC: multiply x2-x3

Sketch of data distribution

Geant4 simulationGeant4 embedded within BaBar software• BaBar Framework - outermost layer

– Handles input/output from database, files, screen, etc.– Processes data by creating objects of classes containing

BaBar code (modules) which • take data from a framework container• run specific algorithms• put results back into container for use by subsequent modules

• BOGUS - next layer down– Performs G4 based simulation within framework– Passes results to detector response module then to

reconstruction module

• Geant4 toolkit - next layer down– used to do large part (but not all) of simulation

G4 in the Babar simulation

B - BaBar

O - Object-oriented

G - Geant4-based

U - Unified

S - Simulation

G4 features used in Babar• Geometry

– Constructed Solid Geometry (CSG) solids– boolean volumes (using CSG constituents)– G4 materials

• Hit scoring– G4 virtual sensitive detector– G4 sensitive detector manager

• Physics processes– EM physics (E < 10 GeV)– G4 hadronic processes also used (E < 10 GeV) but not

fully tuned yet

• Visualization (openGL)

Modeling of physics processes• Particle range cut (instead of energy threshold)

– 0.22mm (δ rays produced in DIRC above Cherenkov threshold)– Eventually allow for material dependence?

• EM physics processes:– Photons: G4ComptonScattering, G4GammaConversion,

G4PhotoElectricEffect– Electrons/positrons: G4eIonisation, G4eBremsstrahlung,

G4eplusAnnihilation– Muons: G4MuIonization, G4MuBremsstrahlung, G4MuPairProduction

– Hadrons: G4Ionisztion– All charged particles: G4MultipleScattering

• Hadronic physics processes (LE models):– Elastic:G4HadronElasticProcess (G4LElastic)– Inelastic, eg p+: G4PionPlusInelastic Process (G4PionPlusLEInelastic)

G4 features NOT used in Babar• Tracking: BgsTransportation

– Developed at SLAC because of required precision, fast execution (reduced number of calls to B field), no bias in tracks

– no Runge-Kutta integration, uses only straight lines and helices and is fast and accurate for nearly uniform BaBar field

– 4 criteria for minimum step

– Intersection with G4 volumes by iterative procedure

– other 9 G4 steppers being tested (RKG3_stepper appropriate?)

• Detector response (most of BaBar detector response code written before G4 was available)

• Persistence (BaBar database and framework use their own persistent objects)

Geometry

Implementation of G4 specifics solids:(polyhedra, polycone, hype, elliptical tube)

PEP geometry and SVT

Geometry diagnostic tools

• Material scanner– returns radiation length, interaction length, path

length, theta, phi, coordinates of track exit point as an n-tuple

• Geometry checker– specifies straight lines through detector

– returns boundaries along line

– useful for finding volume overlaps

Material validation• Bremsstrahlung in

Bhabha events• Conversions in gamma-

gamma events

Tracking performance: µ+µ-, bb

D0 mass

D0 lineshape fits

Sample Mean (GeV) RMS width (GeV) Chi2/dof2001 Data 1.86351 ± 0.00007 0.00697 ± 0.00019 32.1/32

SP4 Geant4 1.86406 ± 0.00006 0.00684 ± 0.00009 27.8/25SP4 Geant3 1.86514 ± 0.00005 0.00476 ± 0.00014 39.5/24

SP3 1.86504 ± 0.00005 0.00491 ± 0.00016 29.5/28

EMC comparisons• G3/G4 show no significant differences

IFRvariables

µ+µ-

Summary of comparisons

• Detector Material Model validated for tracking region of detector

• Resolution and decays– reconstructed mass resolution looks good – track resolution needs more work

• EMC performance similar in G3/G4– G4/data ongoing

• IFR: more tuning needed– not worse than G3

Run-time performance• Fast

– except for initialization time, as fast as G3– initialization time = 30 cpu seconds

CPU times using SLAC Linux farm (seconds/event) : Event type Bogus (G4) Bbsim (G3)B0B0bars 5.43 6.24mu pairs 1.10 0.62Bhabhas 8.52 7.58

• Robust– low crash rate (~1ppm)– No significant memory leaks– running on three platforms (Linux, Sun, Dec)

• Production using G4 has begun– >100M events produced so far– >50% produced outside SLAC

Simulation Production Management• 3-staged simulation production system:

– Simulation using Geant4 (or 3)– Mixing: add detector response and background signals– Reconstruction: like in real data

• Events output from one stage are used as the input to the next stage

• Spread production on several sites• Many user requests, many production jobs

(100,000), many events (500M)– 1 CPU for about 150 years -> use 500 CPUs

• Most production done on >800MHz Linux (bi-processor) machines, but also on Sun and OSF

Structure of the system

• Collect and centrally manage requests for simulation– Web interface to Oracle DB

• Distribute requests to production sites– Separate request into runs (each of them requires 8 hours of

computing for all 3 stages), usually 2K to 10K events– Assign to production sites

• Manage jobs at each site– Set of perl scripts, interacting with Oracle DB (proxy)

• Import events to SLAC– Automatic procedure, perl script– Closed DBs extracted from OBJY and transferred to SLAC– Automatically imported into the SLAC federation

Status of G4 MC production• Start: Aug. 15th (4 sites)

• 15 sites in production

• More to come

• Nominal rate: 3Mevts/day– 2M outside SLAC

Conclusions

• Geant4 toolkit embedded in Babar simulation– Conservatively for now…– Fast and robust

• Better or at least as good as Geant3– Material scan– Tracking– EMC– IFR: still room for improvement, tuning…

• Huge simulation production efforts (>80Mevts/month) started in summer 2001