ieee nss dresden 22 oct. 2008 paolo valente for the computing & software group for the computing...

IEEE NSSDresden22 Oct. 2008

Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group

NA62 Software & NA62 Software & ComputingComputing

Outlook:Outlook:

•From From NA48NA48 data model to data model to NA62NA62•Computing model for the 2007/2008 runComputing model for the 2007/2008 run•Computing model for the KComputing model for the K experimentexperiment•Software architectureSoftware architecture

www.buconero.euwww.buconero.eu Supp

ort I

talia

n re

sear

chSu

ppor

t Ita

lian

rese

arch



NA48 & NA62NA48 & NA62The kaon beam at the SPS North The kaon beam at the SPS North Area:Area:in the heart of the LHC!in the heart of the LHC!



NA48NA48 (K (KLL & K & KSS)) + + NA48/1NA48/1 (K (KSS only) only) + + NA48/2NA48/2 (K (K±±))::•1997,1998,19991997,1998,1999 + + 2000, 20012000, 2001 + + 2003, 20042003, 2004•Up to 135 days/year of runningUp to 135 days/year of running•Up to 5Up to 5××10101010 K decays/year K decays/year•10 kHz trigger rate10 kHz trigger rate•65k events/burst, 600 MB/burst65k events/burst, 600 MB/burst•From From few tensfew tens up to up to 200 TB/year200 TB/year of raw data of raw data

NA48/nNA48/n

•Quasi-online reconstructionQuasi-online reconstruction•Fortran-based reconstruction/simulationFortran-based reconstruction/simulation•Script-based framework for Script-based framework for

•Data-recording (CDR, pioneered by Bernd Painzer-Steindl)Data-recording (CDR, pioneered by Bernd Painzer-Steindl)•Re-processing, data preparationRe-processing, data preparation•User analysisUser analysis

Main result:Direct CP violation established and precisely measured



NA62 NA62 for Ke2/Kfor Ke2/K2 precise measurement2 precise measurement•Collected 120000 signal events Collected 120000 signal events (were 4000)(were 4000)•Data taking in 2007Data taking in 2007 (+ few weeks in 2008) (+ few weeks in 2008)•same DAQ/data recording of NA48/2… same DAQ/data recording of NA48/2… •……added added level-3 data filteringlevel-3 data filtering•300 TB300 TB of raw data of raw data 100 TB 100 TB to tapeto tape

From NA48 to NA62From NA48 to NA62

•Quasi-online reconstruction and Quasi-online reconstruction and event filteringevent filtering•Essentially the Essentially the oldold Fortran-based reconstruction/simulation Fortran-based reconstruction/simulation framework…framework…•… … but with “but with “maintenancemaintenance” of the CDR and reprocessing ” of the CDR and reprocessing environmentenvironment

•Castor2, new streams, event filtering and re-filtering, Castor2, new streams, event filtering and re-filtering, obsolescence of some piece of databases, software…)obsolescence of some piece of databases, software…)

A bridge towards A bridge towards the future the future experimentexperiment……



The future experimentThe future experiment**

•Theoretically cleanTheoretically clean test of Standard Model in the flavor sector test of Standard Model in the flavor sector•Look at KLook at K events (penguin diagram with heavy quarks largely dominating, events (penguin diagram with heavy quarks largely dominating, small hadronic corrections)…small hadronic corrections)…

•ComplementaryComplementary to the high-energy (LHC) approach to the high-energy (LHC) approach•Unfortunately, KUnfortunately, K processes are processes are very rarevery rare! (~10! (~10-11-11) )

•Need Need 12 order of magnitudes12 order of magnitudes rejection power rejection power•Need a very Need a very intenseintense (hadron) (hadron) beambeam… (it’s … (it’s therethere!)!)•Track kaons and track pionsTrack kaons and track pions to use the to use the kinematicskinematics (missing E (missing E from from ))

•with exceptional timing performances to match the tracks in the upstream with exceptional timing performances to match the tracks in the upstream (kaon) downstream (pion) spectrometer(kaon) downstream (pion) spectrometer

•Identify kaonsIdentify kaons andand identify identify pions pions (with dedicated Cerenkov (with dedicated Cerenkov detectors)detectors)•VetoVeto all particles other than all particles other than : :

•new, dedicated veto detectorsnew, dedicated veto detectors for for and and •AimAim: collect : collect ~100 events in two years~100 events in two years… …

•……corresponding to corresponding to ~~10101313 kaon decays kaon decays• New DAQ, new trigger, new computing schemeNew DAQ, new trigger, new computing scheme

**in just 2 slidesin just 2 slides



The future experimentThe future experiment**

Produce hadron beamProduce hadron beam

Define beam momentumDefine beam momentum

Identify kaonsIdentify kaons

Measure kaon momentumMeasure kaon momentum

Look for kaon decays in vacuum…Look for kaon decays in vacuum…

Veto photonsVeto photons

Measure pion momentumMeasure pion momentum

Identify pionsIdentify pions


Veto Veto muonsmuons

**in just 2 slidesin just 2 slides




NA62 detectorsNA62 detectors•Re-use the liquid Krypton calorimeter:Re-use the liquid Krypton calorimeter:

•13,000 cells, new readout electronics13,000 cells, new readout electronics, , no zero-suppression in hardwareno zero-suppression in hardware

•NewNew Silicon pixel kaon tracker Silicon pixel kaon tracker•150 ps time resolution, 150 ps time resolution, 54,000 54,000 channelschannels

•Photon vetoes: Photon vetoes: •re-use OPAL calorimeter lead-glasses, re-use OPAL calorimeter lead-glasses, 2,500 photo-multipliers2,500 photo-multipliers, , new readoutnew readout

•NewNew straw chambers spectrometer: straw chambers spectrometer: •4 stations,4 stations, 8,000 total wires 8,000 total wires , , new new readoutreadout•RICH: RICH: 2,100 photo-multipliers2,100 photo-multipliers, , new readoutnew readout

•CEDAR kaon Cerenkov detector, muon veto, hodoscope:CEDAR kaon Cerenkov detector, muon veto, hodoscope:•newnew readout systems, readout systems, ~500 channels~500 channels



NA62 TDAQ schemeNA62 TDAQ schemeMuon Muon vetoveto

Photon Photon vetoveto

L0 L0 processorprocessor

RICHRICH

L0L0triggertrigger

L0L0primitivesprimitives

PCPCPCPC PCPCPCPCPCPC PCPC

Liquid Liquid KryptonKrypton

Straw Straw chamberschambers

Kaon beamKaon beampixelpixel

L2 triggerL2 triggerEvent building Event building

Detector

Detector

elements

elements

AnalogAnalog

FEEFEE

ADC

ADC

TDC

TDC

FPGA

FPGA

Buffer

Buffer

L0L0triggertrigger

16.8 s/34.8 s CNGS+fixed target16.8 s/34.8 s CNGS+fixed target 0 s/22.8 s LHC+CNGS0 s/22.8 s LHC+CNGS 0 s/21.6 s LHC filling0 s/21.6 s LHC filling~ 30% of spill-time/total time, not synchronous events~ 30% of spill-time/total time, not synchronous events(no bunch crossing!) (no bunch crossing!)

Kaon Kaon CerenkovCerenkov

PCPCPCPCPCPCPCPCPCPCPCPCPCPC ReadoutReadoutL1 triggerL1 trigger

Sub-detectorSub-detector

primitivesprimitives

DATADATA

DATADATASwitchSwitch

Now under development, Now under development, based on the based on the TELL1TELL1 board (LHCb)board (LHCb)

To data To data processinprocessing & g & storagestorage



NA62 data volumeNA62 data volume

L1 + L2L1 + L2 in software, reduction by a factor in software, reduction by a factor ~50~50~ 3 GB/s~ 3 GB/s from Krypton from Krypton withoutwithout zero zero suppression (average event size 250 kB)suppression (average event size 250 kB)~ 100 TB/day ~ 100 TB/day O(10 PB/year) O(10 PB/year)

Only a fraction to be logged to tape:Only a fraction to be logged to tape:•NeedNeed L3 trigger L3 trigger and/or Krypton and/or Krypton zero-zero-suppressionsuppression (with halo expansion) (with halo expansion)

•Reduce by a Reduce by a factor 10-20factor 10-20 •Use pre-scaling & control samplesUse pre-scaling & control samples•Need on-line monitoringNeed on-line monitoring•Keep the system Keep the system highlyhighly flexible/adaptableflexible/adaptable

10 MHz kaon decays 10 MHz kaon decays

Maximum Maximum 1 MHz1 MHz at the at the output of L0output of L0 triggertrigger

Log to tape O(1 PB)/yearLog to tape O(1 PB)/year



NA62 status and plansNA62 status and plansNA62NA62

•Many new groups joined the NA48 coreMany new groups joined the NA48 core•Very advanced R&D status for all detectors:Very advanced R&D status for all detectors:

• Photon and muon vetoes ready, straw chambers ready to Photon and muon vetoes ready, straw chambers ready to start construction; start construction;

• Silicon pixel beam tracker Silicon pixel beam tracker demodemo (detector & readout) in (detector & readout) in 20092009

•2009-20112009-2011: construction of all new detectors, finalize : construction of all new detectors, finalize TDAQTDAQ, finalize , finalize computingcomputing model, run model, run Monte CarloMonte Carlo productions, develop productions, develop softwaresoftware infrastructure infrastructure•2012-20132012-2013 and in case 2014: and in case 2014: data takingdata taking, , processingprocessing•AnalysisAnalysis lasting ~2 years more lasting ~2 years more



Data processing [now]Data processing [now]

RAWRAW

COmPACTCOmPACT(RECO)(RECO)

SuperCOmPACTSuperCOmPACT(DST)(DST)

HyperCOmPACTHyperCOmPACT

ReconstructionReconstruction

Drop variables neededDrop variables neededonly for reconstruction only for reconstruction and calibrationand calibration

Only beam geometry Only beam geometry & spectrum& spectrum

CalibrationCalibration

AnalysisAnalysis

Monte Carlo tuningMonte Carlo tuning

Dat

a si

zeD

ata

size

CPU

pow

erCP

U p

ower

1/61/6

1/21/2

CERN

CERN

Off

-site

Off

-site



CERN CERN computing computing

centercenter

Experiment Experiment computing computing resources resources

Off-site Off-site computing computing resources resources

Data recordingData recordingProcessing (pass-1) / L3 Processing (pass-1) / L3

filteringfilteringnetworknetwork


CalibrationCalibration

networknetwork

Event buildingEvent buildingL1-L2 triggerL1-L2 trigger

RAWRAW

““burstlets”burstlets”

COmPACTCOmPACTMet

a-da

taM

eta-

data




centercenter



User analysisUser analysisSimulationSimulation

networknetwork


Re-processing (pass-2+)Re-processing (pass-2+)networknetwork

RAWRAW

COmPACTCOmPACTMet

a-da

taM

eta-

data

Super-COmPACTSuper-COmPACT




centercenter




networknetwork


Re-processing (pass-2+)Re-processing (pass-2+)networknetwork

Super-COmPACTSuper-COmPACT



networknetwork

networknetwork


centercenter




Processing on-line Processing on-line (pass 1)(pass 1)

L3 filteringL3 filtering

Data processingData processing

Move processing to the Move processing to the experiment farm?experiment farm?- Need enough computing power Need enough computing power in order to avoid back-logs to in order to avoid back-logs to DAQDAQ

Data recordingData recording

Re-processing (pass-2+)Re-processing (pass-2+)CalibrationCalibration




networknetwork

networknetwork


centercenter


Off-site Off-site computing computing resources resources CalibrationCalibration

Re-processing (pass 2+)Re-processing (pass 2+)


Processing on-line Processing on-line (pass 1)(pass 1)

L3 filteringL3 filtering

Data processingData processing

Move re-processing off-Move re-processing off-site?site?- Need access to (filtered) RAW - Need access to (filtered) RAW datadata

Data recordingData recording




Software infrastructureSoftware infrastructureTr

igge

r &

Filt

erTr

igge

r &

Filt

er

Reco

nstr

uctio

nRe

cons

truc

tion

Anal

ysis

Anal

ysis

Sim

ulat

ion

Sim

ulat

ion

LibrariesLibraries

FrameworksFrameworks

No big deal in demonstrating the need of a No big deal in demonstrating the need of a new infrastructure:new infrastructure:•Cannot rely on old technologies Cannot rely on old technologies

•At least for support problems: FORTRAN At least for support problems: FORTRAN code, geant3, cernlib, zebra, hepdb, …code, geant3, cernlib, zebra, hepdb, …

•Existing “framework” based on [different] Existing “framework” based on [different] scripting languages: scripting languages:

•Hard to maintainHard to maintain•Certainly, not GRID-compliantCertainly, not GRID-compliant•AlreadyAlready need to use GRID for Monte Carlo… need to use GRID for Monte Carlo…

•Need to handle 1 PB/yearNeed to handle 1 PB/year•~1 order of magnitude wrt past experiment ~1 order of magnitude wrt past experiment (and cannot scale man-power accordingly…)(and cannot scale man-power accordingly…)



Software frameworkSoftware framework

….WHY DON’T YOU USE ALIROOT / GAUDI / … ?

DO I REALLY NEED THAT FOR NA62 ???

Use cases…Use cases…

Data modelData model

Software Software architecturearchitecture

Software frameworkSoftware framework

User softwareUser software

TheoryTheory ExperimentExperiment



What the framework should do for us…

• Use casesUse cases• Define the algorithms to run in the codeDefine the algorithms to run in the code• Handle input/output file catalogs Handle input/output file catalogs • Handle meta-data (geometry, conditions, calibrations)Handle meta-data (geometry, conditions, calibrations)

• handle different databaseshandle different databases• Handle different running configurations (online, reprocessing, Monte Handle different running configurations (online, reprocessing, Monte Carlo, test, …) of software:Carlo, test, …) of software:

• different configuration for input, output, algorithms to run, load/save meta-different configuration for input, output, algorithms to run, load/save meta-data, connection to databases…data, connection to databases…

• manage versioning of software, multi-user development of codemanage versioning of software, multi-user development of code• manage different versions of datamanage different versions of data• manage changes in OS/system callsmanage changes in OS/system calls• jobs submissionjobs submission• error recoveryerror recovery



AlgorithmAlgorithm

TransientTransientdatadata PersistentPersistent

datadata

ServicesServices

Exploring GAUDI:Exploring GAUDI:•Satisfies NA62 requirementsSatisfies NA62 requirements•Exists (and works!)Exists (and works!)•Good support (and lasting at least as long as LHC)Good support (and lasting at least as long as LHC)•Experience in some NA62 InstitutesExperience in some NA62 Institutes•Natural integration with GRID technology through GANGANatural integration with GRID technology through GANGA



Monte CarloMonte Carlo•Several kinds of Monte Carlo already existing:Several kinds of Monte Carlo already existing:

•It is not It is not conceivableconceivable to generate 10 to generate 101313 kaon decays with full kaon decays with full tracking of particles inside a detailed GEANT description of the tracking of particles inside a detailed GEANT description of the experiment…experiment…•GEANT4 detailed simulations of detectors responseGEANT4 detailed simulations of detectors response

•““Fast” simulation using:Fast” simulation using:•Physics generatorsPhysics generators•Accurate beam and halo parametrizationsAccurate beam and halo parametrizations•Simulation of setup as passive materialsSimulation of setup as passive materials•Parametrization of detectors response from GEANT4Parametrization of detectors response from GEANT4

•Full reconstruction in C++Full reconstruction in C++•Root persistencyRoot persistency



•NA62 is awaiting its NA62 is awaiting its finalfinal approval approval•Even though:Even though:

•Some detectors are still finalizing FEE and readout schemeSome detectors are still finalizing FEE and readout scheme•Data volume is largely dependent from Liquid Krypton readoutData volume is largely dependent from Liquid Krypton readout•TDAQ quickly evolving towards its final formTDAQ quickly evolving towards its final form

some some key pointskey points are firm: are firm:•Need a “smart” way of reducing data (Liquid Krypton zero-Need a “smart” way of reducing data (Liquid Krypton zero-suppression and/or L3 filtering)suppression and/or L3 filtering)•~ 1 PB/year of raw data to:~ 1 PB/year of raw data to:

•log to tape, reconstruct, calibrate, re-process, convert to log to tape, reconstruct, calibrate, re-process, convert to data-summary formatdata-summary format

•Need large Monte Carlo productionsNeed large Monte Carlo productions•Need to do all that from now to ~2015: new [or not-so-old] Need to do all that from now to ~2015: new [or not-so-old] technologies!technologies!

Final remarks



Beyond NA62Beyond NA62

NA??NA??•2015-20202015-2020•Increase beam intensity for ~1000 KIncrease beam intensity for ~1000 K events events•Improve detector for KImprove detector for KLL



Step 2Step 2: ~1000 K: ~1000 K++ eventsevents

Step 3Step 3: ~100 K: ~100 KLL eventsevents

Step 1Step 1: ~100 K: ~100 K++ events, events, NA62NA62

ieee nss dresden 22 oct. 2008 paolo valente for the computing & software group for the computing...

Documents