csn1 assisi - 23/09/2004 s.falciano - infn roma1 atlas high level trigger/daq infn and univ. di...

CSN1 Assisi - 23/09/2004 S.Falciano - INFN Roma1

ATLAS High Level Trigger/DAQ

INFN and Univ. di Genova, Lecce, LNF, Napoli, Pavia, Pisa, Roma1, Roma2, Roma3

S.Falciano - INFN Roma1


Outline

• Breve status report del TDAQ di ATLAS con particolare riferimento al progetto HLT/DAQ e ai contributi italiani.

• Attività del 2004• Aggiornamento sul lavoro svolto• Milestones 2004• Piano finanziario : richiesta di sblocco s.j. CORE

• Attività prevista per il 2005 • Sviluppi HLT/DAQ• Commissioning• Milestones 2005• Richieste finanziarie


Attività prevista nel 2004

• Integrazione completa di tutte le componenti del LVL1 ad H8 (Muon+MUCTPI, Calo, CTP)

• Integrazione del LVL2 e EF nel Test Beam Combinato ad H8

• Integrazione ad H8 di LVL1+LVL2+EF con elx dei rivelatori e il DAQ; run a 25 ns con trigger di primo livello generato dall’elettronica che funzionerà in ATLAS (LVL1 Calo+Muon + CTP) (“pipelined” mode).

• ROD Crate DAQ per tutti i rivelatori (H8 e Commissioning)

• Monitoring dei rivelatori e del trigger : infrastruttura e applicazioni

• Test del software e delle configurazioni di trigger in laboratorio utilizzando i Testbed

• Definizione e acquisto della “pre-serie” di HLT/DAQ ossia il “Modulo-0”.

In parallelo :

• Sviluppi degli algoritmi di trigger per Muoni (Muon Slice) e Pixel

• Studi di Trigger Performance (rates, capacità di reiezione del fondo, etc.)


Attività svolta nel 2004

• Per quanto riguarda il trigger di muoni di Livello-1 nel barrel, nel 2004 sono stati prodotti e collaudati i prototipi finali delle PADs, e' iniziata la produzione finale delle Splitter boards, e' stato realizzato, collaudato e integrato il prototipo finale di Link ottico e infine completato il disegno finale della Sector Logic e del ROD backplane.

• Per il sistema HLT/DAQ ci si e' concentrati sull'integrazione del trigger di Livello-2 e dell'Event Filter nel Testbeam Combinato ad H8, lavoro attualmente in corso e gia' funzionante a fine luglio per gli aspetti di Data Flow e di Online Software. E' in corso l'integrazione degli algoritmi di trigger - con il meccanismo di steering previsto per l'esperimento - e di quelli di monitoring e di calibrazione. Si e' infine integrato il readout dei Pixel nel DAQ ad H8.


Portable HLT “mini” testbeds

SFI

SFO

EFD AthenaPT

L2SV

ROSEROBROB

LVL2 & EBNetworks

DFM

pROS

ROB

L2+EF testbedInput file

• EF testbed • can be implemented in a single host• used to test the EF algorithms

• L2+EF testbed • requires 2 hosts• used to test the pROS communication between L2 algos and EF algos (the HLT slice online)

Contains the LVL2 result that

steers/seeds the EF processing

L2PU AthenaMT

SFIEmulator

EF testbedInput file


Software HLT

• Il software HLT combina 4 progetti software in uno, con complicate inter-dipendenze – I due package DAQ (Online and Dataflow)– “Offline” e “HLT” releases

• Riuso del software offline negli HLT– Beneficia del framework flessibile, dei servizi e degli algoritmi comuni– Stabilità versus nuove features

• Piani futuri– Sono previsti ulteriori step di ottimizzazione basati sui risultati dei testbed

di larga scala e del testbeam


HLT Event Selection Software

HLTSSW

Steering Monitoring Service

MetaData Service

ROB DataCollector

DataManager

HLTAlgorithms

Processing Task

Event DataModel

L2PU Application

<<import>>

Event DataModel

Reconstr. Algorithms

<<import>>

StoreGateAthena/Gaudi

<<import>><<import>>

Interface

Dependency

Package

Event Filter

HLT Core Software

Offline Core Software Offline Reconstruction

HLT Algorithms

Level2

HLT Data Flow Software HLT Selection Software Framework ATHENA/GAUDI Reuse offline components Common to Level-2 and EF

Offline algorithms used in EF


LVL2 Muon Trigger:

Av. total Av. total time 2 mstime 2 ms

Fast speed increase

• Main improvement comes from RPC and MDT data preparation:– Better use of roads defined by LVL1 trigger– total data-preparation time reduced to ~800 s

• Small improvement in algorithm time too– fewer MDT hits to handle and process saves ~60 s

• Data preparation and Fast now take about 1 ms each, total time about 2 ms.– Network data access time an additional ~0.2ms


EF muon trigger: Trig-

Pt = 100 GeV

< 2.7

G4 – PRELIMINARY

• Nuovo rispetto ai risultati del TDR

• Validata la ricostruzione di nuovi dati G4 con Trig-Moore (EF)

• Esempio di algoritmo di ricostruzione offline adattato dagli autori per lavorare in una regione di interesse guidata dai risultati del LVL2 (“seed”)

• Intera slice LVL1-LVL2-EF validata con “seeding” :

• LVL1 RoIs seeds LVL2• LVL2 result (LVL2 muon track

encoded in raw “byte stream” format and appended to event data) seeds EF

• EF unpacks this as part of the raw event and creates the seed objects

• pT resolution from Geant4 simulation

• Timing: ~150/280 ms (20/200 GeV muon)


HLT at the Combined Test Beam• Two roles:

– Service• Run monitoring algorithms in the LVL2 and EF farm. • Coherent histogram monitoring across the HLT system

– Client• Aim to have the 2 slices (muon and e/) running from LVL1 to LVL2 and EF

• Aim– Test as many aspects of the system as possible and collect enough information to

evaluate the integration issues ahead of us

• Status– LVL2 muon trigger being tested on simulated TB data (June)– LVL2 dataflow integrated into the test beam muon readout chain (July)– EF processing task running Athena (July-August)

• tested remotely in test bed at Barcelona and on testbeam data in Pavia– Have a complete trigger dataflow test chain in place LVL1LVL2EF (End of August)– Full Muon slice in preparation : ready this week? Now a sort of “monitoring algorithm”

is collecting and decoding data, fills and display histograms of MDT and RPCs

• New ATLAS DB project set up : trigger and DAQ part of it.– Use of Muon ATLAS DB to exercise the access to configurations and conditions in one

place from LVL1 to offline software.

LVL1-LVL2-EF-DAQ vertical slice

Event Filter MAGNI cluster

CTP RoIBLVL1A

HLT mini-rack

RCC

ROD

ROD…

ROL ROL

RCC

ROD

ROD…

ROL ROL

…

ROS ROS

SFI SFO

DFM

FileServer

L2 Supervisors

L2 Processors

EF Processors

pROS

Switch

L1 Muon L1 CaloLVL1

Detector

LVL2

Event Filter Control+ EB

EF


TB T/DAQ chain

SFI

SFO

EFDAthenaPT

AthenaPTEFD

AthenaPT

AthenaPT

LVL1

RoIB

L2SV

ROS

ROD

ROB

ROD

ROB

LVL2 & EBNetworks

DFM

pROS

ROB

ROS

ROD

ROB

ROD

ROB

L2PU AthenaMT

Contains the LVL2 result that

steers/seeds the EF processing

L2PU AthenaMT


EF Data Flow

EFP EFD InputTask

Sorting

Task

ExtPTs

Task

ExtPTs

Task

Output

Task

Counting

Task

Histogr.Task

PreProc.

Task

PostProc.

Task

PT

PT

PT

PT

PT

SFI

SFO

InputHandler

Event Selectio

nThreads

LVL1Result

LVL2 Processing Task

EventQueue

LVL2Result

LVL1Result

LVL2Supervisor

LVL2 Processing Unit


DAQ Run Control showing the DAQ Run Control showing the L2 partition up and running L2 partition up and running with L1, RPC and MDTwith L1, RPC and MDT

DAQ Run Control showing the DAQ Run Control showing the L2 partition up and running L2 partition up and running with L1, RPC and MDTwith L1, RPC and MDT


Event Display

Distribution of the hitMDT tube and RPCstrips in the Z-T plane

Beam

Counting

roomBIL

BML

BOLRPC Pivot

RPC Low Pt

RPC High Pt

RPC Overlapping RPC Overlapping geometry is seengeometry is seen

Hodoscope is positioned here

Data from TB Run 600610

Muon setup: RPC + MDT

Trigger: Hodoscope


Beam profile

• Beam profile in the BML station.

• Correlation between RPCs and MDT is seen.

Commissione I - 3/5/2000

MDT hit tube residual(Fast MDT pattern

recognition)

Compute Z of the hit tubecentre; no use of drift timemeasurement.

Z = ( Z tube)/n. hit tube

Then compute the hit tuberesidual:

Z = Z tube – Z

Z

T

check the MDT data quality: electronic noise will broad the distribution.

Check the MDT data decoding: MDT data with low noise, must have the residuals peaked with the expected .

= Wire Pitch x = Wire Pitch x cos(15°)cos(15°) = Wire Pitch x = Wire Pitch x cos(15°)cos(15°)

BIL chamber:BIL chamber:muon track hits are insid the peakmuon track hits are insid the peak

Alessandro Di Mattia

Assuming that the hit tube are due only to muon tracks,and that the muon tracks are more or less collinear with the T axis, the residuals distribution will be a gaussian, with a sigma equal to the wire pitch projected onto the Z axis. The wire pitch is 1.5 cm, half of the tube pitch.Hits from electronic noise will broad the residuals distribution: if the background counting rate overrun the muon counting rate, in the MDT chamber, the gaussian peak is destroyed.This technique allows to monitor the MDT data quality, and to check the MDT data preparation. Moreover, since this technique is part of the muFast processing sequence, it can be consider an integration step


T

Z

RPC-MDT Z correlation

BMLBML

RPC station 2(Pivot)

RPC station 1(Low Pt confirm)

T

Z

Z RPC 2

Z RPC 1

Z MDT

Compute the residuals between RPC and MDT clusters in BML and BOL.

RPC station 3(High Pt confirm)

Z MDT

Z RPC 3

BOLBOL

Z = Z RPC 3 – ZMDTZ = (Z RPC 2 + Z RPC 1)/2 – ZMDT


RPC-MDT Z correlation

Residuals distributions shows correlation between RPC and MDT clusters. Fast pattern recognition is working!

The is better thanks to the two RPC stations.

cluster sizeeffects in the third RPC.


Examples of Moore algs at the CTB

= 61 m

(sag) 63 (sag) 63 mm (sag) 63 (sag) 63 mm

mm

Residuals of the segments fit

Moore Algs are being currently usedfor the EF and TB data offline analysis:

Barrel sagitta (average: misalignment,width: resolution)


DAQ nel Combined Test Beam• Items:

– New Data Flow and Online Software releases– Integration of LVL1 and HLT (configuration, control, data collection, etc.)– Monitoring based on Information Service (Online Software) and Gatherer (for

HLT to handle histos from a farm)• Aim

– Test as many aspects of the system as possible and collect enough information to evaluate the integration issues ahead of us

• Status– Detector integration (ROD Crate DAQ)

• Pixel integrated• SCT problems with ROD, expected to be solved soon • TRT integrated• LAr integrated• Tile integrated• MDT integrated• RPC integrated• TGC integrated• LVL1 muon integrated• LVL1 calo integrated in lab, partially integrated at H8 in August

– Detector Monitoring : done for Tile and MDT, RPC and Pixel in progress– HLT integration done (Data Flow and Online control)


ROD Crate DAQ

• Read-out dei rivelatori splittato su due componenti viste dal DAQ :

– ROD Crate (responsabilità Hw/Sw : Rivelatore)– ROS (responsabilità Hw/Sw : DAQ)

– Nuova implementazione Hw del ROS basata su PC invece che VME– Nuova implementazione del Sw del ROD Crate (ROD Crate DAQ) ora di

responsabilità DAQ. Ciò uniforma la DAQ Locale nel ROD Crate per tutti i Rivelatori (supporto centralizzato per local processing, configurazione, event sampling, etc.)


VME bus

•Total number of ROD crates: 90•Total number of ROS PCs : 144•Total number of racks : ~15”==>All in USA15 (underground)

F.E. ElectronicsF.E. Electronics

… ROD Crates

ROD CrateWorkstation

LAN (GbEth.)

GbEth.

… ROS PCs

ROD Fragments

ROB FragmentsROS Fragments

Event Fragments (Detector specific)

L2 & Event Builder NetworksL2 & Event Builder Networks

ROLs

RCD and ROS

… PCI bus

Config & Control

Event sampling & Calibration data

NIC

RO

BIN

RO

BIN

RO

BIN

RCP

ROD

ROD

ROD

ROD

Config & Control

Event sampling & Calibration data


CTB – Pixel Commissioning

Rod + TIM + SBC nel Rod + TIM + SBC nel crate VMEcrate VME

ModuloModulo

PP0 con PP0 con OptoboardOptoboard

Boc Boc

fibre ottiche I/O fibre ottiche I/O

S-Link S-Link


Pixel DAQ @ 100 kHz

Module readout link @ 40 Mb/sHits dal rivelatore

LVL1 accepts

ROD Busy – tempo mortoindotto dalla scrittura dati sudisco a livello del ROS

Il firmware del ROD è stato debuggato e molti problemi sono stati capiti e risolti.Il Pixel ROD è ora in grado di gestire un flusso continuo di dati a 100 kHz e di gestire correttamente la backpressure applicata dal ROS


CTB – Pixel detectorPixel Box contenentePixel Box contenentesei moduli sei moduli

beam

beam

On-line monitorOn-line monitor


HW status

Monitoring can be done at all trigger levels

Requirements on Monitoring System

ROD ROS SFILVL1LVL1 LVL2LVL2SF0

EVFLTEVFLT

Different monitored information

physics quantities (invariant masses,

Eflow ...)


Event Filter Event Filter AthenaAthena

Monitor overall structure

ROD

ROS

SFI

LVL1LVL1

LVL2LVL2

SF0

MPMP

MPMP

MPMP

MPMP


GathererGatherer

OHistogram Service

Commands to Commands to MPsMPs

Operational monitoring

display (trigger rates,...)Info Info

ProviderProvider Information Service

EMS

Eve

nt

Mo

nit

ori

ng

EMS

EMS



Strategy for Monitoring

ROS OHistogram Service

Commands to Commands to MPsMPs

EMSE

ven

t M

on

ito

rin

g

Read Out System

Event Monitoring Sampler

Monitoring Process

Online Histogramming

Service

Interactive Presenter

File on disk EFS

File Sampler

DAQ/Online SW Group

Monitoring Group

CORE

User libUser lib

User lib

Detector Groups

MPMP


Status of the project & people involved

A first version of the complete chain is working at the Combined Test Beam

User Detector groups are Pixel,Tilecal, MDT, RPC, Beam

Hope to integrate more detectors during CBT (SCT,...)

Ongoing developments on reference histograms, trigger selection, automatic, storing of histograms...

Monitoring Developers : P. Adragna, A. Dotti, R. Ferrari, C. Roda, W. Vandelli, F. Zema.

Users : M. Alviggi, I. Boyko, M. Brinet, D. Dedovitch, M. Della Pietra, L. Santi & people from Monitoring Developers.

DAQ support : M. Caprini, S. Kolos.


Monitoring demonstration


Monitoring at CTB – Tile + Beam


Milestones 2004

• Le milestones scelte implicano lo svolgimento del lavoro della Collaborazione italiana secondo :– i piani del progetto globale TDAQ (e le sue milestones)– step di complessità crescente e passaggi importanti sotto controllo– sviluppi generali che facilitano il lavoro d’integrazione di componenti simili

(ad esempio i vari rivelatori)

• Marzo 2004 : Integrazione Livello-2/Event Filter/DAQ su Testbed al CERN con Livello-1 in emulazione (algoritmi di trigger e HLT software framework descritti nel Technical Design Report) -> OK

• Ottobre 2004 : Integrazione Detector/Livello-1/Livello-2/Event Filter/DAQ su Testbeam in modalità combinata (Pixel+LAr+Tile+MDT+RPC)

-> In progress (~ 50%)


Pre-series : When and where

• LHCC milestone– Pre-series system installed and ready to be used 31.03.2005

--> Finalise technical specification Now Purchase

4Q04 Installation

1Q05

• Several locations considered– Bdgs 32 - 40 - 513 and Point-1

• Point-1 preferred– The most realistic one (ROS rack in USA15, the rest in SDX1 as for final)– No extra investment required– Validate ALL functionality aspects, including real infrastructure– Assess readiness of infrastructure ~ 6 months earlier than required


Pre-series : purpose and scope

• Pre-series System defined as– Fully functional, small scale version of complete HLT/DAQ– Made of components as specified for final system

• Pre-commissioning phase– Validate the complete, integrated, HLT/DAQ functionality– Validate the infrastructure at Point-1

• Commissioning phase– Validate a component (e.g. a ROS) or a deliverable (e.g. a Level-2 rack) prior to

installation and commissioning

• Post-commissioning phase– Validate new components– Validate new software elements or full releases prior to use on experiment– As a TDAQ post-commissioning development system


Pre-series - “Module-0” of final system - 7 racks (10% of final dataflow)

One central switch

-

DAQ rack-

128-port Geth for L2+EB

One ROS rack

-

TC rack+ horiz. Cooling

-

11 ROS44 ROBINs

One Full L2 rack

-

DAQ rack-

32 HE PC

One L2-misc

rack-

DAQ rack-

50% of RoIB

-

3 LE PC(1pROS - 2L2SV)

Part of EFIO rack

-

DAQ rack-

10 HE PC(6 SFI - 2SFO -

2DFM)

Part of EvFilt rack

-

DAQ rack-

12 HE PC

Part of ONLINE

rack-

DAQ rack-

4 HE PC(monitoring)

-

2 LE PC(control)

1ROSrack L2+EB

Switch

5.5

RCC

1L2-misc

rack

1EvFiltrack

1ONLINE

rack

1L2

rack

1EFIOrack

All racks : one or more Local File Servers - One or more Local Switches

USA

15

SD

X1


Costing evaluation steps

• Write technical specifications of components. Currently available specs :– ROS and DAQ Racks (Rack and Cooler)– Rack mountable PCs, two profiles :

• P1 – PCs for ROS racks, 6 PCI slots to host ROBins, 4U chassis• P2 – PCs for Fileservers or for Operations & Control, 1U (in DAQ rack) chassis

– Local Fileserver and Control switch (not yet in formal way)

• Market Survey to understand current costs vs technical specs– Offers available today sometime are not exactly what we need but they help to

understand :• If our costing profile is STILL reasonable• How to proceed to have a product which matches exactly the technical specs (iterative

process)– For some items we have offers which already meet the technical specifications– INFN has made a local price enquiry to contribute in finding products with the required

technical specs and potential companies to be included in the final tendering.

• Purchasing– We shall purchase via CERN but we can specify in detail our sharing (see “Summary”)


ATLAS HLT/DAQ Pre-series For internal use only (page for the ATLAS Referees) Informations on the HLT/DAQ Pre-Series •Introduction •Pre-series in a picture •Pre-series deliverables •INFN deliverables & costing ("ROS Rack" - Partially filled "Online Rack" - Miscellanea) •Summary

Technical Specifications•Rack-mountable PCs for ATLAS DAQ applications (Technical document, Market Survey Qualification Criteria and Questionnaire) •Local Fileserver •Local Switch


ROS Rack (Prices are in kCHF)

Number Unit Cost Cost Description Offers/Estimates Actual cost

Rack (USA15) 1 13 (*) 13

Standard TC electronics rack (52U), includes cooling, power distr., control, etc.

Offer for Rack and CoolerPower distribution and Control

3.4 k€ (offer)

ROS PC 11 8 88

High end PC - redundant PS, multiple PCI slots, etc. (see technical specs)

Offer for a similar PC having the main required features (we may have better offers to match the initial price )

6.6 k€ (offer)

ROBins 44 3 132Custom ROB cards

Local Fileserver 1 5 5Low-end PC + 0.5 TB disk space

Offer for a similar PC (may have less RPS)

4.6 k€ (offer)

Control Switch 1 8 824 port GbE switch, managed

Offer from a reliable company (3COM SuperStack 3 Switch 3870 - data sheet)

3.8 k€

114 246

Cost without ROBinsCost with ROBins

(*) Rack and Cooler (5 kCHF), Power distribution and Control (8 kCHF)

http://www.supermicro.com/products/motherboard/Xeon/GC-LE/X5DL8-GG.cfm

http://www.supermicro.com/products/motherboard/Xeon/GC-LE/X5DL8-GG.cfm

http://www.3com.com/prod/en_IT_EMEA/detail.jsp?tab=features&sku=3CR17450-91

http://www.3com.com/prod/en_IT_EMEA/detail.jsp?tab=features&sku=3CR17450-91

http://www.3com.com/other/pdfs/products/en_US/400845.pdf


Total costing (Prices are in kCHF and k€)INFN contribution (~13% of Pre-series)

Item Cost (kCHF) Cost (k€)

ROS Rack (USA15) 104 69.3DAQ Rack (SDX) 27 18Miscellanea 12 8 Total 143 95.3

Richieste 140 89


2004 2005 2006 2007 2008 2009 Total

Pre-series 140 0 0 0 0 0 140

Detector R/O * 0 275 275 0 0 0 550

LVL2 Proc 0 0 65 195 230 160 650

Event Builder 0 0 50 50 110 70 280

Event Filter 0 0 170 180 570 380 1300

Online 0 45 135 0 0 0 180

Infrastructure 0 0 80 80 20 20 200

INFN Total 140 320 775 505 930 630 3300

TDR Total 1048 3357 4087 4544 7522 4543 25101

INFN Percentage(%) 13.4 9.5 19.0 11.1 12.4 13.9 13.1

INFN cost profile (unit is kCHF) for the chosen categories

(*) The funds for the staged part of the detectors (such as the Pixels) should fall in the year 2008. Therefore they might be subtracted from those of the years 2005-2006, which correspond to the full detector, and moved in 2008.


Attività prevista nel 2005

• Per quanto riguarda il trigger di muoni di Livello-1 nel barrel, nel 2005 verrà sviluppata la produzione dell'elettronica on-detector nella prima metà dell'anno e di quella off-detector nella seconda metà, sarà definito il sistema finale di basse tensioni e inizieranno le procedure per la sua acquisizione.

• Il sistema HLT/DAQ dovrà verificare le sue funzionalità e performance finali utilizzando la cosiddetta "Pre-serie" (~ 10% TDAQ slice) e allo stesso tempo sostenere le attività di commissioning dei rivelatori. A tale scopo verranno acquistati e installati i ROS di Pixel, Tile e Muon, interfacciati ai ROD Crate di questi rivelatori e integrati in una catena completa di acquisizione dati. Continueranno inoltre gli studi di Trigger Performance utilizzando il software di selezione più aggiornato e i dati delle nuove simulazioni in Geant4 con la geometria finale.


Milestones 2005

• Le milestones sono state definite con i seguenti criteri :– Devono rispecchiare le fasi principali del progetto globale TDAQ– Devono accordarsi alle milestone interne (TDAQ) e a quelle esterne (LHCC)– Devono rispecchiare il coinvolgimento finanziario dell’INFN nel progetto

30.09.2005 LVL1 : 100% elettronica on-detector disponibile31.10.2005 LVL1 : definizione sistema finale low voltage31.12.2005 LVL1 : 100% elettronica off-detector disponibile

30.06.2005 HLT/DAQ : Installazione, test e uso della "Pre-serie" (~ 10% TDAQ slice) -> milestone LHCC 30.03.2005

(install.)30.12.2005 HLT/DAQ : Installazione e test dei ROS di Pixel, Tile, Muon (interfacciamento al ROD Crate e integrazione nel DAQ) -> milestone ATLAS 30.09.2005 (ROS

ready)


… as we do in test beam today:from a minimal setup …

(The way to start with a new component)

• Readout Driver (ROD) is detector-specific.

• ROD Crate DAQ: skeleton by TDAQ, detector-specific parts by detector DAQ experts.

• S-link connection to Readout System (ROS). ROS is a TDAQ PC.

• Online Software for configuration, control, monitoring.

RO

D

SB

C

disk

S-link

testeddetector off-detector

readout electronics

ROSOnlineSW

RO

D c

rate

detector

TDAQ


… to the maximum multi-detector combination

DFM

gatewayin bdg.513

SFI

Tra

cker

Cal

oM

uon

det.

LVL1 calo

ROS

ROSRPC

ROSTGC

ROSCSC

ROS

LVL1 mu

Tile

ROS

ROSTRT

ROSSCT

ROSPixel

ROS

LAr

ROS

MDT

LVL1

monitoringrun control

pROS

data

net

wor

k (G

bE)

MAGNI farmin bdg. 513

SFO

remote farms

storageCASTOR (IT)

LVL2farm in H8

EF farm


HLT/DAQ commissioning is driven by detector installation and commissioning, for which we have identified 3 phases:

RCC

ROD

ROD

…

Ethernet

VME

1 - ROD-Crate DAQ (Sub-detector FE commissioning)– ROD read-out on VMEbus

by ROD Crate Controller

RCC

ROD

ROD…

ROL ROL

RCC

ROD

ROD…

ROL ROLROS ROS

Event Building

3 - ROS Read-Out (Multiple partitions)(Sub-detector/TDAQ commissioning)

– Multiple ROD-Crates read-out via ROL-ROS with a minimal EvBuilding

RCC

ROD

ROD…

ROL ROLROS

2 - ROD Read-Out(Sub-detector commissioning)

– ROD read-out on Read-OutLinks with a minimal ROS


Piano finanziario 2005

CORE 2005 :– Detector Read-Out (Rack based, PC based, un rack contiene 11 PCs,

un local file server, un local switch, un rack controller, power e cooling distribution. Il costo di un rack è di 110 kCHF. Vogliamo finanziare :

• 11 ROSes per i Pixel• 26 ROSes per lo Spettrometro a Muoni• 8 ROSes per il TilePer un totale di 5 racks (5 x 110 kCHF = 550 kCHF, 13% del R/O totale)

distribuito su due anni.

– Online (include le farm di processori per l’Online software, ossia Run_control, inizializzazione, etc. e dovrebbe rappresentare anche la frazione di monitoring dei rivelatori. Contributo del 20%)

2005 : 214 k€ + 25 k€ + 4 fte x 4 mesi comm. = 239 k€ + 16 mu comm.HLT/DAQ Roma1 : invent. 7 k Upgrade testbed lab HLT/DAQ (PC, CPU

VME, Switch rete)

apparati 214 k (320 k CHF) come da piano finanziario approvato da lla CSN1 (no

deferral)HLT/DAQ Genova : invent. 10 k Schede DAQ (1 ROS + 1 ROBin)

5 k Single Board Computer per test con RODHLT/DAQ LNF : invent. 3 k PC per sviluppi TDAQ


2004 2005 2006 2007 2008 2009 Total

Pre-series 140 0 0 0 0 0 140

Detector R/O * 0 275 275 0 0 0 550

LVL2 Proc 0 0 65 195 230 160 650

Event Builder 0 0 50 50 110 70 280

Event Filter 0 0 170 180 570 380 1300

Online 0 45 135 0 0 0 180

Infrastructure 0 0 80 80 20 20 200

INFN Total 140 320 775 505 930 630 3300

TDR Total 1048 3357 4087 4544 7522 4543 25101

INFN Percentage(%) 13.4 9.5 19.0 11.1 12.4 13.9 13.1

INFN cost profile (unit is kCHF) for the chosen categories

(*) The funds for the staged part of the detectors (such as the Pixels) should fall in the year 2008. Therefore they might be subtracted from those of the years 2005-2006, which correspond to the full detector, and moved in 2008.


Alcuni dettagli sui finanziamenti richiesti (1)

• Genova : L'idea e' qualle di avere in laboratorio a Genova una fetta completa del read-out dei pixel, diciamo dal rivelatore ai processori di LVL2. Ovviamente il tutto serve sia per studiare il readout che per fare esperienza con gli algoritmi di trigger, entrambe cose di cui Genova ha la responsabilita' in ambito pixel.

• Frascati : Il progetto e' di acquistare un PC da integrare nel rack della farm di calcolo, collegato in gigabit al resto della farm.

• Roma1 : Mantenere in vita il testbed che ci ha gia’ permesso di essere pronti col DAQ dei Muoni lo scorso anno ad H8 e integrare gli algoritmi di trigger nel sistema globale (sostituire i PC che muoiono o diventano vecchi più qualche CPU VME per emulare i ROD dei rivelatori). Continuare le riunioni nazionali sul trigger di muoni che si sono rivelate molto proficue.


Alcuni dettagli sui finanziamenti richiesti (2)

• Commissioning : Il DAQ e' coinvolto nel commissioning dei detector per integrare i sistemi di lettura e leggere i dati comunque generati (test pulse, cosmici, etc.). Le sezioni di Genova, LNF, Pavia e Roma1 partecipano nel 2005 al DAQ per l'integrazione dei ROD e dei ROS, l'event sampling e il Data Flow in generale.

Ricordo che l'installazione dei ROS e' anche nostro compito visto che e' un acquisto CORE dell'INFN e riguarda i rivelatori che costruiamo.

Si pensa che ogni sezione possa fornire 1 fte per 4 mesi, quindi:

4 fte x 4 mesi = 16 mu di ME

Il lavoro si prevede inizi ad aprile e continui fino alla fine dell’anno.


Conclusioni

• Abbiamo raccolto in tempi brevi il frutto di una lunga preparazione -> l’integrazione di Trigger/DAQ ad H8 è stato un successo (e non e’ ancora finita!)

• Il lavoro è stato tanto e duro, ma ci siamo anche divertiti ...

… anche se il “commissioning” è dietro l’angolo.


Riserve


Criteri per la scelta del profilo temporale di spesa

Abbiamo disegnato un sistema che evolve in funzione delle risorse disponibili, che va da un sistema iniziale con una rate massima di LVL1 di circa 40 kHz ad uno finale con una rate massima di 100 kHz

2004 : Acquisto e messa in opera della pre-serie fatta con le componenti

finali del TDAQ (commissioning del Tile?)2005 : Inizia il commissioning dei detector e di HLT/DAQ. Ai detector serve il

75% dei Detector Readout.2006 : Commissioning. Run di cosmici.2007 : LHC start-up. 37% circa delle farm HLT disponibili. 37.5 kHz è la rate

di LVL1 sostenibile. Mancano il 25% dei ROD.2008 : LHC alla luminosità nominale, 75 kHz di rate di LVL1. Questo è il

momento in cui cominciano a tornare indietro i deferral.2009 : Sistema completo con le performance per cui è stato disegnato,

ossia rate di LVL1 di 100 kHz.

Nota : nella fase iniziale la priorità viene data agli aspetti DAQ perche’ essi servono per il commissioning del rivelatore, mentre le farm HLT e in particolare il LVL2, vengono dimensionate al minimo.


Requested INFN funds for the HLT/DAQ system (unit is kCHF)

TDR ItalyPercentag

e (%)More details on funded items

Pre-series 1048 140 13.4

Detector R/O 4060 550 13.5Detector R/O - ROS racks

only

LVL2 Proc 3596 650 18.1 LVL2 racks

Event Builder 2084 280 13.4 EB - EFIO (SFI, SFO)

Event Filter 11451 1300 11.4 EF racks

Online 830 180 21.7Online racks - Control &

Monitoring

Infrastructure 2032 200 9.8 ~ 6% of total INFN funds

Total 25101 3300 13.1HLT/DAQ (no DCS) is

26853 kCHF


ROS Rack

4U

~144 ROS units in total for final system

Pixel SCT TRT LAr Tile Muon

LVL1

ROD Crate

s(links)

8(120)

8(92)

22(232)

16(764)

4(64)

22(272)

6(56)

ROD Rack

sIndicativ

e

4 5 8 8 2 8 312 links / ROS

ROD Rack

5.5

RCC

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

5.5

RCC

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROD

ROS Units

11 8 20 66 8 26 5

ROS Rack

s Indicativ

e

1 1 2 6 1 3 1

ROD crate and ROS all underground

(USA15System size - Full configuration)

Estim

ate -

mig

ht not b

e exact


Hardware components for Phase-2• Availability of hardware components

Component FDR PRR

ROL (Link Source Card) March ‘04

ROBIN May. ‘04 Oct. ‘04

ROS Tech spec’s under review

• ROL production procedure started– First batch (150 boards) : 4Q04 (Nov)– Full production (1750 boards) : 1Q05 (Feb)

• Production procedure for ROBIN identified– Blanket purchase procedure of RAL– Pre-production (50 cards) for pre-series

system : 1Q05– Full production : 2Q05


Relevant DAQ milestonesDAQ – Hardware

PRR/FDR of S-Link Source Card (LSC) 28-02-04 Done

ROBIN FDR completed 31-05-04 Done

Draft tech. Spec for ROS PCs 30-04-04 Done

LSC ready for detector use (1st batch) 31-10-04 On sched.

Installation of ROBIN in ROS (1st batch) 31-07-05

ROS ready for installation 30-09-05

DAQ - Software Releases

Combined test beam (improved control, configuration, monitoring)

30-04-04 Done

Sub-detector readout (final ROD Crate DAQ consolidated) 30-09-04 On sched.

HLT/DAQ installation (final control, config, monitoring + CondDB I/f)

30-09-05

Cosmic-ray run (final large scale perf and full support for partitioning)

30-09-06

LHC start-up (final implementation ready for tuning) 30-04-07

csn1 assisi - 23/09/2004 s.falciano - infn roma1 atlas high level trigger/daq infn and univ. di...

Documents

monitoring e

e iniziata

lesperimento e

ef slide

data flow e

falciano infn roma1

ef validata

ef algos