simple ideas on how to integrate l2cal and l2xft ---> food for thoughts ted may 25th, 2007
Post on 14-Jan-2016
217 Views
Preview:
TRANSCRIPT
Simple ideas on how to integrate L2CAL and L2XFT
---> food for thoughts
Ted
May 25th, 2007
Outline
Comment on the final L2 upgrade commission efforts
Some basics on how FILAR/S32PCI64 works The problem with L2CAL and L2XFT integration due
to large data package size Ideas on possible quick solutions
Comment on the final commissioning efforts
We have a wonderful team: the L2 United team A strong/powerful team in the past two weeks, even with two key people
(the SPLs, Vadim&Gene) on vacation It has been a real pleasure working with them in the past two weeks:
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.QuickTime™ and a
TIFF (Uncompressed) decompressorare needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
people who spent most of their time on final commission in the past ~ two weeks
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
The L2 tough ladies
S32PCI64: single channel FILAR
The four-channel FILAR
S32PCI64: With transmitter firmware: --> Solar With Receiver firmware: --> single channel FILAR
Four-channel FILAR (Rx)
http://hsi.web.cern.ch/HSI/s-link/devices/FILAR
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
CAL1
CAL2
CAL3
CAL4
Reces
SVT
Muon/XTRP/L1
Old cluster
XFT1
XFT2
XFT3
L2 Decision Block:Conceptual view
To L2TS PulsarL2D + (TL2D if L2A)
Control Nodeprocess
New L2 decision
node
Spare node
SolarFilar
Filar
Filar
Inside the Box
decision node
Spare node
data words
CAL1 == 104
CAL2 == 104
CAL3 == 156
CAL4 == 104
Reces
SVT
Muon/XTRP/L1
Old cluster
XFT1 > 120
XFT2 > 120
XFT3 > 120
So what is the the problem then? Both new data paths have large data volume: L2XFT (grow with luminosity, could be very large) + L2CAL (fixed length)
To L2TS PulsarL2D + (TL2D if L2A)
Control Nodeprocess
data packagesfighting over 2 PCI buses, when traffic jam occurs,
data wait in the input FIFO
Input FIFO could overflow IFdata packages too large---> missing data words
--> CPU detects this --> L2DTO
Spare node
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
CAL1CAL2RECESCAL3
SVTMuon/XTRP/L1XFT3
XFT1XFT2Old ClusterCAL4
The actual setup with beam had problem when both L2CAL+L2XFT are included show up as XFT data package corruption --> L2 DTO no problem when only L2CAL is included
PCI bus 1
PCI bus 2
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
CAL1 == 104CAL2 == 104RECESCAL3 == 156
SVTMuon/XTRP/L1XFT3
XFT1XFT2Old ClusterCAL4 == 104
with L2CAL only, i.e. without XFT:--> do not expect FIFO overflow problem
PCI bus 1: on this solar bus,the worst case is likely when L2A with TL2D bank and scaler info sending out from solar… up to ~ 1K words SLINK package… During the time this package is transferringover PCI bus 1, the largest CAL3 data has to wait in the input FIFO. But the maximal number of input events is 3 (since 1 is already on its way L2Aed). 3 x 156 = 468 < 512 FIFO depth… impossible tofill up the FIFO.
PCI bus 2: worst case is 4 L1A in a row…The largest data package is CAL4 == 104
CAL4: 104 x 4 = 416 < 512
No problem
Can we solve this problem?
The Pulsar-based system is very flexible zero-suppression, fast abort, enable back-pressure, use
dedicated PC for dedicated buffer event … etc
Many ways to deal with this, but I will ONLY talk about the simplest possible solutions…(with the potential to be done within weeks, not months).
Will only show two simple ideas today (1) let large XFT data have higher priority over PCI bus (2) use another PC for the XFT data, to do fast-abort or
data suppression, ROI (Region-Of-Interest)…
How to let XFT data packages have higher priority over the PCI bus?
Current way of PCI handling for each data channel: Send requests for up to 4 (effectively) buffer events at begin of run This means that not only the current event package, but also packages from next
event(s) could compete over the PCI bus if they arrive early: in other words, whoever arrives early gets the bus…
This made sense in the past kuz algorithm time is almost zero, and all data path had small size
How to let largest data path have highest priority? This can be done by simply playing tricks with the request FIFO:
Only send requests for up to 4 buffer events for largest data paths at the beginning of run. Do not send request for all other (smaller) data paths
For each event, only after the largest packages arrived to PC memory, send request for the rest of data paths, but only for the current event. Before that, all other smaller data packages will have to wait in their input FIFO: not allowed to fight over the PCI bus.
This should reduce the possibility of FIFO overflow, but may not fix the problem in a fundamental way (as luminosity increases), and likely delay L2CAL path.
But it is the simplest thing to do…this will be done anytime soon…
data words
CAL1 == 104
CAL2 == 104
CAL3 == 156
CAL4 == 104
Reces
SVT
Muon/XTRP/L1
Old cluster
XFT1 > 120
XFT2 > 120
XFT3 > 120
There is simple way to possibly solve the problem:use the spare PC to receive XFT data and do “zero-suppression”in software … then send to main PC …This sounds more complicated than it actually is …
To L2TS PulsarL2D + (TL2D if L2A)
Control NodeProcesscontrolsboth PCs
L2 decisionnode
Simple implementation:
• Send empty package if no need forStereo tracking
• Send aw xft data when needed• (based on L1 bits)• Could also do ROI
More involved implementation:
Do 3D tracking right here for tracks about certain Pt…. your wild ideas here ,,,
Solar
Filar
Filar
(Single channel)Filar
data words
CAL1 == 104
CAL2 == 104
CAL3 == 156
CAL4 == 104
Reces
SVT
Muon/XTRP/L1
Old cluster
XFT1 > 40
XFT2 > 160
XFT3 > 160
Load balancing
Spare PC could use 2 single channel Filars (2048 input FIFO depth) to receive the largest XFT packages…The decision node could also use single channel Filar to receivethe xft package.
To L2TS PulsarL2D + (TL2D if L2A)
Control NodeProcesscontrolboth PCs
L2 decisionnode
Solar
Filar
Filar
(Single channel)Filar
Solar
Filar(single channel)Filar(single channel)Filar The spare PC
This all looks more complicated than it is--- so what is really involved then?
Step 1: how to ask the existing control node process to also control the spare PC? Daniel’s design of the control node can naturally handle multiple PCs (a L3 based design) In Daniel’s words: “just use the CardEditor to change number of PCs from 1 to 2, then input the spare PC IP address”.
Step 2: what do we have to change for the existing code for L2 decision node? Nothing, if one simply use the spare PC to do fast abort based on L1 bits It doesn’t know whether the XFT package is from a PC or a Pulsar merger
Step 3: move XFT fibers from the decision node to the spare node, as well as the copy of muon/xtrp/L1 fiber.
Step 4: could use single channel Filars if so desired (software is the same) Step 5: copy over the decision node software to the spare PC, remove most of the
algorithm stuff, do fast-abort here. If the event doesn’t need stereo tracking, send empty package. Otherwise, send the combined raw data.
Later on, could add more functions for zero-suppression, such as ROI. Or, if you really want, simply do stereo tracking right here in spare PC… e.g. for muons … Your wild idea here… We may want to order 2 more PCs soon, as spares…
System commissioning is fun …
I am leaving tomorrow, for two weeks This is just one of the ideas I have had Didn’t want to talk about it, but young people kept asking Meant as food for thoughts, to encourage young people to brainstorm
Our system is more flexible than we think
About commissioning:
To have a system more or less working is not hard, To have zero error is hard, To keep it that way is even harder…
Whatever you do, think about how to win the peace afterwards…
top related