ADRIAN FIERGOLSKI1,2, MICHELE QUINTO1,3
1INFN-Bari, Italy 2 Warsaw University of Technology, Poland
3University of Bari, Italy
TOTEM READOUT USING THE SRS
RD51 E-School, 3rd of February 2014
Outline
• Introduction of the TOTEM experiment• Integration of SRS with the TOTEM DAQ• Firmware development• Firmware verification• Tests and results of SRS-based DAQ
3-Feb-2014 Adrian Fiergolski, Michele Quinto 2
TOTEM experiment
3-Feb-2014 Adrian Fiergolski, Michele Quinto 3
T1: Cathode Strip ChamberT2: triple GEM
Roman Pot: 10 planes of silicon edgeless detector
• TOTEM uses 3 tracking detectors (T1,T2,RP) located symmetrically with respect to the IP5
• For the luminosity independent measurement of the p-p cross section at low momentum transfer, TOTEM requires magnetic configuration of the accelerator optics with high β* (90m, 1000m, 1535m)
• TOTEM demands special LHC runs, which allow RP to approach the beam
TOTEM DAQ before Long Shutdown (LS1)
3-Feb-2014 Adrian Fiergolski, Michele Quinto 4
In the TOTEM standalone configuration, the VME bus bandwidth limitsthe trigger rate to 1 kHz.
VFAT2 chip
3-Feb-2014 Adrian Fiergolski, Michele Quinto 5
• Trigger and tracking capabilities• 128 channels• 0.25 µm CMOS process node• supports LHC clock frequency of 40 MHz• Radiation hard• Single Event Upset (SEU) protection
Optical transmission
3-Feb-2014 Adrian Fiergolski, Michele Quinto 6
Data
16
TX_en
TX_er
READY 800Mbps1310 nm
9/125 µm single-mode fiber
K28.5 D5.6 K28.5 D5.6 D0.6 D27.5 D13.6 D16.6 D11.7 K28.5 D5.6 K28.5 D5.6
Comma CommaData
Ethernet codding (IEEE-802.3):
8b10b Encoding
3-Feb-2014 Adrian Fiergolski, Michele Quinto 7
Assumptions:• line code• maps 8-bit symbols to 10-bit symbols
Properties:• DC-balance • bounded disparity• reasonable clock recovery
Code structure:• Difference between the count of 1s and 0s in a string of at least 20bits
is no more than 2• No more than five 1s or 0s in a row
All codes that represent the 256 data values: data (D) codes.The codes representing 12 special non-data characters: control (K) codes
8b
10b
TOTEM’s DAQ evolution
3-Feb-2014 Adrian Fiergolski, Michele Quinto 8
Scalable Readout System
3-Feb-2014 Adrian Fiergolski, Michele Quinto 9
Advantages:• Cost effective replacement for the currently used
VME-based solution offering higher bandwidth• TOTEM’s implementation will be compatible with the
CMS DAQ• Allow standalone runs of TOTEM• Enable hardware data filtration
TOTEM’s C-Card: Opto-FEC
The development board linking the OptoRx and the FEC.
32-bit parallel bus following S-Link protocol clocked at 40 MHz
2.5 Gbps SERDES 8 LVDS lines providing 5.36 Gbps Clock generator/jitter cleaner TTC interface I2C configuration lines TTS support JTAG support Independent power supply mode
3-Feb-2014 Adrian Fiergolski, Michele Quinto 10
Firmware development guidelines
• Hardware description and verification in System Verilog language– compactness, syntax structures
→ more re-usable, less error prone code– the language consequently gains attention of industry
→ increasing maturity of the EDA tools– Possibility to use legacy VHDL, VERILOG modules (eg.
open cores)• The communication between entities via standard interfaces
– AMBA AXI4-Stream, AHB• Automatic register generation from register map specification
– IDesignSpec
3-Feb-2014 Adrian Fiergolski, Michele Quinto 12
Firmware scheme
3-Feb-2014 Adrian Fiergolski, Michele Quinto 13
• System UnitProvides set of common interfaces and services. Development of mutual modules can by a shared effort of the SRS community.
• Application UnitApplication specific data processing part.
Firmware verification
3-Feb-2014 Adrian Fiergolski, Michele Quinto 14
The firmware is simulated using System Verilog combined with the Universal Verification Methodology (UVM):
• High level of abstraction (reusable)• Random test vector generation (guided by constraints)• Coverage indicating verification progress• EDA tools provide UVM libraries to test popular interfaces (eg. Ethernet, I2C)
The verification of the FEC defines two kind of simulations:• Partial simulation
→ to achieve faster simulation coverage of complex modules• Full design
SRS based DAQ in the LHC environment
3-Feb-2014 Adrian Fiergolski, Michele Quinto 15
Conditions:• data from 3 full RP detectors containing about 120 VFATs• FEC was read directly by a standard PC running DATE software• commercial SATA storage medium
Results:Without transmission error, the system acquired 10M events reaching maximum trigger rate of 10 kHz
Even faster SRS
3-Feb-2014 Adrian Fiergolski, Michele Quinto 16
• Trigger rate at flat top ~25kHz• Readout bandwidth close to the
link limit 118MB/s• System stability over more than
140M events• None of the event has been lost
Modifications:• new firmware following the presented guidelines• distributed data storage on up to 3 DAQ nodes• custom, combined hardware-software solution to achieve
lossless transmission via unreliable UDP protocol
Conclusion:In term of trigger rate, the new DAQ is more than one order of magnitude faster,
reaching 24.7kHz against 1 kHz of standard, VME based , system.