Download - Towards a CBM-XYTER Readout Chain
Towards a CBM-XYTERTowards a CBM-XYTERReadout Chain Readout Chain
Walter F.J. Müller, GSI, Darmstadt
11th CBM Collaboration Meeting26 February 2008
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI 2
CBM-XYTER DataCBM-XYTER Data
A CBM-XYTER hit can be represented in 6 bytes (48 bit) The 'hit message' might look like (here 42 bits)
message type (1 bit) chip id (~ 8 bit) channel id (7 bit) status flags (~4 bit) time stamp (~14 bits) amplitude (~8 bits)
On a serial link we have some overheads Framing, FEC, ect: ~75% efficiency 8bit/10bit coding
'on-the-wire' we need about 80 bit to encode a hit
2.5 Gbps ↔ 2.5 Gbps ↔ 31.2531.25 MHz hit rate MHz hit rate
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI 3
Some ValuesSome Values
Use hit densities form CBM Technical Status Report2006 Update, Section 13.1 "Hit densities and Rates"
Detector edge hit/cm2 part/cm2 TIDSTS @ 30cm inner 10 7.5·1014 20 Mrad
outer 0.25 1.8·1013 0.5 MradSTS @ 1m inner 1 7.5·1013 2 Mrad
outer 0.03 2.3·1012 60 kradTRD @ 4m inner 0.04 3.0·1012 80 krad
outer 0.002 1.5·1011 4 kradTOF @ 10m inner 0.01 7.5·1011 20 krad
outer 0.0006 5.0·1010 1.2 krad
STS @ 30 cm is now 1st plane in 'all strips' configuration(the hit rate for STS@30 cm is scaled from the STS3 @ 20 cm plot of the CBM TSR)
Hit rates in 1st MUCH plane are similar to STS plane @ 1m
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI 4
STS Data Rates – Rough STS Data Rates – Rough EstimateEstimate With 60 μm pitch: 128 channel ↔ 0.768 cm The hottest place:
10 hit(cent)/cm2 ↔ 2.5 hit(mbias) /cm2 ↔ 25·106 hit/(s· cm2) A 2cm sensor ↔ 1.5 cm2 ↔ 37.5 MHz hit rate/chip Note: background and cluster size not included here
A typical data link to DAQ: Assume 2.5 Gbps link speed from here on
Number of data links needed: A single CBM-XYTER can fill a 2.5 Gbps link most of the CBM-XYTER see less hits in most cases 2, 4, or 8 CBM-XYTER's can be aggregated
to fill a 2.5 Gbps link
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI 5
STS Data Rates – MC ResultsSTS Data Rates – MC Results
Radek made a detailed study. The current 8 station STS with a total of ~1.2 Mchannels, 9296 CBM-XYTER chips 586 modules (with mostly 2 * 8 chips)
Note: cluster size not yet included ( 1 strip hit per particle)
(ALICE SSD has an average cluster size of ~1.4) effects due to non-perpendicular incidence also still
ignored
Following tables under the caveats listed above
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI 6
STS Data Rates – Au+Au @ STS Data Rates – Au+Au @ 25AGeV25AGeV
Hit rate [MHz]
Sta 1
Sta 2
Sta 3
Sta 4
Sta 5
Sta 6
Sta 7
Sta 8
Total
16-32 23 16 26 16 19 0 6 2 105
8-16 37 55 50 60 66 64 55 58 431
4-8 80 75 101 125 84 84 67 72 692
2-4 111 138 123 89 133 141 129 157 1053
1-2 109 142 162 170 176 250 161 141 1275
0.5-1 127 193 74 41 54 21 210 256 971
< 0.5 53 13 0 3 0 0 28 2 121
Chip count only for one of the two sides of the modules Properties of stations quite similar (module size tracks quite
well the inverse hit density hit/chip roughly equal)
Hit rate per chipHit rate per chip
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI 7
STS Data Rates – AggregationSTS Data Rates – Aggregation
75% of the chips have < 4 MHz It is prudent to aggregate the data of several chips
onto one data link before it is send of the module
Hit rate [MHz]
# modules
128-150 16
64-128 126
32-64 188
16-32 260
8-16 312
4-8 242
0-4 28
Hit rate per moduleHit rate per module Only 2.5% of chips have a hit rate of 16-32 MHz
842 of 1172 of the module sides (71%) have a sum hit rate of <32 MHz, can thus be aggregated on a 2.5 Gbps link
in 188 module side two groups of 8 chips can be aggregated on a link
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI 8
STS Data Rates – # of Data STS Data Rates – # of Data LinksLinks 75% of the chips have < 4 MHz It is prudent to aggregate the data of several chips
onto one data link before it is send of the module
Hit rate [MHz]
# module
s
#links
128-150 16 128
64-128 126 504
32-64 188 376
<32 842 842
Hit rate per moduleHit rate per module Total hit rate: 32.6 GHz Net data rate: ~200 GByte/s Number links: 1850
Per station end (top/bot) we have about 120 data links
- Aggregation may be cleverer, links a little faster- Cluster size will be >1 and increase data volume About 2000 'fast' links seems a reasonable estimate About 2000 'fast' links seems a reasonable estimate
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI 9
In many cases information of all 8 chips can be aggregated to fill one 2.5 Gbps link
Some schemes: embedded (aggregation integrated in CBM-XYTER)
with communication controller (aggregation on separate chip)
CBM-XYTER@STS ReadoutCBM-XYTER@STS Readout
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI 10
GBT: The CERN proposal for GBT: The CERN proposal for SLHCSLHC The GBT project at CERN addresses these issues
targeted for SLHC radhard design communication-controller handling
clock distribution trigger distribution slow control traffic data readout traffic
exactly what we talk about since years
In the following a few slides from Paulo Moreira's presentation on the TWEPP-07 workshop.
The GBT, a Proposed Architecture for Multi-Gb/s Data Transmission in High Energy
Physics
P. MoreiraCERN – Geneva, Switzerland
Topic Workshop on Electronics for Particle PhysicsSeptember 3 - 7, 2007, Prague, Czech Republic
http://indico.cern.ch/contributionDisplay.py?contribId=71&sessionId=16&confId=11994
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI
12
Outline
Motivation The proposed link architecture
Link topology GBT chipset Link bandwidth GBT to Front-end link topology
TTC Slow control Development and testing Error correction and line coding Summary
Disclaimer:Disclaimer:“This project is still in its specification phase, detailed features are likely to change prior to the final silicon fabrication…”The objective of this presentation is to communicate the current ideas to stimulate discussion and to obtain feedback from the potential users!
Slide from: TWEPP-07 presentation of Paulo Moreira (see this Slide from: TWEPP-07 presentation of Paulo Moreira (see this link) )
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI
13
GBT link architecture
Bidirectional point-to-point optical fiber links The heart of the link is the GigaBit Transceiver (GBT13) Each link carries simultaneously: DAQ, TTC and SC data Embedded transceiver: the GBT Chipset Counting room transceivers: FPGA and COTS optoelectronics
components
Versatile optical link project
GBT project
GBT project
Slide from: TWEPP-07 presentation of Paulo Moreira (see this Slide from: TWEPP-07 presentation of Paulo Moreira (see this link) )
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI
14
The GBT chipset
GigaBit Transimpedance Amplifier (GBTIA)
GigaBit Laser Driver (GBLD)
GigaBit Transceiver (GBT13) SERDES Communications controller TTC receiver
GBT – Slow Control ASIC
Slide from: TWEPP-07 presentation of Paulo Moreira (see this link) Slide from: TWEPP-07 presentation of Paulo Moreira (see this link)
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI
15
GBT to front-end link topology
GBT to Front-end links: 32 (+1) bi-directional serial links (e-links) E-link bandwidth is 80 Mb/s Several e-links can be grouped together to serve a single front-end device achieving
bandwidths that are multiples of 80 Mb/s 16-bits for TTC 8 phase adjustable clocks An E-Link Port Adaptor (EPA) “macro” will be available for integration
in the front-end ASICs
FE ASICsclk
Link controller
Link controller
Data MoverData
Mover
TimingTiming
portport
portport
portport
Controlport
Controlport
Front-Ende-links
GBTManagement
logic
GBTManagement
logic
Slow Control ASICe-link
clk40T<15:0>
e-link
e-port
e-port
ReadoutControl Din
Dout
clk
e-link (6 wires)LVDS: Clk80, Din, Dout
32 ports80Mb/s
Timing
T<15:0>
GBT-SCAGBT-SCA
GBT13
FE ASICsclk
Link controller
Link controller
Data MoverData
Mover
TimingTiming
portport
portport
portport
Controlport
Controlport
Front-Ende-links
GBTManagement
logic
GBTManagement
logic
Slow Control ASICe-link
clk40T<15:0>
e-link
e-port
e-port
ReadoutControl Din
Dout
clk
e-link (6 wires)LVDS: Clk80, Din, Doute-link (6 wires)LVDS: Clk80, Din, Dout
32 ports80Mb/s
Timing
T<15:0>
GBT-SCAGBT-SCA
GBT13
Slide from: TWEPP-07 presentation of Paulo Moreira (see this link) Slide from: TWEPP-07 presentation of Paulo Moreira (see this link)
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI
16
SC – Two chips solution
Slide from: TWEPP-07 presentation of Paulo Moreira (see this link) Slide from: TWEPP-07 presentation of Paulo Moreira (see this link)
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI
17
Link bandwidth
GBT: 120-bits transmitted during a single bunch crossing interval 4.8 Gb/s.
4 header bits 32 forward error correction bits User field of 84 bits 3.36 Gb/s:
SC: 4-bits 160 Mb/s TTC: 16-bits 640 Mb/s DAQ: 64-bits 2.56 Gb/s
Bandwidth is fixed per frame but can be shared by the front-end devices
Slide from: TWEPP-07 presentation of Paulo Moreira (see this link) Slide from: TWEPP-07 presentation of Paulo Moreira (see this link)
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI
18
Forward error correction (FEC)
Objectives: Correct burst errors:
Generated on the PIN-diode Generated by particles hitting the transceiver Generated in the fast SERDES circuits (which cannot be triplicate) Done with minimal latency Done with good efficiency Merge with line-coding
Proposed code: Compatible with FPGAs capabilities Interleaved Reed-Solomon double error correction 4-bit symbols (RS(15,11)) Interleaving: 2 Error correction capability:
2 Interleaving × 2 RS = 4 symbols 16-bits
Code efficiency: 88/120 = 73% Line speed: 4.80 Gb/s Coding/decoding latency: one 25 ns cycle
Slide from: TWEPP-07 presentation of Paulo Moreira (see this link) Slide from: TWEPP-07 presentation of Paulo Moreira (see this link)
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI
19
Summary
We propose a “Single” Link solution for: Timing Trigger Links Data Acquisition Links Experiment Slow Control Links
The link is based on: The GBT chip set in the detectors:
GBT13, GBTIA, GBLB & GBT-SCA Optoelectronics from the Versatile Link Project An FPGA in the counting room
Both ends implement a error robust transmission protocol over an optical fibre
The GBT13 communicates with the front-end electronics with up 32(+1) e-links
Two or more e-links can be grouped to match the front-end bandwidth
We propose that the e-links communicate using the Ethernet protocol
An E-link Port Adaptor “macro” will be provided by the GBT team for integration in the front-end ASICS
Slow control is implemented by a dedicated channel One e-link Slow control is managed by the GBT-SCA
Slide from: TWEPP-07 presentation of Paulo Moreira (see this link) Slide from: TWEPP-07 presentation of Paulo Moreira (see this link)
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI 20
Today's SummaryToday's Summary
GBT seems to be a very interesting architecture Some aspects are quite SLHC specific (the 80 Mhz world) Some parameters might not be well adapted to our
needs why is the E-link bandwidth only 80 Mbit/sec ? all the TTC specifics
Goal is to start a discussion on the CBM-XYTER readout architecture many CBM-XYTERS will sit in places with a TID > 1 Mrad the path to the first place with TID < 1 krad, where COTS
components can be placed (e.g. FPGA), is at least 5 m. Is Cu+LVDS still feasible, given total needed bandwidth ? Or is it better to go locally (that means radhard) to optical ?
26 February 2008 11th CBM Collaboration Meeting -- Walter F.J. Müller, GSI 21
The EndThe End
Thanks for Thanks for your attentionyour attention