gbt and versatile link status - cern

GBT and Versatile Link Status ATLAS Upgrade Week 16.11.11

Francois Vasey & Paulo Moreira

On-Detector Custom Electronics & Packaging

Radiation Hard

Off-Detector Commercial Off-The-Shelf (COTS)

Custom Protocol

1. Versatile Link

16 Nov 2011 [email protected] 2


Radiation Hard


Custom Protocol

1.1 Versatile Link Versatility Front-End VTRx Fibre Back-End TRx

EE laser, 1310nm

SM LR-SFP+ TRx

VCSEL, 1310nm SNAP12’like Rx

InGaAs PIN, 1310nm

Opto Engine Rx

VCSEL, 850nm

MM

SR-SFP+ TRx

GaAs PIN, 850nm SNAP12’like Tx,Rx, TRx

InGaAs PIN, 850nm

Opto Engine Tx, Rx, TRx


1.2 Versatile Link System Specification

Four Versatile Link flavours defined

Power budget specified with margin for all flavours Jitter budget adapted from FC 4G standard Specs available as EDMS document CERN-0000090391


Fibre Type

Radiation Tolerance Grade Calorimeter-grade Tracker-grade

10 kGy, 5x1014 n/cm2 500 kGy, 2x1015 n/cm2,1x1015 h/cm2

SM

MM

Optical connector interface designed and fabricated using rapid 3D prototyping Different materials Different designs


1.3 Versatile Transceiver (VTRx)

Several VTRx prototype versions built and tested SM, MM GBLD, TI-ONET LDD

Specifications established

J. Troska, V. Bobillier, C. Soos et al.

1.4 SEU Test of VTRx prototype

VTx


VRx

J. Troska et al.

SM

MM

1.5 Radiation-Induced Attenuation (RIA) in cold fibers

Optical Fibres will operate well under radiation in the cold


Dose Rate: 0.66 kGy(Si)/hr

Temperature:

-25 degC

A fibre is qualified if its total RIA is less than 1 dB

D. Hall et al

SM SM

MM MM

1.6 VL project status, Sep 2011


Front-End VTRx Fibre Back-End TRx

EE laser, 1310nm

GBLD Drive current OK

SM LR-SFP+ TRx

VCSEL, 1310nm GBLD compliance Voltage OK

Board-edge Rx SM Tx not (yet) available

InGaAs PIN, 1310nm

Opto Engine Rx SM Tx not (yet) available

VCSEL, 850nm

GBLD compliance Voltage OK

MM RIA in cold at low dose rate OK

SR-SFP+ TRx

Tx OMA to meet Tk power budget

GaAs PIN, 850nm Responsivity drop: Calo grade OK

Board-edge Tx,Rx, TRx Tx OMA to meet Tk power budget

InGaAs PIN, 850nm Assembly to be confirmed: Tk grade

Opto Engine Tx, Rx, TRx Tx OMA to meet Tk power budget

TK grade

Calo & Tk

grade

1.7 Towards Versatile Link Project Phase III


Common R&D

Generic Tendering 2015 (depending on users commitment)

Detector specific R&D 2020

VL Phase II

VL Phase III 5G low power VTRx for Tk application Low power VCSEL driver,

small footprint package

10G opto engine VnTx for Calo application Driver array, VCSEL

array, package

TOSA (MM) ROSA (GaAs 850) Fibre (MM) TRx (MM-calo) Opto-engine (MM-calo)

2. GBT



Radiation Hard


Custom Protocol

2. GBT System


FE Module

FE Module

Phase – Aligners + Ser/Des for E – Ports

FE Module

E – Port E – Port

E – Port

GBT – SCA

E – Port

Phase - Shifter



CDR

DEC/DSCR

SER

SCR/ENC

I2C Master I2C Slave

Control Logic Configuration (e-Fuses + reg-Bank)

Clock[7:0]

CLK Manager

CLK Reference/xPLL

External clock reference

clocks control data

One 80 Mb/s port

I2C Port

I2C (light)

JTAG

JTAG Boundary Scan

80, 160 and 320 Mb/s ports

GBTIA

GBLD

GBTX e-Link

clock

data-up

data-down

ePLLTx ePLLRx

P. Moreira et al.

Versatile Link Transceiver

2.1 GBT Project Status: GBT-SerDes


Serializer • The serializer if fully functional and fully

complies with the specifications: Performance: • Data transmission:

• No errors observed • Jitter:

• Total jitter (1e-12): 53 ps • Random jitter: 2.4 ps (rms) • Deterministic jitter: 19 ps

De-Serializer • The receiver is fully functional but only up to ~3

Gb/s! Performance: • Data reception: Error free up to 3 Gb/s • Clock recovery operates up to 6 Gb/s • Jitter:

• Recovered 40 MHz clock PRBS @ 4.8 Gb/s: • Total jitter (1e-12): 63 ps • Random jitter: 4.9 ps (rms) • Deterministic jitter: 24 ps (pp)

P. Moreira et al.

2.2 GBT Project Status: GBTX

Problem with the GBT-SerDes de-serializer circuit understood: New architecture chosen and implemented for the GBTX serializer Extensive simulation and verification currently being done.

ASIC design on its way to completion Test setup being developed in parallel with the ASIC design Target “tape out”

MOSIS MPW 6 February 2012 MOSIS MPW 7 May 2012


2.3 GBTX Functionality Summary

4.8 Gb/s bidirectional transceiver: Downlink:

GBT mode (FEC) – 3.28 Gb/s Uplink:

GBT mode (FEC) – 3.28 Gb/s 8B/10B – 3.52 Gb/s WideBus – 4.56 Gb/s

VCXO based PLL: Clock reference Jitter Filter

8 Phase and frequency programmable clocks Phase resolution: 50 ps Frequencies: 40/80/160/320 MHz

E-Links: Up to 40 bidirectional:

Programmable data rate: 80/160/320 +1 bidirectional:

Fixed data rate: 80 Mb/s Configuration:

IC channel I2C slave Configuration memory e-Fuse register bank

GBLD interface: I2C master

Testability: JTAG Boundary scan Built-in BERT Data loopbacks

Package: 400-pin BGA package

16 Nov 2011 [email protected] 14 P. Moreira et al.

FE Module

FE Module

Phase – Aligners + Ser/Des for E – Ports

FE Module


E – Port

GBT – SCA

E – Port

Phase - Shifter



CDR

DEC/DSCR

SER

SCR/ENC

I2C Master I2C Slave

Control Logic Configuration

(e-Fuses + reg-Bank)

Clock[7:0]

CLK Manager

CLK Reference/xPLL

External clock reference

clocks control data

One 80 Mb/s port

I2C Port

I2C (light)

JTAG

JTAG Boundary Scan

80, 160 and 320 Mb/s ports

GBTIA

GBLD

GBTX e-Link

clock data-up

data-down

ePLLTx

ePLLRx

Versatile Link Transceiver

2.4 e-Link Modes


SEU tolerant

• GBTX – to – Frontend interface: • Electrical links (e-link) • Bidirectional

• e-Link: • Three 100 Ω pairs: • DOUT: GBTX –to – Frontend • DIN: Frontend – to – GBTX • CLK: GBTX –to – Frontend

• Programmable data rate: • Independently for up/down links

• Independently per group (of up to 8 links each)

• 80/160/320 Mb/s • e-Port count:

• GBT Mode: • Uplink: 40+1 (max)

• Downlink: 40+1 (max) • 8B/10 Mode:

• Uplink: 44 (max) • Downlink: 36 (max)

• WideBus Mode: • Uplink: 56 + 1 (max)

• Downlink: 24 + 1 (max) • EC channel:

• Available in the GBT and WideBus modes • Fixed data rate: 80 Mb/s

• Compatible with GBT – SCA • General purpose data transmission

• Electrical standard: • SLVS electrical levels:

• 100 Ω termination • 400 mV differential • 200 mV common mode • ILOAD = ±2 mA

• e-Link driver: • Programmable current drive

• Power Off function • e-Link receiver:

• SLVS and LVDS compliant • Internal termination (on/off)

• Power Off function

Mode Type Data Rate e-Link count OFF Power off - 2 × serial 80 Mb/s Up to 40/44/56* 4 × serial 160 Mb/s Up to 20/22/28* 8 × serial 320 Mb/s Up to 10/11/14* 8 × lanes > 320 Mb/s See “Lanes”

JEDEC standard, JESD8-13 Scalable Low-Voltage Signalling for 400 mV (SLVS-400) http://www.jedec.org/download/search/JESD8-13.pdf

(*) E-Link counts correspond to the uplink in the GBT, 8B/10B and WideBus mode respectively

P. Moreira et al.

http://www.jedec.org/download/search/JESD8-13.pdf

3. GLIB



Radiation Hard


Custom Protocol

3.1 GLIB typical use case

BENCH-TOP: beam test setup

= SFP+ = TTC FMC 17

Optical Control/Readout of FE modules w/ GBT (through SFP+)

Optical TTC reception (through TTC FMC)

PC w/ 1Gb Ethernet + Power supply

P. Vichoudis et al. 16 Nov 2011 [email protected]

3.2 GLIB Hardware

18

GLIB V.1 (PROTOTYPE) – top view


3.3 GLIB demonstration

19

MGT Tx

GBT Tx

MGT Rx

GBT Rx

MGT Tx

GBT Tx

MGT Rx

GBT Rx

CLK SYNTH & JITTER

CLEANER

MGT REFCLK RX RECCLK

OSCILLATOR

CLK SYNTH & JITTER

CLEANER

MGT REFCLK

CLK GENERATOR

SELECT INCLK

“FRONT-END” GLIB “BACK-END” GLIB


2pcs of GLIB v1 manufactured in Q1 2011, most of functionality ok, only few issues found.

All issues understood and solved in GLIB v2. 2pcs manufactured in Q3 2011. No issues found

yet.

Preproduction of another 4pcs of GLIB v2 ongoing. Delivery to beta users in Mid November

2011

FPGA with higher speed grade will be used (for 6.5Gbps transceivers) Associated firmware, software and TTC FMC mezzanines will also be delivered.

Production version expected to be available by July 2012.

Need to define the quantity to order. If interested, please contact us by April 2012.

xTCA firmware development will start in January 2012 (xTCA infrastructure available)

Software development ongoing

Auxiliary cards under development first prototypes by Q1 2012.

TTC FMC prototype already available (2pcs + 2pcs by mid November 2011) Versatile Link FMC, PCI Express adapter, Digital IO FMC prototypes available by Q1 2012

20

3.4 GLIB status

16 Nov 2011 [email protected]

Backups


Downstream Versatile Link Power Budget

Tracker-grade back-end transmit power must be boosted to compensate radiation penalties

Calorimeter-grade power budget compatible with COTS component specs

29 Sep 2011 [email protected] on behalf of Versatile Link team 22

VTRx Vertical opto block

29 Sep 2011 [email protected] on behalf of Versatile Link team 23

GBT Frame (up and down links)


One e-Port: • Fixed data rate 80 Mb/s

3 groups of 8 output e-Ports 2 groups of 8 input/output e-Ports (operating as outputs)

5 groups of 8 input e-Ports. Number of data e-links (excluding EC e-Link):

• 40 input (max @ 80 Mb/s) • 40 output (max @ 80 Mb/s)

80 bits

H(3:0) IC(1:0) EC(1:0) D(63:48) D(47:32) D(31:16) D(15:0) FEC(31:16) FEC(15:0) D(79:64)

G0 G1 G2 G3 G4

GBT Frame

2 bits 2 bits 32 bits

4 bits

120 bits

EC

8B/10B Mode (uplink only)


8B/10B 8B/10B 8B/10B 8B/10B 8B/10B 8B/10B 8B/10B 8B/10B 8B/10B 8B/10B 8B/10B 8B/10B

5 groups of 8 E-ports 1 group of 4 E-ports

G5 (1/2) G6 (not used) G0 G1 G2 G3 G4

8B/10B Frame

3 groups of 8 output e-Ports 1 group of 8 input/output e-Ports (operating as outputs)

1/2 group of 8 input/output e-Ports (operating as 4 outputs) 5 groups of 8 input e-Ports.

1/2 group of 8 input/output e-Ports (operating as 4 inputs) Number of data e-links (excluding EC e-Link):

• 44 input (max @ 80 Mb/s) • 36 output (max @ 80 Mb/s)

WideBus Mode (uplink only)


One e-Port: • Fixed data rate 80 Mb/s 3 groups of 8 output e-Ports

2 groups of 8 input/output e-Ports (operating as inputs) 5 groups of 8 input e-Ports

Number of data e-links (excluding EC e-Link): • 56 input (max @ 80 Mb/s)

• 24 output (max @ 80 Mb/s)

112 bits

H(3:0) IC(1:0) EC(1:0) D(63:48) D(47:32) D(31:16) D(15:0) D(111:96) D(95:80) D(79:64)

“Out of order” bit sequence for compatibility with the GBT frame

G5 G6 G0 G1 G2 G3 G4

Wide Bus Frame

EC

2 bits 2 bits

120 bits

e-Ports to Frame Mapping (1/3)


I/O Data Rate Group Frame Frame Group

I/O Data Rate x2 x4 x8 x2 x4 x8

dIn[7:0] 0 FRMUP[47:32] FRMDWN[47:32] 0

dOut[7:0] dIn[6,4,2,0] dOut[6,4,2,0] dIn[4,0] dOut[4,0]

dIn[15:8] 1 FRMUP[63:48] FRMDWN[63:48] 1


dIn[23:16] 2 FRMUP[79:64] FRMDWN[79:64] 2


dIn[31:24] 3 FRMUP[95:80] FRMDWN[95:80] 3


dIn[39:32] 4 FRMUP[111:96] FRMDWN[111:96] 4


Up to 40 output E-Links available Up to 40 input E-Links available

Shift order for the E-Link data inputs and outputs is MSB first



4 E-Link “outputs” become inputs

Up to 36 output E-Links available

Up to 44 input E-Links available




dIn[7:0]

0 FRMUP[19:0]

FRMDWN[47:40] (FRMDWN[39:32] not

available)

0

dOut[7:4] dIn[6,4,2,0] dOut[6,4]

dIn[4,0] dOut[4] dIn[15:8]

1 FRMUP[39:20] FRMDWN[63:48] 1 dOut[15:8]

dIn[14,12,10,8] dOut[14,12,10,8] dIn[12,8] dOut[12,8]

dIn[23:16] 2 FRMUP[59:40] FRMDWN[79:64] 2


dIn[31:24] 3 FRMUP[79:60] FRMDWN[95:80] 3


dIn[39:32] 4 FRMUP[99:80] FRMDWN[111:96] 4


dOut3[3:0] 5 FRMUP[109:100] dOut[2,0]

dOut[0]



16 E-Link “outputs” become inputs

Up to 24output E-Links available

Up to 56 input E-Links available




dIn[7:0] 0 FRMUP[47:32] FRMDWN[47:32] 0

n.a. dIn[6,4,2,0] n.a. dIn[4,0] n.a.

dIn[15:8] 1 FRMUP[63:48] FRMDWN[63:48] 1

n.a. dIn[14,12,10,8] n.a. dIn[12,8] n.a.

dIn[23:16] 2 FRMUP[79:64] FRMDWN[79:64] 2


dIn[31:24] 3 FRMUP[95:80] FRMDWN[95:80] 3


dIn[39:32] 4 FRMUP[111:96] FRMDWN[111:96] 4


dOut[7:0] 5 FRMUP[15:0] dOut[6,4,2,0]

dOut[4,0] dOut[15:8]

6 FRMUP[31:16] dOut[14,12,10,8] dOut[12,8]

e-Port Count Summary


Mode GBT WIDE 8B/10B

# Tx groups 5 + 1/8 3 + 1/8 4 + 1/2

# Rx groups 5 + 1/8 7 + 1/8 5 + 1/2

N = 1 e-Port group = 8 e-Ports

1/2 = ½ e-Port group = 4 e-Ports

1/8 = Slow control port = 1/8 e-Port group = 1 e-Port

gbt and versatile link status - cern

Documents