gbt project: present & future
DESCRIPTION
GBT Project: Present & Future. Paulo Moreira On Behalf of the GBT P roject Collaboration 18 March 2014 CERN, Switzerland. Outline. Radiation Hard Optical Link Architecture The GBT System GBT Chipset: Status Schedule GBT-FPGA Status GBT Building Blocks Status LpGBTX: Architecture - PowerPoint PPT PresentationTRANSCRIPT
GBT Project: Present & Future
Paulo MoreiraOn Behalf of the GBT Project Collaboration
18 March 2014CERN, Switzerland
http://cern.ch/proj-gbt
Outline
• Radiation Hard Optical Link Architecture• The GBT System• GBT Chipset:
– Status– Schedule
• GBT-FPGA Status• GBT Building Blocks Status• LpGBTX:
– Architecture– SerDes (Resources optimization)– Power Consumption– Footprint– Project effort
http://cern.ch/proj-gbt
Radiation Hard Optical Link Architecture
http://cern.ch/proj-gbt
On-DetectorRadiation Hard Electronics
Off-DetectorCommercial Off-The-Shelf (COTS)
GBTX
GBTIA
GBLD
PD
LD
Custom ASICs
Timing & Trigger
DAQ
Slow Control
Timing & Trigger
DAQ
Slow Control
FPGA
GBT GBT
Versatile Link
The GBT System
http://cern.ch/proj-gbt
FEModule
FEModule
Phase – Aligners + Ser/Des for E – Ports
FEModule
E – PortE – Port
E – Port
GBT – SCA
E – Port
Phase - Shifter
E – PortE – Port
E – PortE – Port
CDR
DEC/D
SCR
SER
SCR/ENC
I2C MasterI2C Slave
Control Logic Configuration(e-Fuses + reg-Bank)
Clock[7:0]
CLK Manager
CLK Reference/xPLL
External clock reference
controldata
One 80 Mb/s port
I2CPort
I2C (light)
JTAG
JTAGPort
80, 160 and 320 Mb/s ports
GBTIA
GBLD
GBTXe-Link
clock
data-up
data-down
ePLLTxePLLRx
clocks
GBLD StatusGBLD V4.1• Main Specs
– Bit rate 5 Gb/s (min)– Modulation:
• Current sink• Single-ended/differential
– Laser modulation current: 2 to 24 mA– Laser bias: 2 to 43 mA– Equalization:
• Pre-emphasis/de-emphasis• Independently programmable for rising/falling
edges– Supply voltage: 2.5 V– Die size: 2 mm × 2 mm– I2C programming interface
• Status– Available in small quantities
• Integrated in the VTRx and VTTx– Fully functional– Excellent performance– Radiation hardness proved (total dose) – Heavy Ion and Neutron SEU tests:
• Revealed “some sensitivity” of the configuration registers
• Proton tests shown no upsets• Majority of the errors related with reset function
(not triplicated) can be easily improved.– Technology: 130 nm DM metal stack– Device is production ready
http://cern.ch/proj-gbt
4.8 Gb/s, pre-emphasis on
Total jitter: ≈ 25 ps
GBTIA StatusGBTIA V2.0 / V2.1• Main specs:
– Bit rate 5 Gb/s (min)– Sensitivity: 20 μA P-P (10-12 BER)– Total jitter: < 40 ps P-P– Input overload: 1.6 mA (max)– Dark current: 0 to 1 mA– Supply voltage: 2.5 V– Power consumption: 250 mW– Die size: 0.75 mm × 1.25 mm
• Status:– Fully functional– Integrated in the VTRx– Excellent performance– Radiation hardness proved
• Tested up to 200 Mrad (SiO2)
– Device is production ready• LM metal stack
http://cern.ch/proj-gbt
GBTX Status• Main specs:
– 4.8 Gb/s transceiver– User bandwidth:
• 3.28 Gb/s up/down-links, GBT mode• 3.52 Gb/s up-link, 8B/10B mode• 4.48 Gb/s up-link, wide-bus mode
• Status:– Chip is fully functional– A few modifications will be
introduced in the production version:• XPLL will be adapted for a higher
value of the motional resistance of the quartz crystal
• Tolerance to SEUs of the e-link programing register will be improved
• Startup state machine expanded– Prototypes availability:
• 22 + 32 (drilled) GBTX available• Additional 240 will be available in
August
http://cern.ch/proj-gbt
17 m
m
17 mm
Total height including solder balls: ~3 mm
GBT – SCAGBT Slow Control Adapter:• ASIC dedicated to slow control functions.• System Upgrades for SLHC detectors.• “Replacement” for the CCU & DCU ASICs • Flexible enough to match the needs of different
Front-End systems.Key Features:• Dual redundant e-Ports for GBTX e-links.• 16 I2C master controllers @ 100k/s or
1 Mb/s.• 1 SPI master controller• 1 JTAG master controller• 32 multiplexed ADC channels:
– 12-bit dual slope integrating ADC @ 3.5KHz• 4 DAC channels• 32 Digital I/O lines individually programmable.• 8-bit memory bus• 4 Interrupt inputs• Technology: CMOS 130nm using radiation
tolerant techniques.Status:• Chip ready for prototyping:
– Tapout May 2014http://cern.ch/proj-gbt 11.6 mm
Total height including solder balls: ~3 mm
BOTTOM WIEW
ADC_in_pad<0>
ADC_in_pad<2>
ADC_in_pad<5>
ADC_in_pad<8>
ADC_in_pad
<11>
ADC_in_pad
<14>
ADC_in_pad
<17>
ADC_in_pad
<20>
ADC_in_pad
<23>
ADC_in_pad
<26>Pad_
resOutDAC_ou
t_pad<2>
DAC_out_pad<3>
JTAG_reset_pad
ADC_in_pad<1>
ADC_in_pad<1>
ADC_in_pad<4>
ADC_in_pad<7>
ADC_in_pad
<10>
ADC_in_pad
<13>
ADC_in_pad
<16>
ADC_in_pad
<19>
ADC_in_pad
<22>
ADC_in_pad
<25>
DAC_out_pad<28>
DAC_out_pad<0>
DAC_out_pad<1>
TDO_pad
RESET_B_pad
FuseProgram
Pulse_pad
ADC_in_pad<3>
ADC_in_pad<6>
ADC_in_pad<9>
ADC_in_pad
<12>
ADC_in_pad
<15>
ADC_in_pad
<18>
ADC_in_pad
<21>
ADC_in_pad
<24>
ADC_in_pad
<27>
ADC_in_pad
<29>
ADC_in_pad
<30>TDI
_pad
rx_sdRx_sd_nAGNDAGNDAGNDAGNDAVDDAVDDAVDDAVDD
SCL_pad<14>
SCL_pad<15>
TCK_pad
link_clklink_clk_n
SPI_ss_pad<0>
AGNDAGNDAGNDAGNDAVDDAVDDAVDDAVDDSDA_pad<14>
SDA_pad<15>
TMS_pad
tx_sdtx_sd_n
SPI_ss_pad<1>
DVSSDVSSAGNDAGNDAVDDAVDDDVDDSCL
_pad<13>
SCL_pad<12>
SCL_pad<11>
SCL_pad<10>
rx_sd_aux
Rx_sdaux_n
SPI_ss_pad<2>
DVSSDVSSDVSSDVSSDVDDDVDDDVDDSDA_pad<13>
SDA_pad<12>
SDA_pad<1>
SDA_pad<10>
link_clk_aux
link_clk_aux_n
SPI_ss_pad<3>
DVSSDVSSDVSSDVSSDVDDDVDDDVDDSCL
_pad<9>
SCL_pad<8>
SCL_pad<7>
SCL_pad<6>
tx_sd_aux
tx_sdaux_n
SPI_ss_pad<4>
GNDGNDGNDGNDVDDVDDVDDSDA_pad<9>
SDA_pad<8>
SDA_pad<7>
SDA_pad<6>
link_aux_disable
_pad
SPI_miso_pad
SPI_ss_pad<5>
GNDGNDGNDGNDVDDVDDVDDSCL
_pad<5>
SCL_pad<4>
SCL_pad<3>
SCL_pad<2>
auxPortTestEn_pad
SPI_mosi_pad
SPI_ss_pad<6>
GNDGNDGNDGNDVDDVDDVDDSDA_pad<5>
SDA_pad<4>
SDA_pad<3>
SDA_pad<2>
auxPortSCL
_pad
SPI_clk
_pad
SPI_ss_pad<7>
GPIO_pad<4>
GPIO_pad<7>
GPIO_pad
<10>
GPIO_pad
<13>
GPIO_pad
<16>
GPIO_pad
<19>
GPIO_pad
<22>
GPIO_pad
<25>
GPIO_pad
<28>
SCL_pad<1>
SCL_pad<0>
auxPortSDA_pad
GPIO_pad<0>
GPIO_pad<2>
GPIO_pad<5>
GPIO_pad<8>
GPIO_pad
<11>
GPIO_pad
<14>
GPIO_pad
<17>
GPIO_pad
<20>
GPIO_pad
<23>
GPIO_pad
<26>
GPIO_pad
<29>
SDA_pad<1>
SDA_pad<0>
pad_GPIO_EXTCLK
GPIO_pad<1>
GPIO_pad<3>
GPIO_pad<6>
GPIO_pad<9>
GPIO_pad
<12>
GPIO_pad
<15>
GPIO_pad
<18>
GPIO_pad
<21>
GPIO_pad
<24>
GPIO_pad
<27>
GPIO_pad
<30>
GPIO_pad
<31>
N
M
L
K
J
I
H
G
F
E
D
C
B
A
N
M
L
K
J
I
H
G
F
E
D
C
B
A
1413121110987654321
1413121110987654321
pwr3_3pad
11.6
mm
GBT Chipset Schedule“Scenario 1” – MPW followed by production• 19 May 2014 – MOSIS MPW
• GBTX• GBT - SCA• GBLD
• Q3 – 2014– Chips available from the foundry– ASIC Packaging
• Q4 – 2014– Prototypes medium quantities: ~200– Samples distributed to the users
• Q1 – 2015:– Launch the production
• Q3 – 2015:– Production quantities available
“Scenario 2” – Production only• Q2 – 2014 (May)
– Split Engineering Run to produce in quantities:
• GBTX• GBTIA• GBLD V4• GBT - SCA
• Q3 – 2014– Chips available from the foundry– ASIC Packaging
• Q4 – 2014– ASIC production testing– First production ASICs distributed to the
users
http://cern.ch/proj-gbt
GBT – FPGA Status
http://cern.ch/proj-gbt
• Aim:– Implement the GBT serial link in all its
flavours as an IP core for most of the current FPGAs used on Back-End boards for upgrades
• Firmware versions– Serial link encoding schemes
• Reed-Solomon (aka “GBT frame” ),• 8b/10b (to be done)• Wide-bus
– Standard and Fixed latency versions• Targeted FPGA and reference design
– Altera:• Cyclone V GT
– Eval kit & GBTx-SAT board• Stratix V
– AMC 40
– Xilinx:• Virtex 6
– ML605 & GLIB• Kintex 7
– KC705• Virtex 7
– VC705
• Project Resources– 50% of one Fellow since last summer– More than 85 users registered– A sharepoint site: https://
espace.cern.ch/GBT-Project/GBT-FPGA/default.aspx
• A SVN repository:– https://
svnweb.cern.ch/cern/wsvn/ph-ese/be/gbt_fpga
• Contact us:– [email protected]– [email protected]
See poster by Manoel Barros forfurther information on the GBT – FPGA
MAJOR RELEASE 11 APRIL 2014
GBT Building Blocks (IP) StatusAvailable “IP” to facilitate the implementation ofe-Link transceivers in the frontend ASICs:• SLVS Receiver
– Wire-bond, DM metal stack– C4, LM metal stack
• SLVS Driver– Wire-bond, DM metal stack– C4, LM metal stack
• SLVS Bi-directional– C4, LM metal stack
• HDLC transceiver– Synthesizable Verilog
• 7B/8B CODEC– Synthesizable Verilog
• ePLL-FM– Frequency Multiplier PLL– Radiation Hard– 130 nm CMOS technology with the DM metal stack
(3-2-3).– Input frequencies: 40/80/160 MHz– Output frequencies: 160/320 MHz regardless the
input frequency– Programmable phase of the output clocks with a
resolution of 11.25° for the 160 MHz clock and 22.5° for the 320 MHz clock
– Programmable charge pump current, loop filter resistance and capacitance to optimize the loop dynamics
– Supply voltage: 1.2 V - 1.5 V– Nominal power consumption: 20 mW @ 1.2 V - 30
mW @1.5 V– Operating temperature range: -30°C to 100°C
• ePLL-CDR (currently under testing)– Data rate: 40/80/160/320 Mbit/s– Output clocks: data clock + 40/80/160/320 MHz
with programmable phase– Internal or external calibration of the VCO frequency– Possibility to use it as a frequency multiplier PLL
without applying input data– Programmable charge pump current, loop filter
resistance and capacitance to optimize the loop dynamics
– Supply voltage: 1.2 V - 1.5 V– Operating temperature range: -30°C to 100°C– Prototype fabrication: May 2013
http://cern.ch/proj-gbt
ePLL- FM
The LpGBTX
• Low Power Dissipation and Small Footprint:– Critical for pixel detectors– Critical for tracker/triggering detectors
• Bandwidth:– Low-Power mode
• 4.8 Gb/s for Up and Down links– High-Speed mode:
• 9.8 Gb/s for the Up-link• 4.8 Gb/s for the Down-link
• e-Links:– 80, 160 and 320 Mb/s for down-links– Low-Power Mode:
• 80, 160 and 320 Mb/s for up-links– High-Speed Mode:
• 160, 320 and 640 Mb/s for up-links– Programmable: Single-ended / differential
• Functionality:– “Replica” of the GBTX– Small subset of the GBT-SAC functionality will be included
http://cern.ch/proj-gbt
LpGBTX Architecture
http://cern.ch/proj-gbt
SerDes
4.8 Gb/s
4.8 or 9.6 Gb/s
refClk40MHz
cdrOut[119:0]
serIn[239:0]
DEC&
DSCR
rxData[79:0]
SCR&
ENC
rxEcData[1:0]ePortTx
eLinkOut[39:0]
ecOut
ePortRx
eLinkIn[39:0]
eLinkIn[57:40]
ecIn[1]
40/80/160/320/640 MHz 40 MHz
40 MHz
160/320 MHz
160/320/640 MHz
• Downlink: 4.8 Gb/s– GBT mode
• Uplink: 4.8/9.6 Gb/s– GBT / Wide-Bus / 8B/10B
• EC link: 80 Mb/s• SCA functionality added• Output e-Links:
– 80/160/320 Mb/s 40 e-Links (max)
• Phase programmable clocks:– 40/80/160/320/640 MHz 8 Clocks (max)
• Output e-Clocks:– 40/80/160/320 Mb/s 40 e-Clocks (max)
• Input-Links:– 80/160/320 Mb/s @ 4.8 Gb/s– 160/320/640 Mb/s @ 9.6 Gb/s
– GBT mode 40 e-Links (max)– Wide-bus mode 56 e-Links (max)
160/320/640 MHz
“Idle & data headers”could be used to distinguishbetween the “odd and even frames”when operating at 9.6 Gb/s!
SCA(Reduced set)
txData[159:0]
txEcData[3:0]
txData[223:160]
PhaseShifter
eClock39:0]
Newcomer!
SerDes Optimization
• 1 high frequency PLL drives the CDR and SER:– Half Rate CDR (same architecture as GBTX)– Serializer:
• Full rate @ 4.8 Gb/s• Half rate @ 9.6 Gb/s
• PLL can work as:– CDR– Frequency Multiplier– This enables: TX, RX and TRANS
Clock Divider• For the e-Links the clock frequencies are:
– 40/80/160/320/640 MHz
• Half rate CDR requires:– 2.4 GHz– I and Q phases– Differential for CML
• Serializer: Half rate 9.8 Gbps / Full rate 4.8 Gbps– 4.8 GHz
http://cern.ch/proj-gbt
CDR/FMPLL
De-Serializer4.8 Gb/s
4.8 or 9.6 Gb/s
refClk40MHz
Oscillator
1/N
Serializer
cdrOut[119:0]
serIn[239:0]
40/80/160/320/640 MHz
Aided Lock Control
For efficiency the SER and DES have to
be co-designed
LpGBTX: Power (1/2)
LpGBTX power estimate
• Assumes:– 65 nm CMOS technology– Supply voltage: 1.2V– Merged SerDes architecture– 4.8 Gb/s on both up and down
links– e-Links with 200 mV signaling– “SCA” functionality not taken
into account in this preliminary study
http://cern.ch/proj-gbt
I/OSERDESCOREPhase-ShifterClock Manager
Power [mW] FractionI/O 548 54%SERDES 157 15%CORE 121 12%Phase-Shifter 120 12%Clock Manager 73 7%Total (max) 1019
Preliminary&
Non-Binding!
LpGBTX: Power (2/2)Case study:• Hypothetical configuration:
– E-Links:• 20 × Data-In @ 160 Mb/s• 1 × Clock @ 160 MHz• 2 × Data-Out @ 160 Mb/s
– Phase-Adjustable clocks:• 2 × Clock @ 40 MHz• 2 × Clock @ 160 MHz
http://cern.ch/proj-gbt
Preliminary&
Non-Binding!
Mode: TRANSCEIVERE-Links: Data rate [Mb/s]: 160 # Data outputs: 2 # Clock outputs: 1 # Data inputs: 20EC-Channel: State: DisabledPhase-Shifter: State: Enabled # Channels 4Circuit: Power [mW] SERDES: 157 Clock Manager E-PLL: 33 XPLL: 40 Total: 73 Phase-Shifter: PLL: 17 Channel: 26 SLVS-TX: 26 Total: 69 I/O: Data SLVS-Tx: 13 CLK SLVS-Tx: 7 Data SLVS-Rx: 5 Total: 24 CORE: Standard Cells: 110 Phase-Aligners: 6 Total: 115 Total Power [mW]: 439
LpGBTX: Footprint
http://cern.ch/proj-gbt
General PurposeLpGBTX with programmable I/O
(Single-ended /Differential)
Tracker Specific Development(Single-ended I/O only)
Two ASICs:• LpGBT-SerDes (tracker specific)!• LpGBTX (general purpose)Although many functions are the same in thetwo ASICs, these are effectively two projects!• Almost doubles the budget and the
manpower needs!
I/O set to differentialI/O se
t to sin
gle-ended
• Pin-count: ≈ 500• BGA pitch: 0.65 mm• Package Size: ≈ 16 mm × 16 mm
• Pin-count: ≈ 150 (to be confirmed)• BGA pitch: 0.65 mm• Package Size: ≈ 10 mm × 10 mm
Preliminary&
Non-Binding!
LpGBTX: 2014 – and beyond, Activities / Manpower / Budget2014• Q3: First draft of LpGBTX specifications• Q3: Discussions with the Experiments• Q4: LpGBTX final specifications2015 and beyond• LpGBTIA
– (The GBTIA is already relatively low power. Perhaps the LpGBTIA is not a strictly necessary development…)– Technology: 65 nm CMOS– Manpower: 2 MY (design and testing)
• LpGBLD10– (Some work already going on)– Technology: 130 nm CMOS– Manpower: 2 MY (design , packaging and testing)
• LpGBTX– Technology: 65 nm CMOS– Two packaging flavours:
• “Tracker” & “General Purpose”– Manpower:
• Design: 8 MY• Packaging: 1 MY• Testing: 2 MY
– We have to seriously consider “building” a stable LpGBT team if a LpGBT chipset is to be a reality in useful time!
– The move to 65 nm and Low Power is not just a “copy-paste exercise”!!!• The LpGBT has not yet been defined as an approved “project” or “common project”:
– Effort going on to secure budget and manpower for the project!http://cern.ch/proj-gbt
Total manpower: 15 MY
[email protected] 19http://cern.ch/proj-gbt
Extra Slides
M. Barros Marin, P. Leitao, S. Baron 20
E-LINK SERDES
GBT-FPGA TX MGT TX
GBT-FPGA RX
MGT RX
SFP
GBTxGBT-FPGA
SFP
GBTx SAT board
GLIB
GBT SERDESE-LINK SERDES
CDR
CLOCK
CDR
BACK END FRONT END
FE EMULATOR
Cyclone V
PLL
DOWNLINK
UPLINK
EportRx: phase-aligner (set to maximum and minimum; FE
position dependant)EportTx: no phase align necessary
on GBTx
Flags are latched with e-link’s clock and triggered by
the e-links dataGBT-FPGA uplink and downlink flag latched with
40 MHz clock
18/03/2014
21
E-LINK SERDES
GBT-FPGA TX MGT TX
GBT-FPGA RX
MGT RX
SFP
GBTxGBT-FPGA
SFP
GBTx SAT board
GLIB
GBT SERDESE-LINK SERDES
CDR
CLOCK
CDR
BACK END FRONT END
FE EMULATOR
Cyclone V
PLL
4 ns5 ns1 ns
DOWNLINK
UPLINK
M. Barros Marin, P. Leitao, S. Baron
51.2 ns 30.6 ns
29 ns 21 ns
18/03/2014
22
UPSTREAM LATENCYUP
Encoding
E-link speed All All - passive All - passive - GBT_FPGA*
GBT Frame
80MHz224 ns 214 ns
~9 BC163 ns
~7 BC237 ns 227 ns 176 ns
160MHz190 ns 180 ns
~8 BC129 ns
~5 BC202 ns 192 ns 141 ns
Wide Bus
Frame
80MHz224 ns 214 ns
~9 BC163 ns
~7 BC237 ns 227 ns 176 ns
160MHz190 ns 180 ns
~8 BC129 ns
~5 BC202 ns 192 ns 141 ns
M. Barros Marin, P. Leitao, S. Baron18/03/2014
Remaining: MGT, SFPs, GBTx
23
DOWNSTREAM LATENCYDOWN
Encoding
E-link speed All All - passive All – passive - GBT_FPGA*
GBT Frame
80MHz 174 ns 164 ns ~7 BC 135 ns ~5 BC
160MHz 173 ns 163 ns ~7 BC 134 ns ~5 BC
M. Barros Marin, P. Leitao, S. Baron18/03/2014
Remaining: MGT, SFPs, GBTx