Beam InstrumentationTechnical Board(03/12/2015)

The GBT-based Expandable Front-End (GEFE)

Manoel Barros Marin (on behalf of the GEFE team)

BE-BI-QP

Outline:• Introduction• First contact with the GEFE board• Radiation campaign at PSI• Status & Outlook

The GBT-based Expandable Front-End (GEFE)

BE-BI-QP

• Introduction

BE-BI-QP

The GBT-based Expandable Front-End (GEFE)

4

IntroductionWhat is the GBT-based Expandable Front-End (GEFE) BE-BI-QP

• General purpose FPGA-based radiation tolerant board

• Features different components of the GBT-Versatile Link ecosystem from CERN PH-ESE

• FPGA Mezzanine Card (FMC) High-Pin Count (HPC) connector

• Clocks

• Upgradable:

• 2x GPIO connectors (24 & 13 user I/Os)

• Flexible schemes for:

• Resets

• FPGA programming

• Power• Slow Control (SC) E-link

Target Total Ionizing Dose (TID): up to 75 krad

• Optical & Electrical interfaces

• Rad-Hard FPGA ProAsic3 (ACLA3PE3000-FGG896) from Microsemi

Front-EndCards

Back-EndSystem

Data Acquisition(DAQ)

Radiation

Timing/Trigger/Ctrl (TTC/BST)

• First contact with the GEFE board

BE-BI-QP

The GBT-based Expandable Front-End (GEFE)

6

First contact with the GEFE boardBE-BI-QPThe GEFE at a glance

(1 of 7]

200 mm

100 mm

Block diagram

7

BE-BI-QP

FMC HPC

ProAsic3FPGA

GBTx

VTRx

Rad-HardPowering

Main componentsFirst contact with the GEFE board (2 of 7)

Block diagram

8

BE-BI-QP

Low-SpeedElectrical Link

TX

Secondary components

Block diagram

GPIO ConnectorsA & B

RX

First contact with the GEFE board (3 of 7)

9

BE-BI-QP

LEDs(x6)

GPIO/CLKMMCX

Board IDConnector

DIP Switch& Push Button

JTAG

GPIOLemo

Auxiliary components

GEFE Config. IDResistors Block diagram

First contact with the GEFE board (4 of 7)

10

BE-BI-QP

First contact with the GEFE board (5 of 7)

Test setupBack-End board (GLIB)

FPGA

GBT-FPGA

Patt.Gen.

Patt.Check.

VTRx

GBTx

FPGA

FMC HPC

On-b

oard

Co

nnec

tors

11

BE-BI-QPFPGA FirmwareFirst contact with the GEFE board (6 of 7)

HDL (Verilog) example

GEFE

ProAsic3 FPGA

TopSystem Core User Logic

Test Routines

GBTx

VTRx

FMC HPC

GPIO Connectors A & BTX

RX

12

BE-BI-QPFirst test resultsFirst contact with the GEFE board (7 of 7)

• Rad-Hard Powering (1.5, 2.5 and 3.3 V) presents very low ripple (< 10 mVrms @ ICs pins)• Main components:

• High-Speed Optical Link (VTRx + GBTx) @ 4.8 Gbps fully operational

• GBtx from/to ProAsic3 communication (E-Links) operational with comments:

o Downstream E-Links (GBtx-to-ProAsic3) fully operationalo Upstream E-Links (ProAsic3-to-GBTx) operational but high GBTx temperature (~70 °C)

Missing resistor network for ProAsic3 LVDS drivers (GBTx receives 2.5 V of common signal instead of 1.2 V)

• FMC HPC user I/O and DP lines successfully tested @ 160 Mbps using CMOS 2.5 V levels

(Resistor network for ProAsic3 LVDS driver not presented on GEFE (shall be featured on FMC))

• Form factor:

• PCB too thick for being inserted in crate (2.2 mm instead of 1.8 mm) (to be corrected in v2)

• Secondary components:

• Low-Speed Electrical Link not tested yet • GPIO Connectors successfully tested @ 160 Mbps

• Auxiliary components (e.g. LEDs, Crystal Oscillator, etc.) successfully tested

• Some special functions (e.g. FMC JTAG, FMC I2C, etc.) not tested yet

• Radiation campaign at PSI

BE-BI-QP

The GBT-based Expandable Front-End (GEFE)

14

Radiation campaign at PSIBE-BI-QP

(1 of 2)

• Carried out in October 2016 at the Paul Scherrer Institut (PSI)

The radiation campaign

• In beamline of COMET cyclotron:• Proton beam

• 250 MeV

• Collaboration between CERN BE-BI & RadWG

• Active components tested for the GEFE project:• Dual-bit transceiver (SN74LVC2T45DCTT) (test passed)

• Dual diode (BA99) (test passed)

• Crystal oscillator (25 MHz) (SPXO018077) (test passed)

COMET cyclotron

• Radiation levels:• Total Ionizing Dose (TID) = 75 krad

• Fluence = 9.31*10^11 p/cm2

15

Radiation campaign at PSIBE-BI-QP

Qualified up to 75 kradQualified bellow 75 krad

Not Qualified

(2 of 2)

Radiation tolerance of GEFE’s active components

• Status & Outlook

BE-BI-QP

The GBT-based Expandable Front-End (GEFE)

17

BE-BI-QPStatus

Status & Outlook (1 of 2)

17

BE-BI-QP

• GEFE v1 almost fully tested (Main features operational)• The GEFE board

• The GEFE community

• Low-Speed Electrical Link and some special functions to be tested• Minor modifications required for pre-production prototype (GEFE v2) • Radiation hardness qualification required for two active components & full GEFE board

• Specification in version 0.8.5 (may be used as User Guide)

- New Multi-Orbit POsition System SPS (MOPOS SPS) (BE-BI-QP): Requested 300 pieces

• Projects in the GEFE community:

- Motor Controller Optical Interface (MCOI) (BE-BI-PM): Requested 25 pieces

- CLIC Acquisition and Control Module (CLIC-ACM) (BE-CO): Interested- Rad-tol high-bandwidth fieldbus devkit (BE-CO): Interested

- Beam Wire Scanners (BE-BI-BL): Interested

- Advanced Wakefield Experiment BPM (AWAKE BPM) (BE-BI-QP): Interested

• Open HardWare Repository (OHWR) Wiki and Email Lists (OHWR & CERN e-group)

- Function Generator Controller Lite (FGClite) (TE-EPC): Interested

- CHARM test board (EN-STI): Interested- Beam Gas Ionization (BGI) monitors (BE-BI-BL): Interested

18

BE-BI-QPOutlook

Status & Outlook (2 of 2)

18

BE-BI-QP

• Update specifications after testing the first GEFE v1 prototypes

• Receive small orders of pre-production GEFE v2 for prototyping (First half of 2016)

• Update of schematics & PCB layout for pre-production prototype (GEFE v2) (January 2016)

- LVDS receiver (SN65LVDS2DBVR) & full GEFE board (First half of 2016)

• Design and implementation of a fully automated test bench for production testing (First half of 2016)

• The GEFE board

• The GEFE community

• Update Open HardWare Repository (OHWR) Wiki

Ph.D. Student (Christophe Donat Godichal) in charge of the radiation tests

• Organise periodical meetings

• Radiation hardness qualification of:

- NMOS transistor (BST82) to be qualified up to 75 krad (December 2015)

• Organise production of GEFE for the different projects (Second half 2016)

• Production stage (Second half of 2016)

• Pre-production stage (First half of 2016)

- Addition of on-board GBTx-SCA ASIC for remote FPGA programming???

• Discuss whether to assembly the two GEFE v1 PCBs available

AcknowledgementsBE-BI-QPThe GEFE team

• Andrea Boccardi• Christophe Donat Godichal

• Thibaut Lefevre• Tom Levens

• Manoel Barros Marin

Acknowledgements• Paschalis Vichoudis, Pedro Leitao & other colleagues from my old group PH-ESE

• Slawosz Uznanski & other colleagues from TE-EPC• Felix Andreas Arnold, Stephane Burger & other colleagues from BE-BI-PM• Javier Serrano, Erik Van Der Bij, Evangelia Gousiou & other colleagues from BE-CO• Bartosz Przemyslaw Bielawski & other colleagues from BE-RF

• Juan Boix Gargallo, Thierry Bogey and the rest of my section (BE-BI-QP)

• Special thanks to Fernando Carrio Argos (PH-UAT) for his simulations of GEFE’s Power Distribution Network• Markus Brugger, Salvatore Danzeca, Georgios Tsiligiannis, Raffaello Secondo & other colleagues from RadWG

• Jose Luis Gonzalez

• Tullio Grassi (PH-UCM)

• Balint Szuk

Thank you

gefe@ohwr.org

BE-BI-QP

http://www.ohwr.org/projects/gefe

gefe-community@cern.org

21

BE-BI-QP

References

• The GBT project web page: https://espace.cern.ch/GBT-Project/default.aspx

• The VTRx application note: Application Note V2.4.pdf

• FEASTMP project web page: http://project-dcdc.web.cern.ch/project-dcdc/

• Microsemi web page: http://www.microsemi.com/

• Microsemi ProAsic3E FPGA Fabric user’s guide: PA3E_UG.pdf

• The GBTx ASIC manual: gbtxManual.pdf

• ST LHC4913 data sheet: http://atlas.web.cern.ch/Atlas/GROUPS/FRONTEND/documents/LHC4913.pdf

• ANSI/VITA specifications web page: http://www.vita.com/Specifications

• Microsemi ProAsic3E Flash Family FPGAs datasheet: PA3E_DS_v14.pdf

• Microsemi application note AC380: LPF_AC380_AN.pdf

• The GEFE project web page: http://www.ohwr.org/projects/gefe

• The VFC-HD project web page: http://www.ohwr.org/projects/vfc-hd

GEFE

FEASTMP 1.5V FEASTMP 2.5V

FMC HPCGBTx

GPIO Connector A (13 user pins)

SMASMA

LEDs

FMC HPC (160 pins)E-links

(40 Bidirectional)

Main PowerConnector

SCA E-link

GPIO SW(8-bit)

PushButton

MMXC (LVDS)GPIO/Clk IO

JTAG

M2C clocks (x2)BIDIR clocks (x2)

SFP+Power

VTRxPower

Vadj

ESLT

MMCX (LVDS)SecRefClk

E-links clocks (x3)

LDORegulator

3.3V

Board IDConnector

(13-bit) GPIO Connector B (24 user pins)

Reset

RSSIGBT M2C clocks (x2)

Prog. clocks (x4)

Osc. (25MHz)

VioBM2c

FMC DP (40 pins)Std. IOs

Std. IOs

FPGA (ProAsic3)

MMCX GPIO (x4)

IOs (Hb)

IOs (La/Ha)Core

Prog. clocks (x1)

V12p0 PowerConnector

V12p0

V3p3 & V3p3Aux

GEFE Config. ID Resistors (10-bit)

I2CI2C Connector

VTRx

GEFE boardBE-BI-QP

Detailed Block Diagram

GEFE boardBE-BI-QP

GBTx ASIC

17 mm

17 mm

VTRx

VTRx & GBTx• Main components of the new “Rad-Hard Optical Link for Experiments”

• Developed at CERN by PH-ESE

• VTRx

• Rad-Hard Optical Transceiver• Bidirectional Link at 4.8 Gbps (Versatile Link)

• GBTx

• Rad-Hard ASIC• Single Electrical Link with VTRx at 4.8 Gbps

• Multiple Electrical Links with Front-End Electronics (E-Links) - 40 Links @ 80 Mbps- 20 Links @ 160 Mbps- 10 Links @ 320 Mbps

• Widely used at CERN (Experiments & Accelerators) and External institutions (Desy, etc.)

• One dedicated E-Link for slow control (SC) @ 80 Mbps

GEFE boardBE-BI-QP

GBTx-SCA ASIC

GEFE boardBE-BI-QP

Particularities of the FMC connector in GEFE• High-pin count (HPC) socket

• The different control signals of the FMC standard are supported

• The targeted user-specific I/O data rate is up to 700 Mbps

• The high-speed lanes (DP) connected to user-specific I/Os

• Special functions of DP lanes:• Direct SC e-link between GBTx and the FMC HPC connector

• FPGA programming by JTAG master (e.g. SCA) placed on FMC

• Accessible from the rear of the board

• Up to 160 Single-Ended (or 80 Bidirectional) user-specific I/Os

• 6 differential clock lines (4 inputs and 2 bidirectional)

• Direct signals from GPIO connector A to FMC HPC connector

Bpm Digital front-end Fmc (BDF)

Single width conduction cooled FMC Module

• Resistor network required on FMC when driving LVDS signals from ProAsic3 FPGA

26

GEFE boardBE-BI-QP

FPGA (ProAsic3)

27

GEFE boardBE-BI-QP

PoweringFEASTMP

28

GEFE boardBE-BI-QP

Slow Control (SC) E-Link• Single E-link @ 80 Mbps

GEFE

FMC HPC

GBTxVTRxGbtxElinkScXx_p/n

FPGA (ProAsic3)

ResistorsNetwork

FmcDpM2c_p/n[5:7]

GbtxElinkScXx_p/n_Fpga

FmcDpM2c_p/n_Fpga[5:7]

29

GEFE boardBE-BI-QP

Low-Speed Electrical Link• Tested up to 10 Mbps over 2 Km of coper cable

GEFE

FPGA (ProAsic3)

SMA

SMA

Low-Speed Electrical Link

30

GEFE board(1 of 2)

BE-BI-QP

Clocking Scheme

GEFE

FMC HPC

VTRx

MMXC (LVDS)GPIO/Clk IO

CLK_M2C[0:1]

CLK_BIDIR[2:3]

Osc. 25MHz

MMCX (LVDS)

DCLK[0:2]GBTCLK_M2C[0:1]

CLOCKDES[0:3]

FPGA (ProAsic3)

MMCX GPIO (x4)

Hb_cc[00,06,17]

Ha_cc[00,01,17,18]

La_cc[00,01,17,18]

GBTxCDR

Osc.40MHz

SCCLKREFCLK

FmcDpM2c[5]

CLOCKDES[4]

FmcDpC2m[8]

31

GEFE board(2 of 2)

BE-BI-QP

Clocking SchemeDCLK[0]

MmcxGpio[0]

CLOCKDES[0]MmcxGpio[5]

MmcxGpio[6]

MmcxGpio[7]

CLK_M2C[1]

La_cc[0]

La_cc[18]CLK_BIDIR[3]

La_cc[17]

Ha_cc[17]

GBTCLK_M2C[0]

La_cc[1]

Osc25MHz

CLK_BIDIR[2]

ClkFeedbackClkM2c[0]

GBTCLK_M2C[1]

Ha_cc[0]

Ha_cc[1]MmcxGpio[3]

MmcxGpio[2]

MmcxGpio[1]

Hb_cc[0]

Hb_cc[6]

Hb_cc[17]MmcxGpio[4]

CLOCKDES[3]

DCLK[2]

DCLK[1]

CLOCKDES[2]MmcxClkIo

CLOCKDES[1]

SCCLK

Clock Source A

Clock Source B

Clock Source C

Ha_cc[18]

32

GEFE boardBE-BI-QP

Resets Scheme

GEFE

FMC HPC

GBTx

GPIO Connector A (13 user pins)

GPIO Connector B (24 user pins)

Reset

FPGA (ProAsic3)

ResetButton

VTRx RSSIPOR/PPR

ResetLogic

33

GEFE boardBE-BI-QP

FPGA Programming Scheme

GEFEJTAG

ConnectorFMC HPC

FPGA (ProAsic3)

FpgaXxxFmcDpM2c_p[0:4]_Xxx

ConnXxx

VoltageTranslator

(Vadj from/to V3p3) FmcDpM2c_p_r[0:4]

FmcDpM2c_p_r_Fpga[0:4]

Jumpers

Jumpers

Std. IOs

JTAG Prog.

34

Application Examples (1 of 2)

BE-BI-QP

Multi-Orbit POsition System SPS (MOPOS SPS) (BE-BI-QP)

Motor Controller with Optical Interface (MCOI) (BE-BI-PM)

STANDARD BACK-END (VFC-HD)FPGA

GBT-FPGAUSER LOGIC

GEFE

VTRx GBTx FPGA BPM ANALOG FE BPM

FMC

BPM DIGITAL FE

FMC

STANDARD BACK-END (VFC-HD)

FPGAGBT-FPGA

GEFE

VERSA EUROPE 3UBACKPLANE

MOTORUSER LOGIC

GEFE

VTRx GBTx FPGAFMC

PM DIGITAL FE

FMC

35

Application Examples (1 of 2)

BE-BI-QP

Multi-Orbit POsition System SPS (MOPOS SPS) (BE-BI-QP)

Motor Controller with Optical Interface (MCOI) (BE-BI-PM)

STANDARD BACK-END (VFC-HD)FPGA

GBT-FPGAUSER LOGIC

GEFE

VTRx GBTx FPGA BPM ANALOG FE BPM

FMC

BPM DIGITAL FE

FMC

STANDARD BACK-END (VFC-HD)

FPGAGBT-FPGA

GEFE

VERSA EUROPE 3UBACKPLANE

MOTORUSER LOGIC

GEFE

VTRx GBTx FPGAFMC

PM DIGITAL FE

FMC

BPM Digital Front-End FMC(BDF)

36

Application Examples (1 of 2)

BE-BI-QP

Multi-Orbit POsition System SPS (MOPOS SPS) (BE-BI-QP)

Motor Controller with Optical Interface (MCOI) (BE-BI-PM)

STANDARD BACK-END (VFC-HD)FPGA

GBT-FPGAUSER LOGIC

GEFE

VTRx GBTx FPGA BPM ANALOG FE BPM

FMC

BPM DIGITAL FE

FMC

STANDARD BACK-END (VFC-HD)

FPGAGBT-FPGA

GEFE

VERSA EUROPE 3UBACKPLANE

MOTORUSER LOGIC

GEFE

VTRx GBTx FPGAFMC

PM DIGITAL FE

FMC

BPM Digital Front-End FMC(BDF)

Motor Driver Interface(MDI) board

37

Application ExamplesBE-BI-QP

Advanced Wakefield Experiment BPM (AWAKE BPM) (BE-BI-QP)

New rad-tol high-bandwidth fieldbus devkit (BE-CO)

STANDARD BACK-END (VFC-HD)FPGA

GBT-FPGAUSER LOGIC

GEFE

VTRx GBTx FPGAFMC

WorldFIPDIGITAL FE

FMC

WorldFIP FE

... ...

WorldFIP FE

WorldFIP FE

WorldFIP FE

(2 of 2)

STANDARD BACK-END (VFC-HD)

FPGA

GEFE

FMC

DIGITAL FE FMC

USER LOGIC

SMA

GBTx FPGA

SMA

VTRx

Low-Speed Electrical Link

BPM ANALOG FE BPM