gbt-sca slow control adapter asic lhcb upgrade electronics meeting 12 june 2014

GBT-SCASlow Control Adapter ASIC

LHCb Upgrade Electronics meeting

12 June 2014

Design Team Core Digital Logic:

Christian Paillard, Alessandro Caratelli e-port interface:

Sandro Bonacini ADC:

icSparkling IP block DAC:

Xavier Llopart (Medipix IP block) Chip Assembly:

Christian Palliard, Rui F. Oliveira AOB:

Kostas Kloukinas, Paolo Moreira

12 June 14 [email protected] 2

SCA in the GBT system


The GBT System

A Single Link for:

Readout (DAQ)

High speed unidirectional (up-link)

Trigger data (up-link)

Timing Trigger and Control (TTC)

Clock reference and synchronous control (down-link)

Trigger decisions and control (down-link)

Low and fixed latency

Experiment control (SC/DCS/ECS)

Modest bandwidth (bidirectional link)


Embedded Electronics Control Room

Optical Link

Custom ASICs in the detectors: Radiation Tolerant: Total dose & Single Event Upsets

Commercial components in the control room FPGAs used to implement multi-way transceivers

[email protected] 5

GBT-SCA front-end interconnects

12 June 14

e-port

GBTX GBT on FPGA

Slow Control

DAQ

Timing and Trigger

e-port

e-port

e-port

e-port

e-port

e-port

e-port

e-port

GBT-SCA

FE ASICs

Embedded electronics

Control room

User busses and I/O for Control and monitoring

DataTiming

ASIC dedicated to slow

control functions. System Upgrades for SLHC

detectors. Replacement for the CCU &

DCU ASICs CCU: Communication Control

Unit DCU: Detector Control Unit in

CMS).

Flexible to match the needs of different Front-End systems.

Technology: CMOS 130nm using radiation tolerant techniques.

The Slow Control Adapter ASIC Dual redundant e-Ports for

GBTX e-links. Network Controller Arbiter 16 I2C masters (7bit/10bit

addressing) 1 JTAG master controller 1 SPI master (with 8 slave

select) 32 General purpose

digital I/O lines individually programmable as input / output / interrupt input

31 Analog inputs multiplexed in a 1 bit single slope integrative ADC

4 independent analog output (8 bits resolution)

Auxiliary I2C port for testability


PORTI2C Master

x 16

SPI Master

JTAG Master

General purpose I/O

Interrupts

AD

C In

terf

ace

ADC

Calibration Logic

DAC x 4

DA

C In

terf

ace

E- PORT (Auxiliary)

32

31

4

Temperaturecontrol E-FUSES Arbiter

Net

wor

k Co

ntro

ller

SDASCLAuxiliary I2C Port

E-clock

TX

RX

RX FIFO

SERD

ES

TX FIFO

Config

TX

RX

clock

TX

RX

RX FIFO

SERD

ES

TX FIFO

Config

TX

RX

Clock and reset control logic

Single event upset counter

Figure 1. The SCA top level block diagram

Communication architecture


Communication Architecture The Three Communication Layers of the SCA:

GBT-link Layer Connects the GBT ASIC to the Control Room electronics via a point-to-point, bi-

directional, 4.8Gbps, optical link using a special, SEU robust, transport protocol. The transport protocol is totally transparent to the user data.

e-link Layer Connects to the SCA ASIC via a point-to-point, bi-directional, 80Mbps link, using a

packet oriented transport protocol (HDLC protocol). Data packets are encapsulated by the GBT-link protocol.

Channel Layer Connects to the Front-End ASICs via bi-directional links. SCA on-chip peripherals are also Channel Blocks. Channel Blocks are controlled via a command driven protocol. The Channel Protocol Commands are encapsulated by the e-link protocol.


[email protected] 9

GBT-link Packet Format Fixed packet length: 120bits

Packet transmission rate: 1/25ns Data transmission rate: 4.8 Gbps

Fixed bandwidth allocation: Trigger path: 640 Mbps Control path: 160 Mbps

1 internal e-link (for GBT management) 1 external e-link (for GBT-SCA chip) 40 MHz DDR (80 Mbps)

Data path: 2.56 Gbps 10 e-links @ 320 Mbps 20 e-links @ 160 Mbps 40 e-links @ 80 Mbps

Data flow: Symmetrical, Bi-directional data transmission. Transmission of GBT-packets is continuous. Data from e-link ports are muxed/demuxed in the GBT-link stream. GBT data path is unaware of the e-link transfer protocol.

12 June 14

2 bits

H IC EC FEC

80 bits 32 bits4 bits

EC

2 bits

HDLC Packet Format


8 bits

SOF01111110

Address Control DATA FCS EOF01111110

8 bits 16 bits16∙n bits6 bits 6 bits

HDLC packet structure:

HDLC Packet: SOF/EOF: Frame delimiter character. Address: Packet destination address. Control: Indicates the type of the data packet. FCS: Frame Check Sequence for transmission error detection. ()

Uses bit-stuffing techniques in order to achieve symbol synchronization. The frame delimiter contains 6 consecutive ones; Any sequence of 5 ones in the stream is stuffed with one following zero at the transmitter side Any sequence of more than 6 ones is considered to be frame abort or channel idle signalling

Bi-directional operation. Each received command packet get acknowledged or rejected by the SCA

SCA Packet Formats


SCA Packet: CH#: Specifies the SCA Channel interface to be addressed TR#: Incremental identification number for each transmitted command. CMD: Is a command code that specifies the request. The operation can refer to a specific

internal register of the channel or a front-end ASIC destination address. In this case an address field follows the command.

ERR: In case of an error has encountered, return an error otherwise returns 0x00. LEN: Is a field that specifies the DATA field length. DATA: Is an optional variable length field upon LEN value.

8 bits

SOF Address Control DATA FCS EOF

8 bits 16 bits16∙n bits6 bits 6 bits

HDLC packet :

8 bits 16∙n bits8 bits 8 bits 8 bits

CH#TR# CMD DataLENSCA up-link packet structure:

CH#TR# ERR DataLENSCA down-link packet structure:

Channel Commands At the reception of any request packet, the SCA replies with an

acknowledge packet The acknowledge packet include information of the status of the operation and eventually the

data value required.

Each command can be directed to a specific channel interface. Upon reception of a command, the channel performs the required operation on its interface. The command field and the data content of a given packet is interpreted differently according

to the channel to which it is addressed.

Each channel type has a specific set of valid commands. Channels receiving an invalid command do not execute any action and the error condition is

reported to the user.

Enable/Disable command for each channel When not enabled, the interface core is kept in Power Down Mode to reduce power

consumption. At the enable, the channel interface get reset.


Implementation


e-link & e-port Block Diagram


e-port implementation for the GBT-SCA e-link interface: electrical, bi-directional chip-to-chip interconnect at 80Mbps. HDLC (High Level Data Link Control) packet oriented communication protocol. Special commands: CONNECT, RESET, TEST (loopback) Acknowledge transactions using the HDLC protocol mechanism (packet numbering).

RXcontrol

TXcontrol

TX FIFO

RX FIFO

RXDATA

TXDATA

ATLANTICBus

e-port e-link

SCA Network

Controller

Clk 40 MHz

PHY

RXCLK

RESET

Dual Redundant e-ports

14/05/2014 GBT-SCA Design review 15

The GBT-SCA is equipped with a Primary and a Secondary e-port to be connected with two adjacent GBTX chips for redundancy Only one e-port is needed for operation and can

be active at a time Data from the inactive e-port is discarded Switching between ports is command driven.

(HDLC “CONNECT” command) Clock activity or noise on the differential input of

the inactive link is discarded Upon start up, the user application software

should send a “CONNECT” command to activate the Primary e-port from the Primary e-link.

To switch to the Secondary e-port a “CONNECT” command should be send through the Secondary e-link.

Figure 2. GBT-SCA to GBTX connectivity

Figure 3. Logical view of HDLC e-ports

Physical Layer (electrical)

SLVS (Scalable Low Voltage Standard) JEDEC standard: JESD8-13 Differential voltage based signaling protocol.

Voltage levels compatible with deep submicron processes. Low Power, Low EMI

Line impedance: 100 Ohm Differential mode: 400 mV Common mode: 0.2V


0.2V

V

200mV200mV

Auxiliary I2C port

Auxiliary I2C port for testability

Allows direct access to Network Controllerby-passing the e-ports.

Receive SCA commands and returns replies

Hardwired “Test Enable” pin

Possibility to use as an expansion port


The SPI channel


Synchronous bit oriented serial link. Full duplex synchronous serial data

transfer Configurable single transaction length up

to 128 bits RW Configurable communication speed from

156KHz to 20MHz settable with 128 possible values

Asynchronous reset signal with configurable pulse length

Acknowledge packet to report the end of the communication

Supported working modes:

(0,0) (0,1) (1,0) (1,1). Possibility to invert the logical level of the

output signals during the inactivity time.

com

m. c

ontr

ol

Config. registers

MISOMOSI

/SS <7:0

wis

hbon

e in

terf

ace

SCLK

Figure 5. The SPI channel scheme

The JTAG channel


128 bit shift registers to hold the TDO/TDI and TMS bit-streams

Protocol implemented in software Configurable single transaction length up to 128

bits RW Configurable communication speed from

156KHz to 20MHz settable with 128 possible values

Asynchronous reset signal with configurable pulse length

Acknowledge packet to report the end of the communication

Possibility to invert the TCK edge synchronously to which the transmitted bits on the TDO and TMS lines are latched

Possibility to invert the TCK edge synchronously to which the received bits on the TDI line are evaluated.

Possibility to invert the logical level of the output signals during the inactivity time.

com

m. c

ontr

olre

set

Config. registers

TCK

TMS

TDI

TDO

ARST

wis

hbon

e in

terf

ace

Figure 6. The JTAG channel scheme

The I2C channels

16 independent I2C master channels Concurrent operation of all 16 channels Support is limited to single bus master

topologies. 7-bit and 10-bit addressing modes. Single-byte and Multi-byte transfer

modes. Read-Modify-Write mode that allows for

AND, OR, XOR logic operations with a Mask register.

Programmable transfer rate from 100Kbps to 1Mbps.

Power Down mode per channel


Figure 4. I2C channel block diagram

Command buffer

Config. registers

SCL

wish

bone

inte

rfac

e

SDAI2C protocol

state machine

The GPIO channel


32 general-purpose I/O signals. All general-purpose I/O signals are bi-

directional having individually programmable direction.

All general-purpose I/O signals can be three-stated.

General-purpose I/O signals programmed as inputs can cause interrupt requests.

General-purpose I/O signals programmed as inputs can be registered at raising edge of system clock or at user programmed edge of external clock.

Alternative input reference clock signal from external interface.

Power Down mode

wish

bone

inte

rface

Interrupt controller

Int. clk

General purpose I/O (x32)

Ext. clk

Figure 7 - The GPIO channel

The DAC channel

4 independent Digital to analog converters.

The single DAC is an IP Block from MEDIPIX team

Voltage output: 0.0V – 1.0V 8 bit resolution


Figure 8 – The DAC channel

wish

bone

inte

rface

DAC 1

DAC 2

DAC 3

DAC 4

8

8

8

8

Analogoutput 1

Analogoutput 2

Analogoutput 3

Analogoutput 4

ADC channel 12-bit ADC (LSB = 244 µV) Single -slope architecture Input range: GND < Vin < 1 V Single supply voltage: VDD = 1.5 V System clock frequency: 40 MHz Max. conversion rate ≈ 3.5 KHz

Max conversion time ≈ 600 μsec Operating temperature: -30°C to +80°C 32 Multiplexed Input channels 1 channel occupied by an on chip Temp

sensor All channels feature a switchable 10 μΑ

current source to support Pt sensors. Power consumption

350uA @ 1.2V (420uW analog part ) Power Down mode Automatic Offset cancelation

& Gain correction Pre-calibration at production testing

phase


DIGITALANALOG

ADC Control

logicCalibration

logic

E-fuses read logic

E-fuses bankE-fuses bankE-fuses bankE-fuses bankE-fuses bank

Supervising logic +Wishbone interface

Figure 9. The ADC channel scheme

Layout


E-Fuses

Prompt

ADC analog

DAC

ADC calib. logic

E-Link

Network controller

SPI channel

JTAG channel

GPIO channel

I2C channels

3.6 x 3.6 mm

Packaging (draft)

12 June 14 [email protected] 25TOP WIEW

ADC_in_pad<0>

ADC_in_pad<2>

ADC_in_pad<5>

ADC_in_pad<8>

ADC_in_pad

<11>

ADC_in_pad

<14>

ADC_in_pad

<17>

ADC_in_pad

<20>

ADC_in_pad

<23>

ADC_in_pad

<26>Pad_

resOutDAC_ou

t_pad<2>

DAC_out_pad<3>

JTAG_reset_pad

ADC_in_pad<1>

ADC_in_pad<1>

ADC_in_pad<4>

ADC_in_pad<7>

ADC_in_pad

<10>

ADC_in_pad

<13>

ADC_in_pad

<16>

ADC_in_pad

<19>

ADC_in_pad

<22>

ADC_in_pad

<25>

DAC_out_pad<28>

DAC_out_pad<0>

DAC_out_pad<1>

TDO_pad

RESET_B_pad

FuseProgram

Pulse_pad

ADC_in_pad<3>

ADC_in_pad<6>

ADC_in_pad<9>

ADC_in_pad

<12>

ADC_in_pad

<15>

ADC_in_pad

<18>

ADC_in_pad

<21>

ADC_in_pad

<24>

ADC_in_pad

<27>

ADC_in_pad

<29>

ADC_in_pad

<30>TDI

_pad

rx_sd Rx_sd_n AGND AGND AGND AGND AVDD AVDD AVDD AVDD

SCL_pad<14>

SCL_pad<15>

TCK_pad

link_clk link_clk_n

SPI_ss_pad<0>

AGND AGND AGND AGND AVDD AVDD AVDD AVDDSDA_pad<14>

SDA_pad<15>

TMS_pad

tx_sd tx_sd_n

SPI_ss_pad<1>

DVSS DVSS AGND AGND AVDD AVDD DVDDSCL_pad<13>

SCL_pad<12>

SCL_pad<11>

SCL_pad<10>

rx_sd_aux

Rx_sdaux_n

SPI_ss_pad<2>

DVSS DVSS DVSS DVSS DVDD DVDD DVDDSDA_pad<13>

SDA_pad<12>

SDA_pad<1>

SDA_pad<10>

link_clk_aux

link_clk_aux_n

SPI_ss_pad<3>

DVSS DVSS DVSS DVSS DVDD DVDD DVDDSCL_pad<9>

SCL_pad<8>

SCL_pad<7>

SCL_pad<6>

tx_sd_aux

tx_sdaux_n

SPI_ss_pad<4>

GND GND GND GND VDD VDD VDDSDA_pad<9>

SDA_pad<8>

SDA_pad<7>

SDA_pad<6>

link_aux_disable

_pad

SPI_miso_pad

SPI_ss_pad<5>

GND GND GND GND VDD VDD VDDSCL_pad<5>

SCL_pad<4>

SCL_pad<3>

SCL_pad<2>

auxPortTestEn_pad

SPI_mosi_pad

SPI_ss_pad<6>

GND GND GND GND VDD VDD VDDSDA_pad<5>

SDA_pad<4>

SDA_pad<3>

SDA_pad<2>

auxPortSCL

_pad

SPI_clk

_pad

SPI_ss_pad<7>

GPIO_pad<4>

GPIO_pad<7>

GPIO_pad

<10>

GPIO_pad

<13>

GPIO_pad

<16>

GPIO_pad

<19>

GPIO_pad

<22>

GPIO_pad

<25>

GPIO_pad

<28>

SCL_pad<1>

SCL_pad<0>

auxPortSDA_pad

GPIO_pad<0>

GPIO_pad<2>

GPIO_pad<5>

GPIO_pad<8>

GPIO_pad

<11>

GPIO_pad

<14>

GPIO_pad

<17>

GPIO_pad

<20>

GPIO_pad

<23>

GPIO_pad

<26>

GPIO_pad

<29>

SDA_pad<1>

SDA_pad<0>

pad_GPIO_EXTCLK

GPIO_pad<1>

GPIO_pad<3>

GPIO_pad<6>

GPIO_pad<9>

GPIO_pad

<12>

GPIO_pad

<15>

GPIO_pad

<18>

GPIO_pad

<21>

GPIO_pad

<24>

GPIO_pad

<27>

GPIO_pad

<30>

GPIO_pad

<31>

N

M

L

K

J

I

H

G

F

E

D

C

B

A

N

M

L

K

J

I

H

G

F

E

D

C

B

A

14 13 12 11 10 9 8 7 6 5 4 3 2 1

14 13 12 11 10 9 8 7 6 5 4 3 2 1

pwr3_3pad

Package Type: LFBGA Low Profile Fine Pitch BGA

(Chip Scale Package) Ball Pitch: 0.8 mm Size: 12 x 12 mm Height: 1.2-1.7 mm Pin count: 196 Preliminary pinout Manufacturers:

ASE via IMEC NOVAPACK

Final Verifications

Functional simulation at the top level Post Layout simulations Simulation with SEU injection Ultrasim Mixed-Signal simulations SCA emulation on FPGA Protocol tests with commercial devices


FPGA test bench 1/2


SCA logic

I2C Slave x4

USB control logic

I2C Slave x4

SC

A p

acke

t ge

ner.

E-Port Master

I2C Slave x4

E-Port Master

I2C Slave x4

Parallel I/O ctrl

SPI Slave Multimode

Comm. control

Comm. control

JTAG slave

Transf. control flags

SCA send packet

SCA rec packet

GPIO data

SPI data

JTAG data

Slaves Configure

Test System config.

I2C0 data

I2Cn data

Drivers +

USB bluster

+ interface

etc.

Low level func. to handle GBT-

SCA communication

Low level func. to handle slaves

Testing software: Start random transaction on random

channels Whait with time-out replies from the

SCA logic Compare data received on slave

interfaces with the expected behaviour respect sent requests

Write operation log and error log Test passed when no errors are

encountered

Software FPGA test firmware

JTAG test with commercial hardware

14/05/2014 28GBT-SCA Design review

SCA logic

I2C Slave x4

USB control logic

I2C Slave x4

SC

A p

acke

t ge

ner.

E-Port Master

E-Port Master

I2C Slave x4

Parallel I/O ctrl

SPI Slave Multimode

Comm. control

Comm. control

Transf. control flags

SCA send packet

SCA rec packet

GPIO data

SPI data

JTAG data

Slaves Configure

Test System config.

I2C0 data

I2Cn data

Drivers +

USB bluster

+ interface

etc.

Low level func. to handle GBT-

SCA communication

Low level func. to handle slaves

Program en external commercial FPGA through the GBT-SCA emulated on an other FPGA: Read FPGA configuration file (ISE output

file containing series of directives for JTAG interface to program the target Xilinx FPGA)

Find the target FPGA in the JTAG chain Generate corresponding commands in

SCA format Send the commands to the GBT-SCA

emulated in the main FPGA Write error log of unsuccessful

operations

Software FPGA test firmware

JTAG interface

SPI interface

I2C interface

FPGA test bench 2/2


SCA logic

I2C Slave x4

JTAG port

I2C Slave x4

E-Port Master

I2C Slave x4

E-Port Master

I2C Slave x4

Parallel I/O ctrl

SPI Slave Multimode

Comm. control

Comm. control

JTAG slave

Drivers andUSB bluster

Command line interface

FPGA test firmware

Nios II Processor

Handle shell interface E-port drivers Slaves drivers Handle communication with SCA Real-time verification of concurrent operations

performed by the SCA Report Operation information and identified

errors

Timer banks

SDRAMcontroller

PLL

Specifications Documentation:

“Users Manual” document (draft) “Table of commands”


Status & Planning

Design Readiness review on 14/5/2014 Suggestions for design verifications Minor design modifications (suppression of external biasing resistor)

Design Status Ready for submission Participate on the GBT-chipset engineering run

scheduled for this month (June 2014) Estimated TAT: 4-5 months Yield a few hundred SCA chips

Test Bench Preparation



Thank you

gbt-sca slow control adapter asic lhcb upgrade electronics meeting 12 june 2014

Documents

synchronous control

control sequences

slow control system

control room fpgas

detector control unit

slow control functions

slow control protocol

control ttcclock reference