the hierarchical cbm network structure and the cbmnet v2.0 protocol

Frank Lemke

The Hierarchical CBM Network Structure and the CBMnetThe Hierarchical CBM Network Structure and the CBMnetV2.0 ProtocolV2.0 Protocol

DPG – Frühjahrstagung Mainz 2012

HK 57.2

University of Heidelberg

Computer Architecture Group

Frank Lemke, Sven Schenk, Ulrich Brüning

23.03.2012

Frank LemkeDPG – Frühjahrstagung Mainz 20122

OutlineOutline

FAIR at GSI Motivation CBMnet V2.0 Network Protocol CBM Network Structure ASIC adaptions Conclusion & Outlook

Frank Lemke3

FAIR at GSI Darmstadt GermanyFAIR at GSI Darmstadt Germany

Facility for Antiproton and Ion Research (FAIR) will extend the existingGSI accelerator and synchrotron (until 2016) with

Accelerator Facility focuses on five beam properties: Highest beam intensities Brilliant beam quality Higher beam energies Highest beam power Parallel operation

Eight different Experiments andCollaborations

http://www.fair-center.deScience Vol 318, www.sciencemag.org, 2 November 2007



Compressed Baryonic MatterCompressed Baryonic Matter

One of the FAIR experiments and collaborations is

Compressed Baryonic Matter (CBM):

Investigating the highly compressed nuclearmatter using nucleus-nucleus collisions

Self-triggered detector system Providing eight different kinds of detectors

http://www.gsi.de/forschung/fair_experiments/CBM/index.html

“Design and implementation of a hierarchical DAQ network,” Deutsche Physikalische Gesellschaft e.V. Fruehjahrstagung , March 10, 2008

“Design and implementation of a hierarchical DAQ network,” Deutsche Physikalische Gesellschaft e.V. Fruehjahrstagung , March 10, 2008


MotivationMotivation

Standard Protocols do not support all special demands within CBM data acquisition (DAQ) system and an optimized design for the DAQ can results in better performance considering:

High bandwidth Synchronization mechanisms Limited space for hardware Radiation tolerance Self-triggered frontend electronics Reusability of protocol modules for different

hierarchical devices => no protocol conversions

Frank Lemke6

Features of CBMnet V2.0

Communication over one optical link supporting Data Transport Messages (DTM), Detector Control Messages (DCM) and Deterministic Latency Messages (DLM)

Optimized data utilization about 91 % (about 73 % considering 8b/10b)

Easy to use Interface Highly modular CBMnet structure Retransmission for Control and Data Messages Fast and efficient administration packets System wide clock recovery and synchronization Deterministic link latency feature for well defined DLMs Additional routing schemes available Different physical layer implementations available


Frank Lemke

CBMnet V2 interfaceCBMnet V2 interface

7

16

16

ctrl2send_startctrl2send_end

ctrl2send

data2send_startdata2send_end

data2send

data2send_stop

ctrl2send_stop

4

4

16

16

dlm2send_vadlm2send_type

dlm_rec_typedlm_rec_va

data_recdata_rec_startdata_rec_end

ctrl_recctrl_rec_startctrl_rec_endctrl_rec_stop

data_rec_stop

link_active

clk

CBMProtocolModule

Device

link_clk

res_n

Detector

DataTransport

DeterministicLatency

ControlMessage

Message

andPHY

UserLogicModules

Message

DetectorControlMessage

DataTransportMessage

Interfaces for three CBM network traffic classes for Data, Control and Synchronization

CBM Protocol Module Interface encapsulates communication Layer 1- 4 (4 partly)

All protocol specific features like initialization, retransmission are not visible for user Core Modules

Interface optimized for easy and efficient usage


Frank Lemke

FPGA ImplementationsFPGA Implementations

For prototyping three different FPGA boards were used and supported by the CBMnet technology:

Active Buffer Board (ABB) Data Combiner Board (DCB) Read-Out Controller (ROC)

Implementation specifics: No elastic buffers Only one clock source in the DAQ, recovered clocks used for sending FPGA specific configurations

V4: Barrel Shifter Position, reset V5 + V6: no Barrel Shifter, Alignment check reset

Priority reset insertion for DLMs and optimized HW

8 DPG – Frühjahrstagung Mainz 2012

Frank Lemke9

CBMnet – Beam Time ReadoutCBMnet – Beam Time Readout

FEB

FEB

FEBROC

FEB


DCB

ROCROC

ROC

FEB

FEB

FEBROC

FEB

ROCROC

ROC

FEB

FEB

FEBROC

FEB

ROCROC

ROC

FEB

FEB

FEBROC

FEB

ROCROC

ROC

Com

puter

ABB Com

puter

ABB Com

puter

ABB Com

puter

ABB

Clock and S

ync M

asterABBDCB

DCB

DCB

DCB

Front-End B

oards (FEB

) for different detectors

Frank Lemke10

Supporting new FPGA VersionsSupporting new FPGA Versions

A Spartan 6 based ROC 3 Board is currently designed within the CBM Collaboration Delivering a lot of new use cases for the ROC It can now also serve as DCB for prototyping Enabling the full features of the CBMnet

Also a Fles Interface Board (FLIB) is planned Optimizes the First Level Event Selection (FLES) connection May serve as ABB substitution for prototyping

Both boards will be supported by the CBMnet


Frank Lemke

Spartan 6 – Evalboard TestsSpartan 6 – Evalboard Tests


Frank Lemke

Generic CBMnet ASICGeneric CBMnet ASIC

Support for CBMnet build-in modules for ASICs enables usage of one protocol in the complete network

Provided modules: Register File (RF) I2C support for RF access CTRL decode for CBMnet ctrl messages (a first version) ASIC proven SERDES in 65nm CBMnet with Master/Slave support

Unbalanced links: 1 up-stream and up to 4 down-stream links Shift register chain support for analog designed ASIC parts


Frank Lemke

Generic CBMnet ASIC StructureGeneric CBMnet ASIC Structure


CB

Mnet V

2.0

Analog/DigitalRead-out HW

SE

RD

ES

LVDS

GPRF

SPRF GPRF : General Purpose Register FileSPRF : Special Purpose Register File

LVDS

LVDS

Control

Data

RFControl

Analog L

VD

S B

locks

Standard C

ell SerD

es

Analog D

elay Elem

ents

CB

Mnet interface

CB

Mnet P

rotocolM

odules

CBMnet Digital Front-end Block

Data 1x, 2x or 4x

I2C/JTAG

DL

M

8b/10bD

e-/Encoder

CLK

Data

Frank Lemke14

First ASIC with CBMnet V2.0 –First ASIC with CBMnet V2.0 –SPADIC V1.0SPADIC V1.0

HK 34.7 : SPADIC – Self triggered readout ASIC for the CBM transition radiation detectors Tim Armbruster, Peter Fischer, Michael Krieger, and Ivan Peric

Tapeout in November 2011

ASIC received in February 2012


Frank Lemke15

Conclusion Conclusion && Outlook Outlook

Build up prototyp beam time read-out chains supporting current detectors and front-end boards combined with new ones

Support beam time DAQ systems for next beam times in June and October 2012

Improve and expand generic CBMnet ASIC modules portfolio

Our goal is the design of ASICs and DAQ elements required for the final CBM DAQ system


Frank Lemke

Thank you for your attention !

Questions ?


the hierarchical cbm network structure and the cbmnet v2.0 protocol

Documents