data concentrator - svd data multiplexer with fpga-based ... filesvd data multiplexer with...

Data ConcentratorSVD data multiplexer with FPGA-based tracking

DAQ-Meeting Bayrischzell

Jochen Dingfelder, Carlos Mariñas, Michael [email protected]

University of Bonn

July, 18th 2012

Michael Schnell (University of Bonn) Data Concentrator July, 18th 2012 1 / 14

Content

1 Motivation

2 Hardware Status

3 FPGA-based tracking

4 Simulation


Motivation

Motivation

CopperSystem

DataConcentrator

ROI

optical links


Hardware Status

Content

1 Motivation

2 Hardware Status

3 FPGA-based tracking

4 Simulation


Hardware Status

Hardware Solution

• Advanced Mezzanine Card (AMC)• Redesign with 4 SFP connectors

(Zhen-An)→ 11 cards• Received during DEPFET meeting

in Seeon• Given back to Zhen-An for fixing

backplane problems• Start testing, when it returns


Hardware Status

“SFP+ Communication”

RJ45

SFP+

SFP+

Connector

ATCA

AMCDatcon

SFP

SFP

SFP

SFP

FPGA4:1 Mux

Connector

SVD

AMCDatcon

SFP

SFP

SFP

SFP

FPGA4:1 Mux

Connector

SVD

AMCDatcon

SFP

SFP

SFP

SFP

FPGA4:1 Mux

Connector

SVD

Backplane

AMCDatcon

SFP

SFP

SFP

SFP

FPGA4:1 Mux

Connector

AMCDatcon

SFP

SFP

SFP

SFP

FPGA4:1 Mux

Connector

SVD

AMCDatcon

SFP

SFP

SFP

SFP

FPGA4:1 Mux

Connector

SVD

AMCDatcon

SFP

SFP

SFP

SFP

FPGA4:1 Mux

Connector

SVD

FPGA

Backplane

GPU Farm

FPGA

AMCCollector

FPGACollector+Tracking

SFP+

SFP+

SFP+

3 carrier boards with 4 AMCs

→ Stage 1: 4:1 / 3:1 Multiplexer ineach AMC card

→ Use one SFP+ transceiver forinter-carrier board transmission

→ AMC card interconnection overRocketIOs

1 AMC as data collector

→ Stage 2: 3:1 Multiplexer with trackreconstruction algorithm

→ Ethernet connection fortransmission of tracks to ATCA


Hardware Status

“mTCA Option”

9 or 12 Slot full-size mTCA crate with PCM, IPMI...

Full-mesh or D-Star backplane

2 Crates (18 / 24 AMC), one AMC for intercrate communication


Hardware Status

Simplified Block Diagram

FD

FD

FD

FD

4:1Mux

FD

FD

FD

FD

4:1Mux

Framer

11:1Mux

FD

FD

FD

FD

4:1Mux

Decoder

Scheduler &MMU

4 GB DDR 2Memory

Trackreconstruction

EthernetFrame

GPU ProtocolFramer

AmplitudeAnalysis

Collection

AMC "Datcon"

ATCAGPU Farm

FD

FD

FD

Framer

Framer

.

.

.

AMC "Collector"


Hardware Status

AMC “DatCon” Layout

FD

FIFO

Trafo

Ringbuffer

FD

FIFO

Trafo

Ringbuffer

FD

FIFO

Trafo

Ringbuffer

FD

FIFO

Trafo

Ringbuffer

Data Selector Ctrl

AuroraCore

ProtocolFramer

Back-plane

1 2 3 4 FD: Frame DetectionUnit (Belle II Link)

Trafo: Transformationon incoming data, likecoordinate translation,Hough transformation...

Ring buffer:Implemented in DDR2memory.

Ctrl: Control unit, sendsdata when next event isrequested by collectingFPGA


Hardware Status

Memory Management Unit (MMU)

MMU

phy.addr.

data

virt.addr.

data

TLB

LUT

Cache

BurstFramer D

DR2

burstdata

phy.addr.

Memory Management Unit for easy access to DDR2 memory

Implementation for ring buffer and memory mapping.

TLB, Cache not necessarly required


Hardware Status

Internal Protocol

0 3115

Size

Event ID TTRX-ev

23

CRC

.

.

.

CMD / Data #0

Data #1

Type

2

Flags

Type: 2 Bit

00: Command (CMD)

01: User Data (DATA)

Flags: 14 Bit

(0000)16: Normal operation

(0001)16: Test the connection,waiting for reply (only command)

(0002)16: Test the connection, reply(only command)

(1XXX )16: Error, kind of errorindicated by XXX


Hardware Status

Proposed Data Packet

0 31

ADC

20

Strip ID X

29

X Y

Y Z

96 Bit in total per strip

20 Bit: Strip ID, 10 Bit: ADC value

Fixed point arithmetic for coordinates (66 bits)

3 bits for sign

X, Y interval: [-16.99999, 16.99999] cm

Z interval: [-32.99998, 32.99998] cm


Hardware Status

Summary

Waiting for first 4-SFP AMC prototype to test

Simulation successful for intercommunication between the AMCcards

Design of internal protocol and data format

Next Steps:Test simulated system in hardware

Combined test with Giessen group to transmit ROI


Hardware Status

Thank you for your attention!


Backup Technical Specifications

Technical Specification on I/O

Input:Up to 42 (32)optical-fibre links

1.5 Gbps max

On average110 Mbpsexpected

Dynamicbandwidth

Output:6.25 Gbps overbackplane orSFP+

440 Mbpsaverage datarate

Dynamicbandwidth

→ Total: 4.6 Gbps ≈ 550 MiB/s


Backup Idea

Tracking Options

Hough Transformation to findstraight and arc tracks (afterconformal transformation)

Can be fast and efficientimplemented in the FPGA overFast Hough Transformation (FHT)

First 2D resulsts of the next slides

Open for other algorithm, likesector-neighbour finding cellularautomaton or Kalman-Filter -1

-0.5

0

0.5

1

0 0.5 1 1.5 2

a

m

0

0.5

1

1.5

2

2.5

3

3.5

4

0 0.5 1 1.5 2 2.5 3 3.5 4

y

x

Houghtransformation


Backup FPGA Implementation

Trigonometric Functions and Memory Requirements

Need trigonometric functions like sine and cosine e.g. forcoordinate transformation

CORDIC: Basic Lookup table for nodes and a shift-addimplementation of approximation functionPure LUT: Large Lookup table for every value (high memoryrequirements)

FPGAs’ DSPs used as Multiplier/Divider unit.

Block Memory needed to translate strip IDs into coordinates andROI creation


Backup FPGA Implementation

Status of the Different Blocks in Simulations

FD

FD

FD

FD

4:1Mux

FD

FD

FD

FD

4:1Mux

Framer

11:1Mux

FD

FD

FD

FD

4:1Mux

Decoder

Scheduler &MMU

4 GB DDR 2Memory

Trackreconstruction

EthernetFrame

GPU ProtocolFramer

AmplitudeAnalysis

Collection

AMC "Datcon"

ATCAGPU Farm

FD

FD

FD

Framer

Framer

.

.

.

AMC "Collector"


Backup Multiplexer Theory

Multiplexing

Three general types of multiplexing:Frequency Division Multiplexing (FDM)

Time Division Multiplexing (TDM)

Statistical Multiplexing

Because of dynamic bandwidth “Statistical Multiplexing” would bemost suitable

Demultiplexing is trivial, if only multiplexing of 32 bit words isensured


Backup Multiplexer Theory

Aurora overview

Xilinx Aurora protocol as physical link layer over GTP

Takes care of every “low level” operation like clock correction,symbol decoding and 8B/10B Encoding

Easy data transmission/framing over locallink

Error detection and initialization

Status wires for channel and lanes

Lane: One communication lineChannel: Concentration of one or more lanes


Backup Hough-Transformation

Simple Hough Trafo in 2D with straight tracks

Hits in 2 D

Want to find straight tracks i : yi = m · xi + a in this mess

→ Go to Hough-Space: a = −m · xi + yi

0

0.5

1

1.5

2

2.5

3

3.5

4

0 0.5 1 1.5 2 2.5 3 3.5 4

y

xMichael Schnell (University of Bonn) Data Concentrator July, 18th 2012 15 / 14


Hough-Space

-1

-0.5

0

0.5

1

0 0.5 1 1.5 2

a

m

Find interception of 5 Tracks around zero!

Found 2: y1 = 0.6 · x ; y2 = 1.5 · xMichael Schnell (University of Bonn) Data Concentrator July, 18th 2012 14 / 14


With tracks

Found 2:y1 = 0.6 · xy2 = 1.5 · x

0

0.5

1

1.5

2

2.5

3

3.5

4

0 0.5 1 1.5 2 2.5 3 3.5 4

y

x

tr 1tr 2



Coordinate Transformation

This is bad for tracks close or parallel to y-axes (m = inf.)!

Make a coordinate transformation like: r = x · cos(Φ) + y · sin(Φ)

0

0.5

1

1.5

2

0 0.5 1 1.5 2

y

x

trackr



Implementation in FPGA?

Use Fast Hough Transformation!

Divide and Conquer type algorithm

-1

-0.5

0

0.5

1

0 0.5 1 1.5 2

a

m



Implementation in FPGA 2

Vary with minimum and maximum size of a unit area the tolerance

Efficient and highly parallelisable, a sector for each unit

-1

-0.5

0

0.5

1

0 0.5 1 1.5 2

a

m




Also support of different shapes

One more to go...

-1

-0.5

0

0.5

1

0 0.5 1 1.5 2

a

m




Error comes for “free”

Done... finally

-1

-0.5

0

0.5

1

0 0.5 1 1.5 2

a

m


data concentrator - svd data multiplexer with fpga-based ... filesvd data multiplexer with...

Documents