increasing design changeability using dynamical partial reconfiguration kip heidelberg norbert abel,...

Increasing Design Changeability using Dynamical Partial Reconfiguration

KIP HeidelbergNorbert Abel, Udo Kebschull

General DAQ Setup

Norbert Abel & Udo Kebschull, KIP Heidelberg

DAQ (Cluster)

Beam

Collision Detectors

DAQ (FPGAs)

FEE (ADC)

DAQ Setup


FEE 1(Analog -> Digital)












Board F1

(Communication & Filtering)

Board F1


Board C1

(Communication & Combining)

Board C1


FPGA

Reconfiguration














Board F1


Board F1


Board C1


Board C1


FPGA

Functionality:Filter 2.4

Reconfiguration














Board F1


Board F1


Board C1


Board C1


FPGA

ReconfigurationProcess

X

X

X

X

X

Reconfiguration














Board F1


Board F1


Board C1


Board C1


FPGA

Functionality:Filter 2.5

Reconfiguration














Board F1


Board F1


Board C1


Board C1


FPGA


X

X

X

X

X

Due to these interruptions it is not possible, to use this setup

for online reconfiguration!

Due to these interruptions it is not possible, to use this setup

for online reconfiguration!

Online Reconfiguration














Board F1


Board F1


Board C1


Board C1


FPGA















Board F1Board F1

Board C1


Board C1


FPGA / ASICBridge

FPGAWorker

Communication Filtering















Board F1Board F1

Board C1


Board C1


FPGA / ASICBridge

FPGAWorker

Communication

X X
















Board F1Board F1

Board C1


Board C1


FPGA / ASICBridge

FPGAWorker

Communication Filtering

This connection is part of the fixed Board logic and therefore

unchangeable.

Dynamic Partial Reconfiguration














Board F1


Board F1


Board C1


Board C1


FPGA















Board F1


Board F1


Board C1


Board C1


FPGABridge Worker















Board F1


Board F1


Board C1


Board C1



FPGABridge Worker

X X















Board F1


Board F1


Board C1


Board C1


This is the maximumpossible changeability!

FPGABridge Worker

The worker is online reconfigurable.

The complete FPGA is offline reconfigurable.



FPGABridge Worker



FPGA

WorkerBridge



FPGA

Bridge Worker 2Worker 1



FPGA


X



FPGA


DPR Framework


FPGA


Worker 3 Worker 4

DPR Framework


FPGA

In Worker 2Worker 1

Worker 3 Worker 4

Out

Scheduling - Update


FPGA

In

Track Finder 2.3Track Finder 2.3

Worker 2


Worker 1

Zero Suppression 3.2Zero Suppression 3.2

Worker 3

Data Compression 1.7Data Compression 1.7

Worker 4

Out

Scheduling - Update


FPGA

In


Worker 2


Worker 1

Worker 4

Out

XZero Suppression 3.2Zero Suppression 3.2

Worker 3

Scheduling - Update


FPGA

In


Worker 2


Worker 1


Worker 4

Out

Zero Suppression 3.2Zero Suppression 3.2

Worker 3

Scenario-Based Scheduling


FPGA

In


Worker 2


Worker 1


Worker 3


Worker 4

Out

New Scenario!



FPGA

In Worker 2Worker 1 Out

XX


Worker 4


Worker 3



FPGA

In

Track Fitter 3.2Track Fitter 3.2

Worker 2

Track Fitter 3.2Track Fitter 3.2

Worker 1 Out


Worker 4


Worker 3

Runtime Scheduling


FPGA

In Worker 2Worker 1

Worker 3

Out

Worker 4

TrackFinder

3.7

DataComp

1.8

TrackFinder

3.7

VertexFinder

2.4

TrackFinder

3.7

Runtime Scheduling


FPGA

In Worker 2Worker 1

Worker 3

Out

Worker 4

TrackFinder

3.7

TrackFinder

3.7

TrackFinder

3.7

DataComp

1.8

VertexFinder

2.4

Runtime Scheduling


FPGA

In Worker 2Worker 1

Worker 3

Out

Worker 4

TrackFinder

3.7

DataComp

1.8

TrackFinder

3.7

TrackFinder

3.7

VertexFinder

2.4

Runtime Scheduling


FPGA

In Worker 2Worker 1

Worker 3

Out

Worker 4TrackFinder

3.7

TrackFinder

3.7

TrackFinder

3.7

DataComp

1.8

VertexFinder

2.4

Runtime Scheduling


FPGA

In Worker 2Worker 1

Worker 3

Out

Worker 4TrackFinder

3.7

VertexFinder

2.4

TrackFinder

3.7

TrackFinder

3.7

DataComp

1.8

Runtime Scheduling


FPGA

In Out

DataComp

1.8

TrackFinder

3.7

VertexFinder

2.4

TrackFinder

3.7

TrackFinder

3.7

Requirements:- Data of absent workers has to be buffered- Scheduler has to be responsive to changing conditions- Reconfiguration overhead has to be as low as possible- Worker interconnections have to be very flexible- Throughput should stay as high as possible

This leads to the need of a Reconfiguration Framework

Reconfiguration Framework


VHDL

Development of the workers and the In & Out modules in VHDL

Partitioning of the design in VHDL

Implementation of the scheduler

Logical implementation of the inter module communication (IMC)

Partitioning of the Chip

Physical implementation of the IMC

Timing

Using the conventional tools...

Developers have to care about:



OO & VHDL

Development of the workers and the static In & Out modules

Predefined scheduler, partitioning and

communication matrix

Developers just have to care about:Encapsulated

Using our DPR-Framework...

Framework



Functionality:- Data of absent workers is buffered- Scheduler is triggered by the buffer levels- The separation of long and short reconfiguration based on object orientation leads to a minimal reconfiguration overhead- Worker interconnections are free programmable- Runtime Scheduling covers Scenario-Based Scheduling as well as Update Scheduling

T1 = new TrackFinder3_7();T2 = new TrackFinder3_7();T3 = new TrackFinder3_4();

T1.s.connect(T3.a);T2.s.connect(T1.b);

while (1) { ... if (scn_ch == 1) T1.finish();

V = new VertexFinder(); ...}

FPGA

In Out

DataComp

1.8

TrackFinder

3.7

VertexFinder

2.4

TrackFinder

3.7

TrackFinder

3.7



Throughput:22 MB/s with 16 Bit Databus44 MB/s with 32 Bit Databus88 MB/s with 64 Bit Databus

176 MB/s with 128 Bit Databus

Example:Worker-Size: 534 CLBs <=> 212 kBWorker-Context: 4 BRAMs <=> 10 kB

=> Delay:Short Reconfiguration: 26µsLong Reconfiguration: 530µs

FPGA

In Out

DataComp

1.8

TrackFinder

3.7

VertexFinder

2.4

TrackFinder

3.7

TrackFinder

3.7


FPGABridge Worker

X X

Summary


High Energy Physic Experiments produce very high data rates => DAQ has to be very performant

High Energy Physic Experiments have a very long lifetime => DAQ has to be very flexible

====> Usually reconfigurable FPGAs are used for DAQ

There are two kinds of Reconfiguration:- Offline Reconfiguration- Online Reconfiguration

Online Reconfiguration often depends on the separation in Bridge & Worker

FPGAIn Out

DataComp

1.8

TrackFinder

3.7

VertexFinder

2.4

TrackFinder

3.7

TrackFinder

3.7

DPR makes it possible to place Bridge and (even multiple) Workers on one FPGA

But DPR requires a complex toolflow and much know-how

Our DPR Framework encapsulates the DPR specific tools and methods =>Developers can use DPR without going in detail with it

increasing design changeability using dynamical partial reconfiguration kip heidelberg norbert abel,...

Documents

analog digital fee

kip heidelberg fee

fpga slide

norbert abel udo kebschull

udo kebschull slide

online reconfiguration

fpga functionality

fixed board logic