overview the road to automatic - users.elis.ugent.be

Scalable Hardware and Automatic Hardware Design – Dirk Stroobandt – 2007-04-27Faculty of Engineering – Electronics and Information Systems Department

pag. 1

Reconfigurable Hardware for a Scalable Video Codec and

the Road to Automatic Hardware Design

Prof. Dirk Stroobandt

Ghent University, PARIS group

Scalable Hardware and Automatic Hardware Design – Dirk Stroobandt – 2007-04-27

Faculty of Engineering – Electronics and Information Systems Department

pag. 2

Overview

• The RESUME project

– Motivation and goals

– Scalable video codec

– The hardware demonstrator

– Energy measurements

• Towards automatic hardware generation (FlexWare)

– The problem: automation needed

– The solution: loop transformations

– Hardware generation

– Guiding the search space exploration



pag. 3

Overview













pag. 5

RESUME users' group



pag. 6

Overview













pag. 7

Context

Conflict

Multimedia applications(on mobile terminals)

Demand huge

computational power

(real time)

Hardware (HW)

FixedHigh parallelism

Expensive

FlexibleHigh parallelism

Good compromise

Demand flexibility,

changes during execution

Software (SW)

Very flexibleLow parallelismSlow

Other option: FPGAs: “Field Programmable Gate

Array”: Large array of hardware blocks

with reconfigurable functions and interconnections



pag. 8

Why is so much flexibility needed?

• Time multiplexing of hardware resources

• New applications emerge and terminal has to be able to run them

• Application Specific HW accelerator is useless if applications are not known beforehand

• HW is tailored to each application that runs: efficiency

• Scalable applications to change QoS to the needs of the moment

?



pag. 9

Scalable VideoClients

Quality ~ Deployed

hardware resources

Server

Encode once

Node

Node

Intelligent Network

Rescale video stream



pag. 10

Main project goal

• To demonstrate the feasibility and the benefits of

using reconfigurable hardware to deliver scalable

quality of service to multimedia handheld devices



pag. 11

DemonstratorServer

Client

Stored videostreams

OperatingSystem

QoSnegotiation

ClientGUI

QoSnegotiation

Videotransformation

Videostreaming

Decodercontrol

software

HardwareFPGA on

PCI-board



pag. 12

Overview













pag. 13

Overview video codec

Spatial+

ResolutionScalability

Statistical +

Resolution &Quality

Scalability

Uncompressed

frames

Decompressed

frames

Temporal+

Temporal Scalability

MotionEstimation

WaveletTransform

EntropyEncoding

Pack

MotionComp.

InverseWavelet T.

EntropyDecoding

Unpack

Pullbit stream



pag. 14


Uncompressed

frames

Decompressed

frames

Temporal+

Temporal Scalability

MotionEstimation

WaveletTransform

EntropyEncoding

Pack

MotionComp.

InverseWavelet T.

EntropyDecoding

Unpack

Pullbit stream



pag. 17


Spatial+

ResolutionScalability

Uncompressed

frames

Decompressed

frames

MotionEstimation

WaveletTransform

EntropyEncoding

Pack

MotionComp.

InverseWavelet T.

EntropyDecoding

Unpack

Pullbit stream



pag. 18

Wavelet transform

• Separate high- and low-frequency

• Provide scalability in resolution

• ↔AVC: DCT, block based



pag. 19


Statistical +

Resolution &Quality

Scalability

Uncompressed

frames

Decompressed

frames

MotionEstimation

WaveletTransform

EntropyEncoding

Pack

MotionComp.

InverseWavelet T.

EntropyDecoding

Unpack

Pullbit stream



pag. 20

Quality scalability



pag. 21

Overview













pag. 22

Development board256 MiB PC333

DDR SDRAM

PCI interface64 Mibit Flash

JTAG, Ethernet,

Serial IO

Switches and indicators Altera Stratix S60



pag. 23

From algorithm to demonstrator

Software FPGA VGA-

cardPCI PCI (DMA)

Dec.

frames

MotionComp.

InverseWavelet T.

EntropyDecoding

Enc.

video stream

Un-pack

Control



pag. 27

From algorithm to demonstrator MotionComp.

InverseWavelet T.

Bitplaneassembler

ColorConv.

WED

Data Objects are too large to store in the FPGA

MotionComp.

InverseWavelet T.

Bitplaneassembler

ColorConv.

WED

MotionComp.

InverseWavelet T.

Bitplaneassembler

ColorConv.

WED

DDRBottleneck



pag. 28

Infrastructure: SOPC-builder

PCI

DDR

FPGA

Avalon

switch fabricPCI-

core

DMA DDR-core

WED IDWT ...

Custom components

SOPC-builder (Quartus, Altera):

Automatic generation of Avalon switch fabric

Board



pag. 29

From algorithm to demonstrator

• WED+Assembler: 50+ million symbols/s

⇒ real time lossless CIF

• IDWT: row/column implementation should be

replaced with more memory friendly line based

version

⇒ automatically generated scalable design

• MC: short and random DDR accesses

⇒ more difficult than expected

• DDR bottleneck:

locality problems ⇐ non block based approach



pag. 31

The hardware at work



pag. 32

Overview













pag. 33

Energy measurements

• TCP202 15 Ampere AC/DC current probe



pag. 34

> 1J

Different sequences



pag. 35

Per component 10 5 GOPs ⇒ Energy = ~ 1/2



pag. 36

Conclusions

• Working Demonstrator

• Scalable Video Decoding:

– Practically feasible

– Leaving out block based approach

is heavy burden for system bandwidth

– Real time decoding possible with better FPGA

board

• Negotiation/adaptation:

– Real time implementation of MPEG-21 BSDL-based

framework in streaming environment



pag. 37

Overview













pag. 38

FlexWareExploitation of Flexible Hardware Platforms for Massively Parallel Bioinformatics Applications

Parallel Information Systems group (UGent)– Techniques and methodology for

(reconfigurable) HW design

Design Technology Group (IMEC)– Novel processor architectures and mapping of

applications onto them

Dekimo Products NV– Ghent based engineering company

– Provides solutions for third party companies (intelligent electronics)

Bioinformatics & Evolutionary Genomics group (UGent)– Gene and genome annotation

– Comparative and evolutionary genomics



pag. 39

Main goals

• Explore the flexibility and exploitation of parallelism in

– DSP

– FEENECS

– ADRES

– FPGA

• Test example: Smith-Waterman

• Tools for “automatic” HW generation

• Evaluation on different applications



pag. 41

Overview













pag. 42

Recall: 2D-IDWT

• Inverse Discrete Wavelet Transform



pag. 43

Specifications

• Real-time:

– 45 frames/s

– CIF resolution (288x352 pixels)

• Target platform:

– FPGA-board

– Estimated clock speed: 50 MHz



pag. 44

1st Design

• Simulation results:

– 869530 cycles/frame

• Synthesis results

– Clock speed: 68.91 MHz

• Expectation: 79 frames/s

• Measurements on hardware:

– 29 frames/s: Memory bottleneck !!!



pag. 45

Memory bottleneck



pag. 46

Problem

• Memory bottleneck

• Design automation needed

– Manual design process = slow, error-prone, ...

– Lots of designs to be made

• Reconfigurable HW (QoS)

• Different platforms



pag. 47

Overview













pag. 48

Loop Transformations

• Loop Transformations

– improve spatial and temporal locality of data

accesses

– polyhedral model

– common practice for software

• Hardware Generation from the polyhedral model



pag. 49

Loop Transformations

Original algorithm in, e.g., C

Representation in the

Polyhedral Model

Optimized

algorithm

in, e.g., C

Optimized

algorithm

in HW (VHDL)

Loop

Transformations



pag. 50

for i = 0 .. N for j = 0 .. N-i if i==0 B[j]=1; B[i+j]=B[i+j]*A[i][j];

for i = 0 .. N for j = 0 .. N-i if i==0 S1(i,j); S2(i,j);

S1(i,j): B[j]=1; S2(i,j): B[i+j]=B[i+j]*A[i][j];

Control flowStatement

definitions

Iteration domains Memory accesses

The Polyhedral Model



pag. 51

Iteration domains:

j0 1 N

i

0

1

N Execution order

S2(i,j): R/W B[i+j] R A[i][j]

S1(i,j): W B[j]

Accesses to B[N-1]

Memory accesses:





pag. 52

[ i 'j ' ]=[1101].[ ij ]

Transformationi'

0

1

N

j'0 1 N

Accesses to B[N-1]

S2(i'-j',j'): R/W B[i'] R A[i'-j'][j']

S1(0,i'): W B[i']

[ i 'j ' ]=[0110].[ ij ]S1:

S2:




pag. 53

for i = 0 .. N B[i]=1; for j = 0 .. i B[i]=B[i]*A[i-j][j];

for i = 0 .. N S1(0,i); for j = 0 .. i S2(i-j,j);

S1(0,i): B[i]=1; S2(i-j,j): B[i]=B[i]*A[i-j][j];

Code generation

(CLooG)

Statement

definitions

Polyhedral representation

Iteration domains


Optimized

program



pag. 55

Overview













pag. 56

for i = 0 .. N for j = 0 .. N-i if i==0 B[j]=1; B[i+j]=B[i+j]*A[i][j];


S1(i,j): B[j]=1; S2(i,j): B[i+j]=B[i+j]*A[i][j];

Control flow Statement

definitions

Control Hardware Data Path + Memory hierarchy

Code generation

(CLooGVHDL)

Hardware generation

Trans-

formations



pag. 58

Experiment: 2D-IDWT

• Find good loop transformations

– SLO: Suggestions for Locality Optimizations

• Apply transformations

– URUK

• Generate Hardware

– CLooGVHDL



pag. 59

Experiment: 2D-IDWT

log2(Reuse Distance)

# Accesses



pag. 60

Experiment: 2D-IDWT

SLO: Suggestions for Locality Optimizations



pag. 61

Experiment: 2D-IDWT

SLO: Suggestions for Locality Optimizations



pag. 62

Experiment: 2D-IDWT

• Generation of control block:

– fully automatically: CLooGVHDL

– sequential execution of statements

• Generation of statements:

– small tools and ad-hoc scripts

– manual optimizations possible

– reusable for different transformations



pag. 63

Experiment: 2D-IDWT

• Data flow to external memoryData flow Burst Usage Variant

Original 5.25 RC 50% RC-basedFused 2.625 RC 100% line-basedFused+Tiled 2 RC 100% stripe-based



pag. 64 Scalable Hardware and Automatic Hardware Design – Dirk Stroobandt – 2007-04-27


pag. 65

Overview













pag. 66

Guiding the searchspace exploration• Guiding loop transformations by profiling: SLO

• Systematic design space exploration by application specific 'subsequences'

• Static analysis: cost functions that can be reduced to counting problems



pag. 68

Application specificsubsequences• Example on IDWT

– Vertical tiling (tile V)

– Horizontal tiling (tile H)

– From row-column-based to line-based (RC2LB)



pag. 69

Example: DWT/IDWT

• Original algorithm



pag. 70

Example: DWT/IDWT

• Vertical tiling (tile V)

• Horizontal tiling (tile H)



pag. 71

Example: DWT/IDWT

• From row-column-based to line-based (RC2LB)



pag. 72

Example: DWT/IDWT

• Primary transformations expressed as a sequence of elementary transformations

Str

etch

Inte

rcha

nge

Fus

ion

Shi

ftS

trip

-min

eP

eelin

gP

eelin

g +

dup

licat

ion

Ske

wA

dd C

onst

rain

ts

RC2LB 1 2 3 4Tile V 2 4 3 1Tile H 4 6 2,5 3 1Overlap 4 3 2 1 5



pag. 73

Example: DWT/IDWT

• Combining Primary transformations to build tree of variants



pag. 74

Example: DWT/IDWT



pag. 75

Example: DWT/IDWT



pag. 76

Guiding the searchspace exploration• Static analysis: cost functions that can be reduced to counting problems

• e.g. 1-D scheduling: number of living elements, parallelism



pag. 77

Guiding the searchspace exploration• Counting problems: solution is quasi-polynomial

• Often the maximum over a polyhedral domain is needed

• Finding bounds on quasi-polynomials over discrete domains

– Extension on bounds on polynomials over

continuous domains



pag. 78

Conclusions

• Memory bottleneck -> Loop transformations

– SW-techniques can be reused for HW

– Polyhedral model eases transformations

• Design automation

– CLooGVHDL: hardware generation from the

polyhedral model



pag. 79

Future work

• Methodology for finding transformations/schedule

• Introduce parallelism in techniques

• Methods for building memory hierarchy and scheduling memory transactions



pag. 80

Future Work

• Estimation methods for

– chip area

– execution time

– power

• Design Space exploration

• Automation of Synthesis methods

– Parallelism extraction

– Memory architecture with corresponding

parallelism



pag. 81

Acknowledgements

• Thanks to all members of my research group, especially:

– Mark Christiaens

– Hendrik Eeckhaut

– Harald Devos

– Kristof Beyls

• Info: [email protected]

overview the road to automatic - users.elis.ugent.be

Documents