load balancing switch

1

LOAD BALANCING SWITCH

By: Oleg Schtofenmaher

Maxim FudimSupervisor: Walter Isaschar

Final presentation for project

Winter 2007 ( Part A)

2

General overview

Software solutions for real-time are too slow

Power dissipation limits work frequencies

Greater computing power neededH/W accelerators can improve S/W

processesMulti-core, multi-threaded systems

are the future

3

Multiprocessor environment for parallel processing of vectors data stream

Maximal ThroughputConfigurable hardwareStatistics reportExpandable design

Project Goals

4

System specifications

1M pulse/sec data streamVectors of 8 ÷ 1024 pulses1K ÷ 125K vectors/secVariable number of processorsSystem span over multiple FPGAs

5

Problem

How to manage Data stream? How to manage multiple parallel units? How to achieve full and effective

utilization of resources?

6

Solution

Load Balancing SwitchConverting shared resources to

“personal” work space.FCFS for input, RR for routing/outputSmart management of systemMonitoring for each unit’s load

7

System Block diagram

Input vectorsLoad Balancing

Switch

(LBS)

Output reports

Statistics reports NIOS

VPU

S/W or H/W

generator

S/W or H/W

consumer

DDR2 Bank A

Data and Control

Stratix II FPGAPROCStar II

DDR2 Bank B

NIOS VPU

PCI

Organization of VPU’s(Vector Processing Units)

NIOS VPUs joined into the clustersConstant number of ClustersVarious number of NIOS VPU’s in

clusterVariable configuration of NIOS Different Priority for different

clusters

9

System Top Diagram

Input vectors

Load Balancing

Switch

(LBS)

Output reports

DDR2 Bank A

Stratix II FPGAPROCStar II

DDR2 Bank B

Gidel’s FIFO

control IP

Data flow

NIOScluster

NIOScluster

NIOScluster

NIOScluster

NIOScluster

NIOScluster

NIOScluster

NIOScluster

NIOScluster

NIOScluster

NIOScluster

NIOScluster

NIOScluster

NIOScluster

NIOScluster

NIOScluster

10

LBS Top Level ViewP

CI

Main Controller

unit

Stratix II FPGA

Output Writer

Cluster ArbiterNIOS II Syste

m

Input Reader

Cluster ArbiterNIOS II Syste

m

Control

Control

FIFO Input Port

FIFOOutput

Port

Control

Cluster ArbiterNIOS II Syste

mMuxed output data bus

Input data bus

Controland Status

Statistics

Reporter

11

System Interfaces

Software to Hardware Interface:

Input and Output MultiFIFO PCI data bus

MultiFIFO status2x32-bit general read purpose

registers2x32-bit general write purpose

registers8-bit information registerSoftware reset signal

12

Input System Interface

LBS Input Interface:64 bit data bus from Input MultiFIFORead request and ack. SignalsMultiFIFO status flagsSW/HW input signals

13

Output System Interface

LBS Output interface:64 bit data bus to Output MultiFIFOWrite request and ack. SignalsMultiFIFO status flagsSW/HW input signals

14

Data Packet Format

Header Data 1 to N of 32-bit

Words

Tail

……

Unused

Nios Numb

er

Data Length N

Vector ID/Command Type

8-bit 32-bit16-bitVersion 4-bit

SW/HW Control 1-bit

Type 1-bit(Data/

Command)

Tail : Sync Data or Checksum(in the future)

Header:

15

NIOS Input Interface

Hardware:64-bit input data bus – from LBS10 bit data slices counter – from

LBSWrite request signal – from LBSChip select signal – from LBSNIOS ready signal – from NIOSData ready signal – from LBS

16

NIOS Output Interface

Hardware:64 bit output data bus – from NIOS7 bit data slices counter – from LBSRead request signal – from LBSChip select signal – from LBSOutput ready signal – from NIOSOutput taken signal – from LBS

17

Twin VPU SystemInput / Output waveform

18

System Demonstration

19

LBS Units DescriptionInput ReaderReading data from input FIFOWriting data to selected clusterProviding header control bits for

main controllerSynchronization checksVector length counter

Main Controller unit

Output

Writer

Cluster

Arbiter

NIOS II

System

Input Reade

r Cluster

Arbiter

NIOS II

System

FIFO

Input

Port

FIFOOutput

Port

Cluster

Arbiter

NIOS II

SystemMuxed output data

bus

Input data bus

Controland Status

Statistics

Reporter

20

Input Reader Diagram

21

LBS Units DescriptionInput Controller - FSM

22

LBS Units DescriptionOutput WriterReading data from selected

clusterWriting data to output FIFOVector length counter

Main Controller unit

Output

Writer

Cluster

Arbiter

NIOS II

System

Input Reade

r Cluster

Arbiter

NIOS II

System

FIFO

Input

Port

FIFOOutput

Port

Cluster

Arbiter

NIOS II

SystemMuxed output data

bus

Input data bus

Controland Status

StatisticsReporter

23

Output Writer Diagram

24

LBS Units DescriptionOutput Controller - FSM

25

LBS Units DescriptionMain Controller

Enabling input and output unitsSelecting control source (S/W or

H/W)Monitoring clusters’ load via

status busesSelecting clusters for input/output

operationsData validity indication

Main Controller unit

Output

Writer

Cluster

Arbiter

NIOS II

System

Input Reade

r Cluster

Arbiter

NIOS II

System

FIFO

Input

Port

FIFOOutput

Port

Cluster

Arbiter

NIOS II

SystemMuxed output data

bus

Input data bus

Controland Status

StatisticsReporter

26

Main ControllerStatus Decoders

27

Status input and output independent decoders

Dynamic port mappingAlways selecting next active

neighborSuits “similar NIOSes” designTo be expanded in part B

LBS Units DescriptionMC Status Alghoritm

28

LBS Units DescriptionMC Status Alghoritm

0011

1314

013

02

114 015

.

.

.

13|14

.

.

. 013|4

02|15

1 1|0

00|1

114|3015|2

14|13

13|14

.

.

.

11|0

114|3

14|13

13

Status input

Dynamic port

mapping

Compare Active

ports

Next port

29

Decoding Flow

30

LBS Units DescriptionStatistics Reporter Monitoring system activity Error reporting for software Counting processed vectors Throughput = Vectors served / Time

of service To be expanded in part B

Main Controller unit

Output

Writer

Cluster

Arbiter

NIOS II

System

Input Reade

r Cluster

Arbiter

NIOS II

System

FIFO

Input

Port

FIFOOutput

Port

Cluster

Arbiter

NIOS II

SystemMuxed output data

bus

Input data bus

Controland Status

StatisticsReporter

31

Cluster parametric enablingCluster controllerWatchdogNIOS System

LBS Units DescriptionCluster Entity

Main Controller unit

Output

Writer

Cluster

Arbiter

NIOS II

System

Input Reade

r Cluster

Arbiter

NIOS II

System

FIFO

Input

Port

FIFOOutput

Port

Cluster

Arbiter

NIOS II

SystemMuxed output data

bus

Input data bus

Controland Status

StatisticsReporter

32

LBS Units DescriptionCluster Structure

33

Input 4-phase REQ/ACK protocol with NIOSNios ReadyData Ready

Output 4-phase REQ/ACK protocol with NIOSOutput ReadyOutput Taken

Smart Status Reporter

LBS Units DescriptionCluster Controller

34

LBS Units DescriptionCluster Controller

35

Cluster Input FSM

36

Cluster Output FSM

37

SOPC components: Input Vector Output VectorNios IIOn-chip memoryTimer

LBS Units DescriptionExample for NIOS System

38

Export signals from LBS 64-bit data Nios/Data Ready Address , Chipselect , Write

request ,Clock , Reset On-chip memory for 1024 32-bit words Avalon slave data port for 32-bit data

to NIOS II Avalon slave data ready port

LBS Units Description Input vector

39

LBS Units Description Input vector component

40

Avalon slave 32-bit data output port from NIOS II

Avalon slave output ready port On-chip memory for 128 32-bit words Export signals to LBS

64-bit data Output Ready / Taken Address , Chipselect , Read

request ,Clock , Reset

LBS Units Description Output vector

41

LBS Units Description Output vector component

42

Resource Usage

Unit ALUTs Memory (bits)

Percent (out of system)

Peripheral IPs (MegaFIFO, PLLs, etc.)

~ 1,600 32,768 ~ 20%

User System ~ 6,450 400,640

~ 80%

Single VPU( 5 in system) ~ 1,177 80,128 ~ 15%(73%)

LBS Logic ~ 560 0 ~ 7%

Total usage of chip resources

~ 8,000 433,408

17% ( of chip )

Total available 48,352 2,544,192

Resource usage data for 5 VPU system

VPU resource usage is based on basic NIOS’s with no accelerators and will only increase when accelerators will be introduced.

43

Tasks

Study PROCStar Board – Done Study Altera’s Stratix II FPGA – Done Study Quartus and HDL designer– Done Study GIDEL API – Done Learn to use Signal Tap tool – Done Study Altera’s NIOS II – Done Define interface with software group –

Done Develop signal generator for testing –

Done

44

Tasks (cont.)

Define interface with accelerator group – Done

Build direct connection with s/w and NIOS II – Done

Expand design for several NIOS’s – Done Define basic algorithm for h/w switching –

Done Implementation and debugging of the

switch – Done Integration with NIOS system – Done SW Test application for operating and

integration with hardware design – Done Integration of LBS with other groups

45

Summary

LBS implementation with SW/HW control and statistics

Up 16 NIOS’s connected to the system Fully functional S/W – LBS , LBS – NIOS

interface Successful hardware and software

integration Working design examples for other teams

46

Conclusions

Switch concept implemented successfully

Vector transit time is queue and processing only

Two layer abstraction concept = minimize changes

Single level of mastering = minimize resources

64-bit buses = maximize throughput

47

Tasks for Part B

Increase number of Nios’s in clusters Improve algorithm for priority cluster

selection Expand statistic reports Expand SW/HW communication Add error correction/handling Add smart vector queue management

(SJT) Spread design to several FPGAs Multiple Stage LBS ???