introduction to interconnection networks. introduction to interconnection network digital...

Introduction to Interconnection Networks

Introduction to Interconnection network

• Digital systems(DS) are pervasive in modern society.

• Digital computers - simulating physical systems, managing large databases , preparing documents and etc

• Audio and video entertainment is increasingly being delivered and processed in digital form.

Finally, almost all products from automobiles to home appliances are digitally controlled.

Digital Systems

Communication (Moves Data)

Memory(Stores Data)

Logic(Transfers and combines

Data)

Digital Systems

Logic(Transfers and combines

Data)

Memory(Stores Data)

Communication (Moves Data)

Interconnection Network

• Performance – limited by communication/interconnection not logic/memory

• Power - most of the clock cycle spent on wire delays not gate delay.

• Technology improves, memories and processors become small, fast, and inexpensive.

• Also, the frequency of communication between components is lagging far beyond the clock rates of modern processors.

• These factors combine to make interconnection the key factor in the success of future digital systems.

Interconnection Network

ApplicationAlgorithmProgramming LanguageOperating SystemInstruction Set Architecture Micro-architectureRegister Transfer Level

CircuitsDevicesTechnology

Com

pute

r Arc

hite

ctur

e Logic Mem MemLogic

Logic MemLogicMem

Interconnection Network

Network Scenario• “On-chip networks may deliver data between

memory arrays, registers, and arithmetic units within a single processor “

Why Networks ?

Computing demandsProcessors requirements

What is the Solution ?

Three question about Interconnection Networks

What is an Interconnection Network ?

An Interconnection Network is a programmable system that transports data between terminal.

Where do you find Interconnection Network ?

• They are used in almost all digital systems that are large

enough to have two components to connect.

• The most common applications computer systems and

communication switches

• Computer System - they connect processors to memories and

input/output (I/O) devices to I/O controllers

• Communication Switches - They connect input ports to output

ports

Why are interconnection networks important ?

• Because they are a limiting factor in the performance of

many systems

• The interconnection network between processor and

memory largely determines the memory latency and

memory bandwidth, two key performance factors, in a

computer system of many systems

• The performance of the interconnection network in a communication

switch largely determines the capacity (data rate and number of

ports) of the switch

Why on-chip networks ?• They provide external connectivity from system to outside

world Also, connectivity within a single computer system at

many levels I/O units , chips, modules and blocks inside chips

• Trends: high demand on communication bandwidth Increased computing power and storage capacity Switched networks are replacing buses

• Integral part of many-core architectures Energy consumed by communication will exceed that of

computation in future systems

NoC: A paradigm Shift in VLSI

s

s

s

s

s s

s

s

Module

Module

s

Module

From: Dedicated signal wires To: Shared network

Network switch

Computing Module

Point to Point Link

Processingelement

NetworkInterface

Router

Inputbuffers

Unidirectionallinks

NoC Architecture

Perspective 1: NoC vs. Bus

• Aggregate bandwidth grows

• Link speed unaffected by N• Concurrent spatial reuse• Pipelining is built-in• Distributed arbitration

However:• No performance guarantee• Extra delay in routers• Area and power overhead?• Modules need NI • Unfamiliar methodology

Bandwidth is shared Speed goes down as N

grows No concurrency Pipelining is tough Central arbitration

However: Fairly simple and familiar

NoC Bus

A B

E

C

D

Shared Bus

Perspective 2: NoC vs. Off-chip Networks

• Cost is in the links• Latency is tolerable• Traffic/applications

unknown• Changes at runtime• Adherence to

networking standards

Sensitive to cost: area power

Wires are relatively cheap

Latency is critical

Traffic may be known a-priori

Design time specialization

Custom NoCs are possible

Example:

Off-Chip NetworksNoC

M odule

M odule M odule

M odule M odule

M odule M odule

M odule

M odule

M odule

M odule

M odule

M odule

M odule M odule

M odule M odule

M odule M odule

M odule

M odule

M odule

M odule

M odule

Performance Metrics:

• Packet Latency

• Effective Bandwidth

Terms and Definitions:

• Bandwidth: Maximum rate at which information can be transferred (including packet header, payload and trailer)

Unit: bits per second (bps) or bytes per second (Bps)

• Time of flight: Time for first bit of a packet to arrive at the receiver

Includes the time for a packet to pass through the network, not including the transmission time Unit: Picoseconds (OCNs), nanoseconds (SANs), microseconds (LANs), milliseconds (WANs)

• Transmission time: The time for a packet to pass through the network, not including the time of flight

Equal to the packet size divided by the data bandwidth of the link

• Transport latency:Sum of the time of flight and the transmission time

Measures the time that a packet spends in the network

• Sending overhead (latency):Time to prepare a packet for injection, including hardware/software

A constant term (packet size) plus a variable term (buffer copies)

• Receiving overhead (latency):Time to process an incoming packet at the end node

A constant term plus a variable term Includes cost of interrupt, packet reorder and message reassembly

Receiver

SenderSending

overheadTransmission time(bytes/bandwidth)

Time offlight

Transmission time(bytes/bandwidth)

Receivingoverhead

Transport latency

Total latency

Time

Latency = Sending Overhead + Time of flight + + Receiving Overhead packet size

Bandwidth

Example (latency): calculate the total packet latency for interconnect distances of 0.5 cm, 5 m, 5,000 m, and 5,000 km

Assume a dedicated-link network with8 Gbps (raw) data bandwidth per linkDevice A sends 100-byte packets (header included)

OverheadsSending overhead: x + 0.05 ns/byteReceiving overhead: 4/3(x) + 0.05 ns/byte

x is 0 μs for OCN, 0.3 μs for SAN, 3 μs for LAN, 30 μs for WANAssume time of flight consists only of link propagation delay (no other sources of delay)

int. networkint. network

Device ADevice A Device BDevice B

8 Gbps raw data bandwidth per link

Cr

int. networkint. network

Device ADevice A Device BDevice B

8 Gbps raw data bandwidth per link

Cr

LatencyOCN = 5 ns + 0.025 ns + 100 ns + 5 ns = 110.025 ns

LatencySAN = 0.305 μs + 0.025 ns + 0.1 μs + 0.405 μs = 0.835 μs

LatencyLAN = 3.005 μs + 25 μs + 0.1 μs + 4.005 μs = 32.11 μs

LatencyWAN = 20.05 μs + 25 μs + 0.1 μs + 40.05 μs = 25.07 ms

Latency = Sending overhead + Time of flight +Packet sizeBandwidth + Receiving overhead