interconnect complimentary slides

7/31/2019 Interconnect Complimentary Slides

1/56

n- pn- pCommunicationCommunication

ArchitecturesArchitectures

Networks-on-Chip

ICS 295Sudeep Pasricha and Nikil DuttSlides based on book chapter 12

1 2008 Sudeep Pasricha & Nikil Dutt


2/56

OutlineOutlineIntroductionNoC TopologySwitching strategies

Routing algorithmsFlow control schemesClocking schemesQoSNoC Architecture ExamplesStatus and Open Problems



3/56

IntroductionIntroductionEvolution of on-chip communicationarchitectures



4/56

IntroductionIntroductionNetwork-on-chip (NoC) is a packet switched on-chipcommunication network designed using a layeredmethodology

routes packets, not wiresNoCs use packets to route data from the source to thedestination PE via a network fabric that consists of

switches (routers)interconnection links (wires)



5/56

IntroductionIntroductionNoCs are an attempt to scale down theconcepts of largescale networks, and applythem to the embedded system-on-chip (SoC)domain

NoC Properties Regular geometry that is scalable Flexible QoS guarantees Higher bandwidth Reusable components

Buffers, arbiters, routers, protocol stack No long global wires (or global clock tree)

No problematic global synchronizationGALS: Globally asynchronous, locally synchronousdesign

Reliable and predictable electrical and physicalro erties 5 2008 Sudeep Pasricha & Nikil Dutt


6/56

IntroductionIntroductionISO/OSI network protocol stack model



7/56





8/56

NoC TopologyNoC TopologyDirect Topologies

each node has direct point-to-point link to a subsetof other nodes in the system called neighboringnodes

nodes consist of computational blocks and/ormemories, as well as a NI block that acts as a router e.g. Nostrum, SOCBUS, Proteo, Octagon as the number of nodes in the system increases,

the total available communication bandwidth also

increases fundamental trade-off is between connectivity and

cost



9/56

NoC TopologyNoC TopologyMost direct network topologies have anorthogonal implementation, where nodes canbe arranged in ann-dimensional orthogonal space

routing for such networks is fairly simple e.g. n-dimensional mesh, torus, folded torus,

hypercube, and octagon

2D mesh is most popular topology

all links have the same lengtheases physical design area grows linearly with the number

of nodes must be designed in such a way as to

avoid traffic accumulating in thecenter of the mesh 9 2008 Sudeep Pasricha & Nikil Dutt


10/56

NoC TopologyNoC Topology Torus topology, also called a k-ary n-cube, is ann-dimensional grid with k nodes in eachdimension

k-ary 1-cube (1-D torus) is essentially a ring network

with k nodeslimited scalability as performance decreases when morenodes

k-ary 2-cube (i.e., 2-D torus) topology issimilar to a regular mesh

except that nodes at the edges are connectedto switches at the opposite edge via wrap-around channelslong end-around connections can, however, 10 2008 Sudeep Pasricha & Nikil Dutt


11/56

NoC TopologyNoC TopologyFolding torus topology overcomes the longlink limitation of a 2-D torus

links have the same size

Meshes and tori can be extended by addingbypass links to increase performance at the

cost of higher area 11 2008 Sudeep Pasricha & Nikil Dutt


12/56

NoC TopologyNoC TopologyOctagon topology is another example of a directnetwork

messages being sent between any 2 nodes require atmost two hops

more octagons can be tiled together to accommodatelarger designsby using one of the nodes is used as a bridge node



13/56

NoC TopologyNoC TopologyOctagon topology is another example of a directnetwork

messages being sent between any 2 nodes require atmost two hops

more octagons can be tiled together to accommodatelarger designsby using one of the nodes is used as a bridge node



14/56

NoC TopologyNoC TopologyIndirect Topologies

each node is connected to an external switch, andswitches have point-to-point links to other switches

switches do not perform any informationprocessing, and correspondingly nodes do notperform any packet switching

e.g. SPIN, crossbar topologies

Fat tree topology nodes are connected only to the leaves of the tree more links near root, where bandwidth

requirements are higher



15/56

NoC TopologyNoC Topologyk-ary n-fly butterfly network

blocking multi-stage network packets may betemporarily blocked or dropped in the network if contention occurs

kn nodes, and n stages of k n-1 k x k crossbar e.g. 2-ary 3-fly butterfly network



16/56

NoC TopologyNoC Topology(m, n, r) symmetric Clos network

three-stage network in which each stage is made up of a number of crossbar switches

m is the no. of middle-stage switches

n is the number of input/outputnodes on each input/output switch r is the number of input and output

switches e.g. (3, 3, 4) Clos network non-blocking network expensive (several full crossbars)



17/56

NoC TopologyNoC TopologyBenes network

rearrangeable network in which paths may have to berearranged to provide a connection, requiring anappropriate controller

Clos topology composed of 2 x 2 switches e.g. (2, 2, 4) re-arrangeable Clos network constructed

using two (2, 2, 2) Clos networks with 4 x 4 middleswitches



18/56

NoC TopologyNoC TopologyIrregular or ad hoc network topologies

customized for an application usually a mix of shared bus, direct, and indirect

network topologies

e.g. reduced mesh, cluster-based hybrid topology



19/56





20/56

Switching strategiesSwitching strategiesDetermine how data flows through routers in thenetworkDefine granularity of data transfer and appliedswitching technique

phit is a unit of data that is transferred on a link in asingle cycle

typically, phit size = flit size



21/56

Switching strategiesSwitching strategies Two main modes of transporting flits in a NoCare circuit switching and packet switchingCircuit switching

physical path between the source and the

destination is reserved prior to the transmission of data message header flit traverses the network from the

source to the destination, reserving links along theway

Advantage: low latency transfers, once path isreserved

Disadvantage: pure circuit switching does not scalewell with NoC size

several links are occupied for the duration of thetransmitted data, even when no data is beingtransmitted 21 2008 Sudeep Pasricha & Nikil Dutt


22/56

Switching strategiesSwitching strategiesVirtual circuit switching

creates virtual circuits that are multiplexed on links number of virtual links (or virtual channels (VCs))

that can be supported by a physical link depends onbuffers allocated to link

Possible to allocate either one buffer per virtual linkor one buffer per physical link

Allocating one buffer per virtual linkdepends on how virtual circuits are spatially distributed

in the NoC, routers can have a different number of bufferscan be expensive due to the large number of sharedbuffersmultiplexing virtual circuits on a single link also requiresscheduling at each router and link (end-to-end schedule)conflicts between different schedules can make it 22 2008 Sudeep Pasricha & Nikil Dutt


23/56

Switching strategiesSwitching strategies Allocating one buffer per physical link

virtual circuits are time multiplexed with a single buffer perlinkuses time division multiplexing (TDM) to statically schedulethe usage of links among virtual circuits

flits are typically buffered at the NIs and sent into the NoCaccording to the TDM scheduleglobal scheduling with TDM makes it easier to achieve end-to-end bandwidth and latency guaranteesless expensive router implementation, with fewer buffers



24/56

Switching strategiesSwitching strategiesPacket Switching

packets are transmitted from source and make theirway independently to receiver

possibly along different routes and with different delays zero start up time, followed by a variable delay due

to contention in routers along packet path QoS guarantees are harder to make in packet

switching than in circuit switching three main packet switching scheme variants

SAF switching packet is sent from one router to the next only if

the receiving router has buffer space for entirepacket

buffer size in the router is at least equal to the sizeof a packet 24 2008 Sudeep Pasricha & Nikil Dutt


25/56

Switching strategiesSwitching strategiesVCT Switching

reduces router latency over SAF switching by forwardingfirst flit of a packet as soon as space for the entirepacket is available in the next router

if no space is available in receiving buffer, no flits aresent, and the entire packet is buffered

same buffering requirements as SAF switching

WH switching flit from a packet is forwarded to receiving router if

space exists for that flit parts of the packet can be distributed among two or

more routers buffer requirements are reduced to one flit, instead of

an entire packet more susceptible to deadlocks due to usage 25 2008 Sudeep Pasricha & Nikil Dutt


26/56





27/56

Routing algorithmsRouting algorithmsResponsible for correctly and efficiently routingpackets or circuits from the source to thedestinationChoice of a routing algorithm depends on trade-

offs between several potentially conflictingmetrics minimizing power required for routing minimizing logic and routing tables to achieve a lower

area footprint increasing performance by reducing delay and

maximizing traffic utilization of the network improving robustness to better adapt to changing traffic

needs

Routing schemes can be classified into several 27 2008 Sudeep Pasricha & Nikil Dutt


28/56

Routing algorithmsRouting algorithmsStatic and dynamic routing

static routing: fixed paths are used to transfer databetween a particular source and destination

does not take into account current state of the network advantages of static routing:

easy to implement, since very little additional routerlogic is requiredin-order packet delivery if single path is used

dynamic routing: routing decisions are madeaccording to the current state of the network

considering factors such as availability and load on links path between source and destination may change

over timeas traffic conditions and requirements of the applicationchange

more resources needed to monitor state of the 28 2008 Sudeep Pasricha & Nikil Dutt


29/56

Routing algorithmsRouting algorithmsDistributed and source routing

static and dynamic routing schemes can be furtherclassified depending on where the routing information isstored, and where routing decisions are made

distributed routing: each packet carries the destination

addresse.g., XY co-ordinates or number identifying destinationnode/routerrouting decisions are made in each router by looking up thedestination addresses in a routing table or by executing ahardware function

source routing: packet carries routing informationpre-computed routing tables are stored at a nodes NIrouting information is looked up at the source NI and routinginformation is added to the header of the packet (increasingpacket size)when a packet arrives at a router, the routing information is 29


30/56

Routing algorithmsRouting algorithmsMinimal and non-minimal routing

minimal routing: length of the routing path from thesource to the destination is the shortest possible lengthbetween the two nodes

e.g. in a mesh NoC topology (where each node can be

identified by its XY co-ordinates in the grid) if source node isat (0, 0) and destination node is at (i, j), then the minimalpath length is |i| + |j|source does not start sending a packet if minimal path is notavailable

non-minimal routing: can use longer paths if a minimalpath is not availableby allowing non-minimal paths, the number of alternativepaths is increased, which can be useful for avoidingcongestiondisadvantage: overhead of additional power consumption



31/56

Routing algorithmsRouting algorithmsRouting algorithm must ensure freedom fromdeadlocks

common in WH switching e.g. cyclic dependency shown below

freedom from deadlocks can be ensured byallocating additional hardware resources orimposing restrictions on the routing

usually dependency graph of the shared networkresources is built and anal zed either staticall or 31 2008 Sudeep Pasricha & Nikil Dutt


32/56

Routing algorithmsRouting algorithmsRouting algorithm must ensure freedom fromlivelocks

livelocks are similar to deadlocks, except thatstates of the resources involved constantly changewith regard to one another, without making any

progressoccurs especially when dynamic (adaptive) routing isusede.g. can occur in a deflective hot potato routing if apacket is bounced around over and over again betweenrouters and never reaches its destination

livelocks can be avoided with simple priority rules

Routing algorithm must ensure freedom fromstarvation

under scenarios where certain packets areprioritized during routing, some of the low priority 32 2008 Sudeep Pasricha & Nikil Dutt


33/56





34/56

Flow control schemesFlow control schemesGoal of flow control is to allocate networkresources for packets traversing a NoC

can also be viewed as a problem of resolvingcontention during packet traversal

At the data link-layer level, when transmissionerrors occur, recovery from the error depends onthe support provided by the flow controlmechanism

e.g. if a corrupted packet needs to be retransmitted, flow of packets from the sender must be stopped, and requestsignaling must be performed to reallocate buffer andbandwidth resources

Most flow control techniques can manage linkcongestion

But not all schemes can (by themselves) 34 2008 Sudeep Pasricha & Nikil Dutt


35/56

Flow control schemesFlow control schemes

STALL/GO low overhead scheme requires only two control wires

one going forward and signaling data availabilitythe other going backward and signaling either a condition of buffers filled (STALL) or of buffers free (GO)

can be implemented with distributed buffering(pipelining) along link

good performance fast recovery from congestion

does not have any provision for fault handlinghigher level protocols responsible for handling flit 35 2008 Sudeep Pasricha & Nikil Dutt


36/56

Flow control schemesFlow control schemes T-Error

more aggressive scheme that can detect faultsby making use of a second delayed clock at every buffer stage

delayed clock re-samples input data to detect anyinconsistencies

then emits a VALID control signal resynchronization stage added between end of link and

receiving switchto handle offset between original and delayed clocks

timing budget can be used to provide greater reliabilityby configuring links with appropriate spacing and 36 2008 Sudeep Pasricha & Nikil Dutt


37/56

Flow control schemesFlow control schemesACK/NACK

when flits are sent on a link, a local copy is kept in abuffer by sender when ACK received by sender, it deletes copy of flit from

its local buffer when NACK is received, sender rewinds its output queue

and starts resending flits, starting from the corrupted one implemented either end-to-end or switch-to-switch sender needs to have a buffer of size 2 N + k

N is number of buffers encountered between source anddestination

k depends on latency of logic at the sender and receiver 37 2008 Sudeep Pasricha & Nikil Dutt


38/56

Flow control schemesFlow control schemesNetwork and Transport-Layer Flow Control

Flow Control without Resource Reservation Technique #1: drop packets when receiver NI full

improves congestion in short term but increases it in longterm

Technique #2: return packets that do not fit into receiverbuffers to sender

to avoid deadlock, rejected packets must be accepted bysender

Technique #3: deflection routing

when packet cannot be accepted at receiver, it is sent backinto networkpacket does not go back to sender, but keeps hopping fromrouter to router till it is accepted at receiver

Flow Control with Resource Reservation

credit-based flow control with resource reservation 38 2008 Sudeep Pasricha & Nikil Dutt


39/56





40/56

Clocking schemesClocking schemesFully synchronous

single global clock is distributed to synchronizeentire chip

hard to achieve in practice, due to process variationsand clock skew

Mesochronous local clocks are derived from a global clock not sensitive to clock skew phase between clock signals in different modules

may differdeterministic for regular topologies (e.g. mesh)non-deterministic for irregular topologies

synchronizers needed between clock domains

Pleisochronous

clock signals are produced locally 40 2008 Sudeep Pasricha & Nikil Dutt


41/56

OutlineOutlineIntroductionNoC TopologySwitching strategiesRouting algorithmsFlow control schemesClocking schemesQoSNoC Architecture ExamplesStatus and Open Problems



42/56

Quality of Service (QoS)Quality of Service (QoS)QoS refers to the level of commitment for packetdelivery

refers to bounds on performance (bandwidth, delay, and jitter)

Three basic categories best effort (BE)

only correctness and completion of communication isguaranteedusually packet switchedworst case times cannot be guaranteed

guaranteed service (GS)makes a tangible guarantee on performance, in addition tobasic guarantees of correctness and completion forcommunicationusually (virtual) circuit switched

differentiated service 42 2008 Sudeep Pasricha & Nikil Dutt


43/56

OutlineOutlineIntroductionNoC TopologySwitching strategiesRouting algorithmsFlow control schemesClocking schemesQoSNoC Architecture ExamplesStatus and Open Problems



44/56

therealtherealDeveloped by PhilipsSynchronous indirect networkWH switchingContention-free source routing based on TDM

GT as well as BE QoSGT slots can be allocated statically atinitialization phase, or dynamically at runtimeBE traffic makes use of non-reserved slots,

and any unused reserved slots also used to program GT slots of the routers

Link-to-link credit-based flow control schemebetween BE buffers

to avoid loss of flits due to buffer overflow 44 2008 Sudeep Pasricha & Nikil Dutt


45/56

HERMESHERMES

Developed at the Faculdade de InformticaPUCRS, BrazilDirect network2-D mesh topologyWH switching with minimal XY routingalgorithm8 bit flit size; first 2 flits of packet containheaderHeader has target address and number of flitsin the packetParameterizable input queuing

to reduce the number of switches affected by ablocked packet 45 2008 Sudeep Pasricha & Nikil Dutt


46/56

MANGOMANGOMessage-passing Asynchronous Network-on-chipproviding GS over open core protocol (OCP)interfacesDeveloped at the Technical University of DenmarkClockless NoC that provides BE as well as GSservicesNIs (or adapters) convert between thesynchronous OCP domain and asynchronousdomainRouters allocate separate physical buffers forVCs

For simplicity, when ensuring GS

BE connections are source routed 46 2008 Sudeep Pasricha & Nikil Dutt


47/56

NostrumNostrumDeveloped at KTH in StockholmDirect network with a 2-D mesh topologySAF switching with hot potato (or deflective)routing

Support for switch/router load distribution guaranteed bandwidth (GB) multicasting

GB is realized using looped containers implemented by VCs using a TDM mechanism container is a special type of packet which loops around

VC multicast: simply have container loop around on VC

having recipients 47 2008 Sudeep Pasricha & Nikil Dutt


48/56

OctagonOctagon

Developed by STMicroelectronicsdirect network with an octagonal topology8 nodes and 12 bidirectional linksAny node can reach any other node with a max

of 2 hopsCan operate in packet switched or circuitswitched modeNodes route a packet in packet switched mode

according to its destination field node calculates a relative address and then packet is

routed either left, right, across, or into the node

Can be scaled if more than 8 nodes are required

Spidergon 48 2008 Sudeep Pasricha & Nikil Dutt


49/56

QNoCQNoCDeveloped at Technion in IsraelDirect network with an irregular mesh topologyWH switching with an XY minimal routing schemeLink-to-link credit-based flow control

Traffic is divided into four different service classes signaling, real-time, read/write, and block-transfer signaling has highest priority and block transfers lowest

priority every service level has its own small buffer (few flits) at

switch inputPacket forwarding is interleaved according to QoSrules

high priority packets able to preempt low priority

packets 49 2008 Sudeep Pasricha & Nikil Dutt


50/56

SOCBUSSOCBUS

Developed at Linkping UniversityMesochronous clocking with signal retiming isusedCircuit switched, direct network with 2-D meshtopologyMinimum path length routing scheme is usedCircuit switched scheme is

deadlock free requires simple routing hardware very little buffering (only for the request phase) results in low latency

Hard guarantees are difficult to give because it

takes a long time to set up a connection 50 2008 Sudeep Pasricha & Nikil Dutt


51/56

SPINSPIN

Scalable programmable integrated network(SPIN)fat-tree topology, with two one-way 32-bit linkdata paths

WH switching, and deflection routingVirtual socket interface alliance (VSIA) virtualcomponent interface (VCI) protocol to interfacebetween PEs

Flits of size 4 bytesFirst flit of packet is header

first byte has destination address (max. 256 nodes) last byte has checksum

Link level flow control 51 2008 Sudeep Pasricha & Nikil Dutt


52/56

XpipesXpipes

Developed by the Univ. of Bologna and StanfordUniversitySource-based routing, WH switchingSupports OCP standard for interfacing nodes withNoCSupports design of heterogeneous, customized(possibly irregular) network topologiesgo-back-N retransmission strategy for link levelerror control

errors detected by a CRC (cycle redundancy check) blockrunning concurrently with the switch operation

XpipesCompiler and NetChip compilers

Tools to tune parameters such as flit size, address spaceof cores max. number of ho s between an two network 52 2008 Sudeep Pasricha & Nikil Dutt


53/56

OutlineOutline

IntroductionNoC TopologySwitching strategiesRouting algorithmsFlow control schemesClocking schemesQoSNoC Architecture ExamplesStatus and Open Problems



54/56

Status and Open ProblemsStatus and Open ProblemsPower

complex NI and switching/routing logic blocks arepower hungry

several times greater than for current bus-basedapproaches

Latency additional delay to packetize/de-packetize data at NIs flow/congestion control and fault tolerance protocol

overheads delays at the numerous switching stages encountered

by packets even circuit switching has overhead (e.g. SOCBUS) lags behind what can be achieved with bus-

based/dedicated wiring

Lack of tools and benchmarks 54 2008 Sudeep Pasricha & Nikil Dutt


55/56

Trends Trends

Move towards hybridinterconnection fabrics NoC-bus based

Custom, heterogeneous topologies

New interconnect paradigms

Optical Wireless Carbon nanotube



56/56

interconnect complimentary slides

Documents