infini band presentation_shekhar
DESCRIPTION
TRANSCRIPT
INFINIBAND
Presented By- Shekhar Kumar 074, CS-’B’Guided By- Latha R. Nair
Contents
IntroductionI/O Architectures- Fabric vs BusTechnical OverviewInfiniBand LayersInfiniBand ArchitectureAdvantages & DrawbacksMarketsConclusionReferences
Introduction
A switch-based serial I/O interconnect architecture
Operates at a base speed of 2.5 Gb/s or 10 Gb/s in each direction (per port)
Provides both QoS (Quality of Service) and RAS (Reliability, Availability and Serviceability)
Used in high performance computing and data centres
I/O Architectures- Fabric vs Bus
Feature Fabric Bus
Topology Switched Shared Bus
Pin Count Low High
Number Of End Points
Many Few
Max Signal Length KMs Inches
Reliability Yes No
Scalable Yes No
Fault Tolerant Yes No
I/O Architectures Contd…Shared Bus Architecture
CPU
System Controller
I/O Controller
I/O Controller
I/O Controller
I/O Controller
I/O Controller
PCI to PCI Bridge
PCI to PCI Bridge
System Memory
I/O Architectures Contd…Switched Fabric Architecture
Technical OverviewSwitch-based point-to-point
interconnect architectureEach individual link is based on a four-
wire 2.5Gb/s bidirectional connection.Architecture defines a layered
hardware protocol (Physical, Link, Network, Transport Layers) as well as a software layer
Each link can support multiple transport services for reliability
InfiniBand Feature Set
Layered Protocol- Physical, Link, Network, Transport, Upper Layers
Packet based communicationQuality Of ServiceThree link speeds1. 1X- 2.5 Gb/s, 4 Wire2. 4X- 10 Gb/s, 16 Wire3. 12X- 30 Gb/s, 48 WirePCB, Copper and Fiber cable
interconnectMulticast and Unicast support
InfiniBand Layers
InfiniBand Layers Contd…1) Physical LayerDefines both electrical and
mechanical characteristics for the system
Includes cables and receptables for fibre and copper media, backplane connectors
Defines three link speeds, 1X,4X,12X
Each individual link is a 4-wire differential connection that provides a full duplex connection at 2.5 Gb/s
InfiniBand Layers Contd…2) Link LayerEncompasses packet layout, point-to-point
operations, and switching within a local subnet
Packets- Two types: Management and Data packets
1. Management Packets- Used for link configuration and maintenance
2. Data Packets- Carry up to 4k bytes of a transaction payload
Switching- Devices within a subnet have a 16-bit Local ID assigned by the subnet manager. This LID is used for addressing.
InfiniBand Layers Contd…
Link LayerQoS- supported through virtual lanes(VL) VLs are separate logical communication
links which share a single physical linkData Integrity Two CRCs per packet, Variant CRC and
Invariant CRC16-bit VCRC includes all fields in the
packet and is recalculated at each hop32-bit ICRC covers only the fields that do
not change from hop to hop
InfiniBand Layers Contd…
Network LayerHandles routing of packets from one subnet
to another(within a subnet a network layer is not required)
Packets contain a global route header(GRH)GRH contains the 128-bit IPv6 addressTransport LayerResponsible for in-order packet delivery,
channel multiplexing and transport servicesAlso handles transaction data segmentation
when sending, and reassembly when receiving
InfiniBand Architecture
InfiniBand Architecture Contd…
Defines multiple devices for system communication: channel adapter, switch, router, and a subnet manager
At least one channel adapter is required for each end node
Subnet Manager sets up and maintains link
Channel adapters and switches contains Subnet Management Agent(SMA) for handling communication with the subnet manager
Channel AdaptersConnects InfiniBand to other devicesTwo types: Host Channel
adapter(HCA) and Target Channel Adapter(TCA)
HCA- Provides interface to a host device and supports all software verbs defined by InfiniBand
TCA- Provides connection to an I/O device from InfiniBand with a subset of HCA features necessary for specific operations of each device
Switches and Routers
Switches and Routers Contd…Switch Contains more than one InfiniBand port and
forwards packets from one of its port to another based on the LID
Can be configured to forward either unicast packets or multicast packets
Can consume or generate management packets
Router Forwards packets from one subnet to another
without consuming or generating packets Reads the Global Route Header to forward the
packet based on IPv6 address
AdvantagesSuperior performanceLow-latencyHigh-efficiencyFabric consolidation and low
energy usageReliable, stable connectionsData IntegrityHighly interoperable environment
DrawbacksComplex in designFew platforms supports it as yetBleeding edge, for now, so users
will need to perform extensive testing
MarketsApplication ClusteringInter-Processor
Communication(IPC)Storage Area Networks
CONCLUSIONGreater processor performanceSupports RASWorks both in-the-box and
enables Bandwidth Out of the Box
Supports scalability
REFERENCES1. InfiniBand Trade Association
www.infinibandta.org 2. InfiniBand Network Architecture,
MindShare, Inc. By Tom Shanley3. InfiniBand Architecture, Development and
Deployment: A strategic guide to server I/O solutions By William T. Futral
4. Qos in InfiniBand Subnetworks: Alfaro,F.J.; Sanchez, J.L.; Duato, J.; Department de Informatica, Univ. de Castilla-La Mancha, Albacete, Spain
THANK YOU
Queries???