chapter 2, server motherboard & chipset design
TRANSCRIPT
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
1/47
Confidential
Server Fundamentals
The performance of a server depends on the speed of data flow
between its processor and the peripheral device that are connected.
These peripheral device use a set of pathways to communicate with
the processor. This set of pathways are called as BUS. In server wemainly use PCI bus technology.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
2/47
Confidential
Types of Buses
System Bus
Data Bus
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
3/47
Confidential
System Bus
Use to connect CPU to its MEMORY.
Also known as Local Bus
Faster than other Buses.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
4/47
Confidential
Data Bus
Connects the other computer peripherals with the memory.
PCI is an example of data bus.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
5/47
Confidential
Usage of PCI
In the early 1990, INTEL introduce PCI with the aim to improve I/O
operation of computers.
PCI is also used with other processors, eg. AMD
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
6/47
Confidential
PCI Design
Initially PCI was a 32 bit bus that operated a speed of 33 MHz
(DTR=132Mbps)
Now 64 bit PCI with a speed of 33 MHz whose DTR=264 Mbps
64 bit PCI with a speed of 66 MHz, with DTR=512 mbps
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
7/47Confidential
Bus Speed Setup
Synchronized--- bus speed is not depends with the memory bus
speed.
Mainly used in today's bus architecture
Asynchronized--- bus speed depends with the memory bus speed.
Normally controlled by jumpers or CMOS setup.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
8/47Confidential
PCI Implementation
Data buses connect to system buses by single chip bridge device.
PCI can be implemented on servers through peer PCI bus or
hierarchical PCI bus technology.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
9/47Confidential
Peer PCI Bus
In Peer PCI, several PCI buses are connected directly to a host bus. Thesehost buses provide connectivity between server and storage devices.
This enables the CPU to access each bus directly.
Peer PCI improves the performance of a server because the processor orthe bus bandwidth is evenly shared between the buses.
Also helps to separate high bandwidth and low bandwidth traffic on different
PCI buses.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
10/47Confidential
Hierarchical PCI
When multiple expansion slots and connectors are needed
hierarchical PCI can be implemented.
A bridge controller is used to connect independent PCI buses so
that they can communicate with each other.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
11/47Confidential
Hot plug
The hot plug facility of PCI enables to install and remove PCI device
without turning off the server.
Today most servers are equipped with the hot plug facility.
Microsoft Windows Server 2003 have native support for hot plug
operations.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
12/47Confidential
Hot Plug Devices
NIC Card
SCSI Card
RAID Card
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
13/47Confidential
Hot Swap
Cards inserted or remove without failure when servers are running.
To support hot swap PCI hot swap adapters are required. Thesedevice provide circuit breaker functions that protect electric suppliesagainst fault conditions resulting from over current. A PCI hot swapadapter provides protection against 12V, 5V, 3.3V and -12V
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
14/47
Confidential
PCI- Hot Swap Levels
Basic hot swap
Full hot swap
High availability
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
15/47
Confidential
Basic Hot Swap VS Full Hot Swap
Requires human intervention for interaction with the host operating
system when a card is being inserted or removed.
It automates the procedure of replacing the card.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
16/47
Confidential
High Availability
The high availability level focuses on having cards that can belogically and electrically added to or removed from a server.
This ensure hot spare or fail over capability for the server. The highavailability hot swap level also enables an automatic reconfigurationof software and hardware components in a running server.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
17/47
Confidential
I2O
PCI is also intelligent input/output (I2O) compliant.
I2O is a standard designed by a consortium of computer companies called asI2O Special Interest Group (SIG). This standard simplifies and speed up the
I/O operations on servers.
According to I2O the drivers used for the operating system of a server must be sameas those used for the SCSI and network cards on the server.
It is recommended to use INTEL 960 chip on the motherboard of a server, to increasethe speed of its I/O subsystem.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
18/47
Confidential
PCI- Bus Mastering
Bus mastering is the capability of devices other than the system chipset, on
the PCI bus to take control of the bus and perform transfer directly.
Bus mastering allows the transfer of data between devices on a PCI bus
and the main system memory even when the other devices are using CPU.
However these devices should have their own built-in processors to operate
independently of the CPU.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
19/47
Confidential
MULTITASTKING
Bus Mastering facilitates multitasking by allowing data transfer
between the devices on a PCI bus and other devices in the system
without involving the CPU.
The devices that take control of the bus keep track of the
transactions. There fore the CPU utilization is low. This enables the
CPU to handle other task in the meantime.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
20/47
Confidential
Bus Arbitration
Bus Mastering is achieved through the bus arbitration process. This process
determines priority levels for the request to a PCI bus so that no devices
lock each other out.
For example, when more than one processor is connected to the same
system bus, there are simultaneous requests for the bus. This request are
resolved using bus arbitration that allows devices to use the bus based on
their priority level.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
21/47
Confidential
PCI-e
The Peripheral Component Interconnect Express architecture is a
serial interconnect technology designed to meet the bandwidth
requirement of advanced processors, memory subsystem and
applications.
It impose some advance features than a normal PCI.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
22/47
Confidential
PCI-e
PCI-e is a point-to-point serial switch bus technology. It aims at reducing
implementation costs and providing higher bandwidth per pin and a scalable
performance.
The advantage of PCI-e over PCI is its point to point bus topology. It
enables each PCI-e device to communicate the PCI controller directly and
autonomously. This communication occurs through a PCI link.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
23/47
Confidential
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
24/47
Confidential
PCI-e link
A PCI-e link comprise of one or more lanes. Depending on the
number of lanes used in a PCI-e link the link is called as x1 link, x2
link and so on. A link which made up for 4 lanes is known as x4 link.
PCI-e supports x1,x2,x4,x8,x12,x16 and x34 lane widths.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
25/47
Confidential
PCI-e Bandwidth
Each PCI-e lane can transfer one byte at a time in both the
directions by using full duplex communication. An individual lane
can transfer data at the rate of 2 Gbps in each direction
simultaneously. The next table shows its detail.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
26/47
Confidential
PCI-e bandwidth
PCI-e implementation Encoded data transferrate
Unencoded data transferrate
x1 5 gbps 4 gbps
x2 10 gbps 8 gbps
x4 20 gbps 16 gbps
x8 40 gbps 32 gbps
x16 80 gbps 64 gbps
x32 160 gbps 128 gbps
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
27/47
Confidential
8b/10b Clock-encoding mechanism
A PCI-e mechanism comprises dual simplex channels, where eachchannel is composed of a transmit pair and a receive pair forconcurrent transmission in both directions. Every pair is composedof dual pairs of signal that are driven by Low Voltage Differential(LVD). To attain a high speed data transfer rate, a data clock is built
into each pair by using an 8b/10b data encoding scheme.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
28/47
Confidential
Point to Point Bus Topology of PCI-e
As PCI-e supports the point to point bus topology the total
bandwidth of each PCI bus is dedicated to the device at the end of
the link. This in turn enables multiple PCI-e devices to be active at
the same time without affecting each other.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
29/47
Confidential
Advanced Features of PCI-e
Advanced power management
Support for real-time data traffic
Hot plug and hot swap
Data integrity and error handling
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
30/47
Confidential
Advanced Power Management
PCI-e maintains an active interface constantly to synchronize thetransmitter and receiver. The interface is kept active continuallytransmitting idle characters when data is not sent. As additionalpower is consumed by the receiver to decode and discard these idlecharacters, PCI-e uses two low-power link states and the active-
state power management (ASPM) protocol.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
31/47
Confidential
Support for Real-Time Data Traffic
PCI-e provides native support for isochronous data transfers and different
Quality of Service (QoS) levels through virtual channels. These channels
ensure timely and reliable delivery of data packets. PCI-e supports
numerous isochronous virtual channels per lane, which may have varying
QoS levels. This feature is developed for applications that require real-time
delivery such as real-time audio and video.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
32/47
Confidential
Hot Plug and Hot Swap
The hot plug and hot swap features of PCI-e are extremely
beneficial to servers and portable computers. The facility to hot plug
I/O devices reduces server downtime when peripheral cards need to
be installed or replaced.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
33/47
Confidential
Data Integrity and Error Handling
PCI-e provides support for link-level data integrity for all kinds of
transactions and data-link packets. As a result, it provides end-to-end data
integrity required by high availability application running on server systems.
Additionally, fault isolation and recovery solutions can be enhanced by
using the PCI error handling and advanced error reporting and handling
capabilities of a PCI-e.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
34/47
Confidential
PCI-X
PCI-X was developed jointly by IBM, HP and Compaq and was
approved by PCI Special Interest Group(PCI-SIG). This group
provides technical support, training and compliance testing. Now
PCI-X has become an open standard that is used by all computermanufactures.
C S
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
35/47
Confidential
PCI-X Data Speed
The PCI-X design, uses one 64 bit slot that runs at 133 MHz while
the other slots run at 66 MHz. This enables data to be exchanged at
the speed of 1.6 GBps.
So using PCI-X technology existing PCI bus can be increased from133 MBps to 1 GBps.
PCI X b k d tibilit
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
36/47
Confidential
PCI-X backward compatibility
PCI-X is backward compatible. It means, a PCI-X card can be
installed in a standard 32 PCI slot. However, this reduces the speed
of the bus to 33 MHz
PCI X F lt T l
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
37/47
Confidential
PCI-X Fault Tolerance
A PCI-X system is more fault tolerant than a Normal PCI system.
When a faulty card is installed on a computer, the PCI-X bus can re-
initialize the faulty card or take it offline before the computer fails.
PCI X i t
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
38/47
Confidential
PCI-X power requirements
Normal PCI----- 5 VOLT
PCI-X ----- 3.3 VOLT
C i f PCI d PCI X S t
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
39/47
Confidential
Comparison of PCI and PCI-X Systems
Bus Width Bus Frequency PCI/PCI-X
bandwidth
No. of slots(PCI) No. of slots(P
X)
32 bit 66 MHz 264 Mbps 4 N/A
64 bit 66 MHz 528 MBps 2 4
64 bit 100 MHz 800 MBps N/A 2
64 bit 133 MHz 1064 Mbps N/A 1
Ad t T i
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
40/47
Confidential
Adapter Teaming
Adapter Teaming is a process of combining two or more physical
adapters into a single logical adapter. This enables the network
administrator to assign a single IP address to the logical adapter.
A software driver is usually used to link the physical adapter to thelogical adapter.
Ad t T i b f b
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
41/47
Confidential
Adapter Teaming- number of members
An adapter team can have a minimum of two and a maximum of
eight adapters. The main advantage of a adapter teaming is that it
increases server uptime. This is because if a physical adapter in a
team fails, it does not affect other adapters in the team as the IPaddress is assigned to the logical adapter and not to a physical
adapter.
AFT
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
42/47
Confidential
AFT
Adapter teaming also enables Adapter Fault Tolerance (AFT)against broken or loose or loose cables, hubs or switch ports.
It can also provide fault tolerance to problems caused by aPeripheral Component Interconnect (PCI) slot malfunction orbreakdown in adapter hardware.
Ad t T i l d di t ib ti
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
43/47
Confidential
Adapter Teaming load distribution
Adapter teaming ensures that the failure of an adapter in a team
does not affect the load distribution through failover capabilities. If
any member in an adapter team fails because of problems in a
network interface card (NIC), cable, switch port, or switch, the loaddistribution is evaluated and then reassigned to the other members
of the team.
Primar Adapter
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
44/47
Confidential
Primary Adapter
For an effective failover, a primary adapters must be specified in an
adapter team. The primary adapter is specified based on the
priorities of the adapters.
Usually, the primary adapter is the adapter that has highest prority
Failover Restoration
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
45/47
Confidential
Failover Restoration
In adapter teaming, only a single active adapter transmits and
receives traffic.
The single adapter is usually the primary adapter. However, if the
primary adapter fails, a secondary adapter takes control.
After the connection of the primary adapter is restored, the control is
automatically passed back to the primary adapter.
ALB
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
46/47
Confidential
ALB
Adapter teaming also enables Adapter Load Balancing (ALB). ALB provides
fault tolerance by ensuring load balancing data traffic.
When load balancing takes place, all adapters in an adapter team share the
transmission load.
To enable load balancing, all the adapters in the team should be connected
to a switch and an IP Address must be assigned to the adapter team.
-
8/12/2019 Chapter 2, Server Motherboard & Chipset Design
47/47
Thank you