primergy rx900 s2 technical manual - fujitsumanuals.ts.fujitsu.com/file/10142/rx900s2-tm-en.pdf ·...

Technical Manual - English

Baseboard D3144, CPUMEMR D3145, PCIR D3146 for PRIMERGY RX900 S2 Technical Manual

Edition May 2011

Comments… Suggestions… Corrections…The User Documentation Department would like toknow your opinion of this manual. Your feedback helpsus optimize our documentation to suit your individual needs.

Feel free to send us your comments by e-mail to [email protected].

Certified documentation according to DIN EN ISO 9001:2008To ensure a consistently high quality standard anduser-friendliness, this documentation was created tomeet the regulations of a quality management system which complies with the requirements of the standardDIN EN ISO 9001:2008.

cognitas. Gesellschaft für Technik-Dokumentation mbHwww.cognitas.de

Copyright and Trademarks

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

22-0

01.fm

Copyright © 2011 Fujitsu Technology Solutions GmbH.

All rights reserved.Delivery subject to availability; right of technical modifications reserved.

All hardware and software names used are trademarks of their respective manufacturers.

– The contents of this manual may be revised without prior notice.

– Fujitsu assumes no liability for damages to third party copyrights or other rights arising from the use of any information in this manual.

– No part of this manual may be reproduced in any form without the prior written permission of Fujitsu.

Microsoft, Windows, Windows Server, and Hyper V are trademarks or registered trademarks of Microsoft Corporation in the USA and other countries.

Intel and Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the USA and other countries.

mailto:[email protected]

http://www.cognitas.de

D3144, D3145, D3146 (RX900 S2) Technical Manual 3

Before reading this manual

For your safety

This manual contains important information for safely and correctly using this product.

Carefully read the manual before using this product. Pay particular attention to the accompanying manual "Safety Notes and Regulations" and ensure these safety notes are understood before using the product. Keep this manual and the manual "Safety Notes and Regulations" in a safe place for easy reference while using this product.

Radio interference

This product is a "Class A" ITE (Information Technology Equipment). In a domestic environment this product may cause radio interference, in which case the user may be required to take appropriate measures. VCCI-A

Aluminum electrolytic capacitors

The aluminum electrolytic capacitors used in the product's printed circuit board assemblies and in the mouse and keyboard are limited-life components. Use of these components beyond their operating life may result in electrolyte leakage or depletion, potentially causing emission of foul odor or smoke.

As a guideline, in a normal office environment (25°C) operating life is not expected to be reached within the maintenance support period (5 years). However, operating life may be reached more quickly if, for example, the product is used in a hot environment. The customer shall bear the cost of replacing replaceable components which have exceeded their operating life. Note that these are only guidelines, and do not constitute a guarantee of trouble-free operation during the maintenance support period.

High safety use

This product has been designed and manufactured to be used in commercial and/or industrial areas as a server.

When used as visual display workplace, it must not be placed in the direct field of view to avoid incommoding reflections (applies only to TX server systems).

The device has not been designed or manufactured for uses which demand an extremely high level of safety and carry a direct and serious risk of life or body if such safety cannot be assured.

4 Technical Manual D3144, D3145, D3146 (RX900 S2)

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

22-0

01.fm

These uses include control of nuclear reactions in nuclear power plants, automatic airplane flight control, air traffic control, traffic control in mass transport systems, medical devices for life support, and missile guidance control in weapons systems (hereafter, "high safety use"). Customers should not use this product for high safety use unless measures are in place for ensuring the level of safety demanded of such use. Please consult the sales staff of Fujitsu if intending to use this product for high safety use.

Measures against momentary voltage drop

This product may be affected by a momentary voltage drop in the power supply caused by lightning. To prevent a momentary voltage drop, use of an AC uninterruptible power supply is recommended.

(This notice follows the guidelines of Voltage Dip Immunity of Personal Computer issued by JEITA, the Japan Electronics and Information Technology Industries Association.)

Technology controlled by the Foreign Exchange and Foreign Trade Control Law of Japan

Documents produced by Fujitsu may contain technology controlled by the Foreign Exchange and Foreign Trade Control Law of Japan. Documents which contain such technology should not be exported from Japan or transferred to non-residents of Japan without first obtaining authorization in accordance with the above law.

Harmonic Current Standards

This product conforms to harmonic current standard JIS C 61000-3-2.

Only for the Japanese market:About SATA hard disk drives

The SATA version of this server supports hard disk drives with SATA / BC-SATA storage interfaces. Please note that the usage and operation conditions differ depending on the type of hard disk drive used.

Please refer to the following internet address for further information on the usage and operation conditions of each available type of hard disk drive:

http://primeserver.fujitsu.com/primergy/harddisk/


Only for the Japanese market:

I Although described in this manual, some sections do not apply to the Japanese market. These options and routines include:

– USB Flash Module (UFM)

– CSS (Customer Self Service)

– Replacing the lithium battery


© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

22-0

01.fm


Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Important information . . . . . . . . . . . . . . . . . . . . . 11

2.1 CE conformity . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2 Environmental protection . . . . . . . . . . . . . . . . . . . 12

3 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.2 Main memory . . . . . . . . . . . . . . . . . . . . . . . . . . 213.2.1 CPU Memory Riser (CPUMEMR) configuration . . . . . . . . . 213.2.1.1 DIMM slot population order . . . . . . . . . . . . . . . . . 233.2.1.2 DIMM slot population rules for differently ranked DIMMS . . 243.2.1.3 QPI topology . . . . . . . . . . . . . . . . . . . . . . . . . 253.2.2 Memory configuration modes . . . . . . . . . . . . . . . . . . 283.2.2.1 Memory mirroring . . . . . . . . . . . . . . . . . . . . . . 283.2.2.2 Hemisphere mode . . . . . . . . . . . . . . . . . . . . . . 313.2.2.3 Memory interleaving . . . . . . . . . . . . . . . . . . . . . 32

3.3 Memory board configuration table . . . . . . . . . . . . . . 32

3.4 PCIe slots . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343.4.1 PCI interrupt assignment . . . . . . . . . . . . . . . . . . . . 353.4.2 PCIe slot to CPUMEMR assignment . . . . . . . . . . . . . . 353.4.3 Expansion card configuration table . . . . . . . . . . . . . . . 38

3.5 I/O space . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.5.1 I/O space requirements . . . . . . . . . . . . . . . . . . . . . 413.5.2 I/O space limitation . . . . . . . . . . . . . . . . . . . . . . . 42

3.6 Screen resolutions . . . . . . . . . . . . . . . . . . . . . . . 46

3.7 Temperature / System monitoring . . . . . . . . . . . . . . . 46

3.8 Connectors and indicators . . . . . . . . . . . . . . . . . . 473.8.1 Onboard connectors and indicators . . . . . . . . . . . . . . . 483.8.1.1 Baseboard D3144 . . . . . . . . . . . . . . . . . . . . . . 483.8.1.2 CPU Memory Riser (CPUMEMR) D3145 . . . . . . . . . . 51


Contents

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

23-0

01.fm

3.8.1.3 PCI Riser (PCIR) D3146 . . . . . . . . . . . . . . . . . . . 523.8.1.4 I/O board . . . . . . . . . . . . . . . . . . . . . . . . . . . 533.8.2 Onboard DIP switch . . . . . . . . . . . . . . . . . . . . . . . . 543.8.3 External connectors and indicators . . . . . . . . . . . . . . . . 553.8.3.1 Baseboard D3144 . . . . . . . . . . . . . . . . . . . . . . . 553.8.3.2 I/O board . . . . . . . . . . . . . . . . . . . . . . . . . . . 59


1 IntroductionThis technical manual describes the key characteristics and the board layout of the boards for the PRIMERGY RX900 S2 server. The basic elements of this server are the Baseboard D3144, 4 to 8 CPU Memory Risers (CPUMEMR) D3145, and the PCI Riser (PCIR) D3146.

For additional driver information (if available), refer to the Readme files located on the server hard disk and on the supplied DVDs, see Installation DVD of ServerView Suite - ServerView Software Products.

You will find detailed information about the BIOS setup in the "D3144 BIOS Setup Utility for RX900 S2" manual.

I PRIMERGY manuals are available in PDF format on the ServerView Suite DVD 2. The ServerView Suite DVD 2 is part of the ServerView Suite supplied with every server.

PRIMERGY Abbreviations and Glossary can also be found on the ServerView Suite DVD 2.

Notational conventions

The following notational conventions are used in this manual:

Text in italics indicates commands or menu items."Quotation marks" indicate names of chapters and terms that are being

emphasized.Ê describes activities that must be performed in the order

shown.V CAUTION! pay particular attention to texts marked with this symbol.

Failure to observe this warning may endanger your life, destroy the system or lead to the loss of data.

I indicates additional information, notes and tips.


Introduction

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

24-0

01.fm


2 Important informationV CAUTION!

With the system board installed you must open the system to access the system board. How to access the system board of your system is described in the appropriate Upgrade and Maintenance Manual of your server.

When handling the system board, refer to the specific notes on safety in the Upgrade and Maintenance Manual of your server.

2.1 CE conformity

The system complies with the requirements of the EC directives 2004/108/EC regarding "Electromagnetic Compatibility" and 2006/95/EC "Low Voltage Directive". This is indicated by the CE marking (CE = Communauté Européenne).


Important information

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

25-0

01.fm

2.2 Environmental protection

Environmentally-friendly product design and development

This product has been designed in accordance with the Fujitsu standard for "environmentally friendly product design and development". This means that key factors such as durability, selection and labeling of materials, emissions, packaging, ease of dismantling and recycling have been taken into account.

This saves resources and thus reduces the harm done to the environment. Further information can be found at:

– http://ts.fujitsu.com/products/standard_servers/index.html (for the EMEA market)– http://primeserver.fujitsu.com/primergy/concept/ (for the Japanese market)

Energy-saving information

Devices that do not need to be constantly switched on should be switched off until they are needed as well as during long breaks and after completion of work.

Packaging information

This packaging information doesn’t apply to the Japanese market.

Do not throw away the packaging. You may need it later for transporting the system. If possible, the equipment should only be transported in its original packaging.

Information on handling consumables

Please dispose of printer consumables and batteries in accordance with the applicable national regulations.

In accordance with EU directives, batteries must not be disposed of with unsorted domestic waste. They can be returned free of charge to the manufacturer, dealer or an authorized agent for recycling or disposal.

http://ts.fujitsu.com/products/standard_servers/index.html

http://primeserver.fujitsu.com/primergy/concept/



All batteries containing pollutants are marked with a symbol (a crossed-out garbage can). They are also marked with the chemical symbol for the heavy metal that causes them to be categorized as containing pollutants:

Cd CadmiumHg MercuryPb Lead

Labels on plastic casing parts

Please avoid sticking your own labels on plastic parts wherever possible, since this makes it difficult to recycle them.

Returns, recycling and disposal

Please handle returns, recycling and disposal in accordance with local regulations.

Details regarding the return and recycling of devices and consumables within Europe can also be found in the "Returning used devices" manual, via your local Fujitsu branch or from our recycling center in Paderborn:

Fujitsu Technology SolutionsRecycling CenterD-33106 Paderborn

Tel. +49 5251 525 1410

Fax +49 5251 525 32 1410

The device must not be disposed of with domestic waste. This device is labeled in compliance with European directive 2002/96/EC on waste electrical and electronic equipment (WEEE).

This directive sets the framework for returning and recycling used equipment and is valid across the EU. When returning your used device, please use the return and collection systems available to you. Further information can be found at http://ts.fujitsu.com/recycling.

http://ts.fujitsu.com/recycling

http://ts.fujitsu.com/recycling



© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

25-0

01.fm


3 Features

3.1 Overview

This section provides an overview of the key characteristics for the baseboard D3144, the CPU Memory Risers (CPUMEMRs) D3145, and the PCI Riser (PCIR) D3146.

Processors

● Up to 8 Intel Xeon® E7-8xxx series (Westmere-EX) processors (up to 130 W); one processor per CPUMEMR

● 4 Intel QuickPath Interconnects; up to 6,4 GT/s in each direction

● Hyper-Threading technology with up to 20 threads/CPU (2 threads/core, 10 cores/CPU)

● Up to 2.0 MB L2 cache and 30 MB L3 cache (Westmere-EX)

● Supports x2APIC

Main memory

● Up to 128 slots for DDR3 memory modules (1333 MHz) with 2 GB, 4 GB, 8 GB, 16 GB and 32 GB capacity; 16 slots per CPUMEMR

● 4 SMI links per CPU and up to 4 DDR3 DIMM memory modules per SMI link; memory modules connected to CPU via memory buffer named Millbrook2; Intel® Scalable Memory Interconnect (SMI) link connects CPU and Millbrook2

● Maximum configuration: 4 TB memory with availability of 32 GB memory modules

● Basic configuration: 32 GB memory; 4 CPUMEMRs with 4 memory modules per CPUMEMR

● Maximum 32 Gbit/s band width (DDR3)

● Supports +1.5 V and +1.35 V memory modules

● Hardware memory scrubbing

● Single Device Data Correction (SDDC) function (Chipkill™)


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

● Memory interleaving function

● Memory mirroring function

Chipset devices on the baseboard

● Intel Xeon® E7-8800 platform:

– 4x Boxboro-EX® I/O Hubs– Intel Southbridge ICH10R– ServerEngines® Pilot II®– 3x GbE dual channel LAN controller (Kawela)– 1x 10 GbE dual channel LAN controller (Niantec)

Internal connectors

● 1x SATA connector

● 1x USB 2.0 connector

● 1x Trusted Platform Module (TPM)

● 1x USB Flash Module (UFM)

External connectors

● Front side:

– 1x video connector– 3x USB 2.0 connectors

● Rear side:

– 2x 10 GB SFP+ System LAN connectors– 5x 1 GB 1000Base-T System LAN connectors– 1x 1 GB 1000Base-T Shared LAN connector– 1x 100 MB 100Base-TX Management LAN connector– I/O board:

– 1x serial connector– 4x USB 2.0 connectors– 1x video connector


Features

PCIe slots

● Baseboard

– 4x PCI Express 2.0 slots (x8 connector) with PCI Hot-Plug (PHP) capability

– 4x PCI Express 2.0 slots (x8 connector) without PHP capability– 1x PCI Express 2.0 slots (x4 connector) without PHP capability– 1x PCI Express 2.0 slots (x8 connector) without PHP capability for a

Modular RAID controller

● PCI Riser

– 6x PCI Express 2.0 slots (x8 connector) without PHP capability– 1x PCI Express 2.0 slots (x4) without PHP capability

Server management controller iRMC S2

The following features are integrated into iRMC S2:

● Server class Super I/O (SIO)

● Remote KVMS

● Baseboard Management Controller (BMC) including the following functionality:

– 250MHz 32-bit ARM9 processor– IPMI 2.0– 6x I2C SMBus– Power Consumption Control/Limiting (PCC/PCL)– Two 10/100 Ethernet controllers with RMII support– LPC ROM interface– SPI Flash interface– USB Device– One Serial Port– Three UART interfaces– DDR2 16-bit 667 MHz memory– Chassis intrusion logic– PWM Control and Fan Tach Monitor– Voltage Level Monitor and Temperature Monitor– Real-Time Clock (RTC)– Two Watchdog Timers– System and Clock Control– Three Multi-Purpose Timers– Interrupt Controller


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

– JTAG interface

● Graphics controller including the following features:

– Integrated Graphics Core– 2D hardware graphics acceleration– DDR2 memory interface supports up to 128 MB of memory– Supports all display resolutions up to 1600x1200 16bpp @ 75Hz– High-speed integrated 24-bit RAMDAC– Single-lane PCI Express host interface

LAN GbE controller Intel 82576

This controller supports the following features:

● Two connector standard IEEE 802.3 Ethernet interface for 10BASE-T, 100BASE-TX, and 1000BASE-T (802.3, 802.3u, and 802.3ab) applications.

● Serializer-Deserializer (SerDes) to support 1000BASE-SX/LX (optical fiber) and gigabit backplane applications.

● SGMII for external PHY or SFP module.

● Management of MAC and PHY Ethernet layer functions.

● Management of PCI Express packet traffic across its transaction, link, and physical/logical layers.

● External SPI (4-wire) serial interface to Flash EEPROM devices up to 8 MB memory.

● NI-SI / SMBus interface to BMC

● Wake on LAN

● SR-IOV for virtulization on Intel NIC

10 GbE LAN controller Intel 82599

This controller supports the following features:

● Remote PXE boot

● iSCSI / iSCSI BOOT(SW) support

● VT-c support as standard

● Adapter fault tolerance, Load Balancing and Switch fault tolerance in combination with existing GB PCIe card is supported, based on the functionality (SW Kit) of onboard GB LAN controllers.


Features

● Single root I/O virtualization (SR-IOV) for virtualization on Intel NIC.

BIOS features

● ROM based setup utility

● BIOS settings save and restore

● Local BIOS update from USB device

● SMBIOS V2.4

● Remote PXE boot support

● Remote iSCSI boot support

Environmental protection

● Battery in holder for recycling

Power management

● Sleep states S0, S1, S4, S5

● ACPI 3.0

CSS (Customer Self Service)

I This feature is not available for the Japanese market.

This system board supports the CSS functionality. You will find a description of CSS functionality in the Upgrade and Maintenance Manual of your server.

USB Flash Module

The baseboard can be equipped with a USB Flash Module (UFM). The module can be used as optional memory for software (e.g. VMware) or as a software dongle.

TPM

The baseboard can be equipped with a Trusted Platform Module (TPM). This module enables programs from third party manufacturers to store key information (e.g. drive encryption using Windows Bitlocker Drive Encryption).


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

The TPM is activated via the BIOS system, see the "D3144 BIOS Setup Utility for RX900 S2" manual.

V CAUTION!

– When using the TPM, note the program descriptions provided by the third party manufacturers.

– You must also create a backup of the TPM content. To do this, follow the third party manufacturer's instructions. Without this backup, if the TPM or the baseboard is faulty you will not be able to access your data.

– If a failure occurs, please inform your service about the TPM activation before it takes any action, and be prepared to provide them with your backup copies of the TPM content.


Features

3.2 Main memory

The baseboard can be equipped with 4 to 8 CPU Memory Risers (CPUMEMRs). Each CPUMEMR includes one processor and can be equipped with up to 16 DIMM memory modules.The DIMM slots are suitable for 2 GB, 4 GB, 8 GB, 16 GB and 32 GB DDR3 DIMM modules with 1333 MHz. Thus, the system supports a maximum memory configuration of 4 TB with availability of 32 GB DIMM modules.

ECC with memory scrubbing and the single-device data correction (SDDC) function are standard. There is no support for non-ECC-DIMMs.

Basically, the memory modules of an individual CPUMEMR are assigned to the processor installed on this CPUMEMR. However, every CPU can access the memory modules installed on all other CPUMEMRs via the corresponding CPUs, see figure 2.

The basic configuration includes four CPUMEMRs. This configuration can be expanded in two steps. Each steps includes two additional CPUMEMRs. In the 4 and 6 CPUMEMR configurations, an air duct is installed in an empty slot. Table 1 shows the population order for CPUMEMRs.

3.2.1 CPU Memory Riser (CPUMEMR) configuration

The memory modules are connected to the CPU by a memory buffer named Millbrook. There are 4 Milbrooks and 8 DDR channels available on the CPUMEMR to accommodate up to 16 DIMM memory modules. Each Millbrook has two DDR channels and two DIMM memory modules are connected to each channel.

The DIMM slot number is defined as 2 characters:

● First digit ("1" or "2")

Sequence number within the same DDR3 channel

Number of CPUMEMRs

Population order

CPUMEMR position Air duct position

4 1 #1 + #2 + #3 + #4 #66 2 #5 + #6 #88 3 #7 + #8 --

Table 1: Population order for CPUMEMRs


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

● Second digit ("A" to "H")

DDR3 channel number on an individual CPUMEMR

Example:

DIMM 1A = DIMM 0, channel 0; DIMM 1B = DIMM 0, channel 1

DIMM 2A = DIMM 1, channel 0; DIMM 2B = DIMM 1, channel 1

Table 2 shows the relation between the DIMM slot number, the Millbrook identifier, the DDR channel number, and the DIMM memory module number within the same DDR channel.

DIMM slot number

Millbrook identifier

DDR channel number

DIMM memory module number within the same DDR channel

1H C 1 02H C 1 11G C 0 02G C 0 12E D 0 11E D 0 02F D 1 11F D 1 01D A 1 02D A 1 11C A 0 02C A 0 12A B 0 11A B 0 02B B 1 11B B 1 0

Table 2: DIMM memory module connection on the CPUMEMR


Features

3.2.1.1 DIMM slot population order

CPUMEMRs are populated in four steps. In each step, four DIMM memory modules are added. At least four DIMM memory modules must be installed on CPUMEMR #1 and #2. CPUMEMRs #3 to #8 can be left empty.

Table 3 shows the DIMM slot population order for an individual CPUMEMR and figure 1 maps the population order to the location of the DIMM slots on the CPUMEMR.

Figure 1: DIMM slot population order for an individual CPUMEMR

Population order DIMM position1 1B + 1D + 1F + 1H2 1A + 1C+ 1E + 1G3 2B + 2D + 2F + 2H4 2A + 2C + 2E + 2G

Table 3: DIMM memory module population order for an individual CPUMEMR

B1C

1D

1E

1F

1G

1H

1A

1B

B2C

2D

2E

2F

2G

2H

2A

2B

#A#B

#C#DI

I

I

I

J J

J J

K

L


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

3.2.1.2 DIMM slot population rules for differently ranked DIMMS

Basic configuration rules for populating the two slots associated with the same DDR3 channel are:

● The slot with the lower sequence number is populated first.

● 1rank and 2 rank DIMM memory modules can be inserted in both slots.

● 4 rank DIMM memory modules must be installed in the slot with the lower sequence number or in both slots.

Table 4 shows the supported DIMM configurations with mixed DIMM rank.

DIMM# x/2 DIMM# x/14 rank 4 rank2 rank 4 rank1 rank 4 ranknot installed 4 rank2 rank 2 rank1 rank 2 ranknot installed 2 rank2 rank 1 rank1 rank 1 ranknot installed 1 ranknot installed not installed

Table 4: Supported DIMM configurations with mixed DIMM RANK


Features

3.2.1.3 QPI topology

The server uses "Intel® QuickPath Interconnect (QPI)" for the connection between the CPUs and CPU I/O Hubs (IOH). Each CPU has 4 QPI ports and the IOH has 2 QPI ports. Figure 2 shows the QPI topology where the numbers (0, 1; 0, 1, 2, 3) indicate the QPI port number of the CPU/IOH.

Figure 2: QPI topology

Initially, the RX900 S2 can be configured with 4, 6, or 8 CPUMEMRs and each CPUMEMR includes one CPU. If one of the CPUs is removed, the system uses all of the remaining CPUs that form a consistent QPI topology. If removing one CPU would result in an unsupported configuration, the system uses the next supported configuration and CPUs not included in this configuration are set to "disabled". As soon as the removed CPU has been reinstalled, the system enables all those disabled CPUs that are needed to form the next supported configuration.

CPU#1

CPU#4

CPU#7

CPU#5

CPU#2

CPU#3

CPU#6

IOH#1

IOH#2

IOH#3

IOH#4

0

1

0

10

1

0

1

1

23

0

1

2

30 1

23

0

1

23

0

1

23

0

1

23

01

23

0

1

23

0

CPU#8


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

Table 5 lists the supported configurations and the transitions between configurations if a CPU is removed. In this table, "x" indicates that a CPU position is populated.

Figure 3 shows the transitions between QPI scenarios depending on the position of the CPU that has been removed. Configurations with one to three CPUs are not to be understood as minimum configurations. They may occur due to the removal of CPUMEMRs.

Conf. Total number of CPUs

CPU number Transition table of removed CPU and next conf.

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8C1-1 1 xC1-2 1 x C1-2 C1-1C2 2 x x C1-2 C1-1C3 3 x x x C1-2 C1-1 C2C4 4 x x x x C1-2 C1-1 C2 C3C6 6 x x x x x x C1-2 C1-1 C2 C4C8 8 x x x x x x x x C1-2 C1-1 C2 C4 C6

Table 5: Supported CPU configurations and transitions between the configurations


Features

Figure 3: QPI topology transitions

CPU#1

CPU#4

CPU#7

CPU#5

CPU#2

CPU#3

CPU#6

IOH#1

IOH#2

IOH#3

IOH#4

0

1

0

10

1

0

1

1

23

0

1

2

30 1

23

0

1

23

0

1

23

0

1

23

01

23

0

1

23

0

CPU#8

CPU#1

CPU#4

CPU#7

CPU#5

CPU#2

CPU#3

CPU#6

IOH#1

IOH#2

IOH#3

IOH#4

0

1

0

10

1

0

1

1

23

0

1

2

30 1

23

0

1

23

0

1

23

0

1

23

01

23

0

1

23

0

CPU#8

CPU#1

CPU#4

CPU#2

CPU#3

IOH#1

IOH#2

0

1 0

1

1

23

0 2

30

1

30

1

23

0

CPU#1

CPU#2

CPU#3

IOH#1

0

1

1

23

0

1

30

1

23

0

2

2

C8

C6

C4

C3

CPU#1

CPU#2

IOH#1

0

1

1

23

0

1

23

0

C2

CPU#2

IOH#1

0

1

1

23

0

C1-2

CPU#1

IOH#1

0

1

1

23

0

C1-1

CPU#4 empty

CPU#3 empty

CPU#3empty

CPU#3empty

CPU#3empty

CPU#1empty

CPU#2empty

CPU#5 empty

CPU#7 or #8empty

CPU#4 empty

CPU#5 or #6empty

CPU#4empty


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

3.2.2 Memory configuration modes

The RX900 S2 supports the following memory configuration modes:

● "Memory mirroring" on page 28

● "Hemisphere mode" on page 31

● "Memory interleaving" on page 32

3.2.2.1 Memory mirroring

The mirroring mode is set by the system and depends on the CPUMEMR configuration:

● Intra-CPU memory mirroring for a 4 CPUMEMR configuration

● Inter-CPU memory mirroring for 6 and 8 CPUMEMR configurations

In mirroring mode, the size of the memory modules must be the same within the following groups of DIMM slots, see figure 4:

● DIMM# (1B, 1D, 1F, 1H)

● DIMM# (1A, 1C, 1E, 1G)

● DIMM# (2B, 2D, 2F, 2H)

● DIMM# (2A, 2C, 2E, 2G)

Figure 4: Grouping of DIMM slots for memory mirroring

B1C

1D

1E

1F

1G

1H

1A

1B

B2C

2D

2E

2F

2G

2H

2A

2B


Features

Intra-CPU memory mirroring for a 4 CPUMEMR configuration

In a 4 CPUMEMR configuration, memory mirroring is configured within an individual CPUMEMR. Figure 5 shows intra-CPU memory mirroring where the memory modules 1B to 1D (light grey color) are mapped to the memory modules 1F to 1H (green color).

Figure 5: Memory mirroring in a 4 CPUMEMR configuration

Inter-CPU memory mirroring for 6 and 8 CPUMEMR configurations

In a 6 or 8 CPUMEMR configuration, memory mirroring is configured between pairs of CPUMEMRs. Figure 6 shows the CPUMEMR mirroring pairs (1,2), (3,4), (5,6), (7,8) where each pair consists of a green and a light grey CPUMEMR.

B1C

1D

1E

1F

1G

1H

1A

1B

B2C

2D

2E

2F

2G

2H

2A

2B


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

Figure 6: Memory mirroring in a 6 or 8 CPUMEMR configuration

On both CPUMEMRs of the mirroring pair, the size of the DIMMs must be the same within the following groups of memory modules, see figure 4:

● DIMM# (1B, 1D, 1F, 1H)

● DIMM# (1A, 1C, 1E, 1G)

● DIMM# (2B, 2D, 2F, 2H)

● DIMM# (2A, 2C, 2E, 2G)

Scenarios for memory mirroring interruption

If failed or disabled DIMM memory modules are detected, the impact on the mirroring configuration is:

1. A group of DIMM memory modules is deactivated:

– 4 CPU configuration: 2 DIMM memory modules

– 6 or 8 CPU configuration: 4 DIMM memory modules

2. If the active DIMM memory modules do not form a valid mirroring configuration due to lacking mirroring pairs, the BIOS boots without mirroring configuration.

3. If the active DIMM memory modules form a valid mirroring configuration, the BIOS keeps mirroring configuration with the remaining DIMM memory modules.


Features

Figure 7 shows example scenarios for a 4 CPUMEMR configuration with intra-CPUMEMR mirroring in the event of DIMM failure and/or deactivation.

Figure 7: Intra-CPUMEMR mirroring scenarios after DIMM module failure/deactivation

3.2.2.2 Hemisphere mode

The Nehalem-EX CPU has two memory-controller modes:

● 1 - 4 CPU configuration: Non-Hemisphere mode

● 5 - 8 CPU configuration: Hemisphere mode

In hemisphere mode, the address space of a CPU is subdivided into two sections called upper and lower hemisphere. Data is distributed over these two sections for better performance.

As hemisphere mode is used for configurations with more than 4 CPUs, an even number of CPUs is necessary to configure memory-mirroring.

OK

OK

Disabled

OK

OK

OK

OK

OK

OK

Failed

Failed OK

OK

OK

OK

OK

OK

Failed

OK

OK

OK

OK

OK

OK

Disabled

Disabled

OK

OK

OK

OK

Disabled

Disabled Disabled

Failed

DIMMfailure

DIMMdisabled

Mirroring pair Disabled

Failed

Memory mirroringenabled

Memory mirroring disabled

Memory mirroring disabled


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

3.2.2.3 Memory interleaving

In Interleaving mode, the physical memory address is decoded at a specific bit and is sliced into several memory controllers. Interleaving mode is supported by default.

3.3 Memory board configuration table

Table 6 shows the DIMM slot population order and the mapping of DIMM slots to CPUs and IOHs for 4, 6, and 8 CPUMEMRs.

CPU No.

DIMM slot pop. order

4 CPUMEMRs 6 CPUMEMRs 8 CPUMEMRs IOH No.1B

1D1F1H

1A1C1E1G

2B2D2F2H

2A2C2E2G

1B1D1F1H

1A1C1E1G

2B2D2F2H

2A2C2E2G

1B1D1F1H

1A1C1E1G

2B2D2F2H

2A2C2E2G

CPU#1 1 x -- -- -- x -- -- -- x -- -- -- #12 x x -- -- x x -- -- x x -- --3 x x x -- x x x -- x x x --4 x x x x x x x x x x x x

CPU#2 1 x -- -- -- x -- -- -- x -- -- -- #12 x x -- -- x x -- -- x x -- --3 x x x -- x x x -- x x x --4 x x x x x x x x x x x x

CPU#3 0 -- -- -- -- -- -- -- -- -- -- -- -- #21 x -- -- -- x -- -- -- x -- -- --2 x x -- -- x x -- -- x x -- --3 x x x -- x x x -- x x x --4 x x x x x x x x x x x x

Table 6: DIMM slot population order and mapping of DIMM slots to I/O Hubs and CPUs


Features

CPU#4 0 -- -- -- -- -- -- -- -- -- -- -- -- #21 x -- -- -- x -- -- -- x -- -- --2 x x -- -- x x -- -- x x -- --3 x x x -- x x x -- x x x --4 x x x x x x x x x x x x

CPU#5 0 -- -- -- -- -- -- -- -- -- -- -- -- #31 -- -- -- -- x -- -- -- x -- -- --2 -- -- -- -- x x -- -- x x -- --3 -- -- -- -- x x x -- x x x --4 -- -- -- -- x x x x x x x x

CPU#6 0 -- -- -- -- -- -- -- -- -- -- -- -- #31 -- -- -- -- x -- -- -- x -- -- --2 -- -- -- -- x x -- -- x x -- --3 -- -- -- -- x x x -- x x x --4 -- -- -- -- x x x x x x x x

CPU#7 0 -- -- -- -- -- -- -- -- -- -- -- -- #41 -- -- -- -- -- -- -- -- x -- -- --2 -- -- -- -- -- -- -- -- x x -- --3 -- -- -- -- -- -- -- -- x x x --4 -- -- -- -- -- -- -- -- x x x x

CPU#8 0 -- -- -- -- -- -- -- -- -- -- -- -- #41 -- -- -- -- -- -- -- -- x -- -- --2 -- -- -- -- -- -- -- -- x x -- --3 -- -- -- -- -- -- -- -- x x x --4 -- -- -- -- -- -- -- -- x x x x

CPU No.

DIMM slot pop. order

4 CPUMEMRs 6 CPUMEMRs 8 CPUMEMRs IOH No.1B

1D1F1H

1A1C1E1G

2B2D2F2H

2A2C2E2G

1B1D1F1H

1A1C1E1G

2B2D2F2H

2A2C2E2G

1B1D1F1H

1A1C1E1G

2B2D2F2H

2A2C2E2G

Table 6: DIMM slot population order and mapping of DIMM slots to I/O Hubs and CPUs


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

3.4 PCIe slots

Figure 8 shows the PCIe slots (C) and (G) on the PCI Riser (A) and the baseboard (E). The PCI Riser (PCIR) is connected to the baseboard via the PCIR connectors (D; F). Faulty PCI cards can be detected in all PCIe slots via the CSS indicators (B) and (H). In addition, PCI hot-plug slots are equipped with the indicator (I).

Figure 8: PCIe slots on the PCI Riser (A) and the baseboard (E; only PCI part is shown)

123467 5

A B

C

D

G

H

I

HHHHHHH HH

III

E F


Features

3.4.1 PCI interrupt assignment

The BIOS assigns the PCI interrupts automatically. Further settings are not possible.

3.4.2 PCIe slot to CPUMEMR assignment

Figure 9 shows the assignment of PCIe slots to the 4 I/O Hubs and the 4 - 8 CPUMEMRs.

I Before adding an expansion card, check carefully whether this PCIe slot is supported with the CPUMEMR configuration of the individual server.

Pos. Slot no. Description Hot-plugA PCI Riser (PCIR)C #5 PCI Express 2.0 (x4), 1/2 length, x8 connector noC #1, #2, #3,

#4, #6, #7PCI Express 2.0 (x8), 1/2 length, x8 connector no

E BaseboardG #8 PCI Express 2.0 (x8), 1/2 length, x8 connector

for modular RAID controller onlyno

G #9, #10, #11, #12

PCI Express 2.0 (x8), 1/2 length, x8 connector yes

G #13, #14, #15, #16

PCI Express 2.0 (x8), 1/2 length, x8 connector no

G #17 PCI Express 2.0 (x4), 1/2 length, x8 connector no


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

Figure 9: Assignment of PCIe slots to CPUMEMRs

CPU #8

CPUMEMR#8

CPU #7

CPUMEMR#7

IOH#4

CPUMEMR#1

CPU #1

CPU #2

CPUMEMR#2

IOH#1

CPU #4

CPUMEMR#4

CPU #3

CPUMEMR#3

IOH#2

CPU #6

CPUMEMR#6

CPU #5

CPUMEMR#5

IOH#3

PCIe#17

PCIe

#7

PCIe

#8

PCIe

#9

PCIe

#10

PCIe

#11

PCIe

#12

PCIe

#13

PCIe

#14

PCIe

#15

PCIe

#16

PCIe

#2

PCIe

#1

PCIe

#5

PCIe

#6

PCIe

#4

PCIe

#3

ModularRAID Hot-plug PCIPCIR

x8x8x8 x8 x8 x8 x8x4 x8 x8 x8 x8 x8 x8 x8 x8 x4


Features

Figure 9 lists the mapping of PCIe slots to I/O Hubs and CPUs.

CPU population order CPU - IOH mapping

Connected devices

Remark

4 CPUs 6 CPUs 8 CPUs CPU No.

IOH No.

CPU#1 x x x #1 or #2

#1 ICH10 PCIe #5 to BMC

CPU#2 x x x BMC (VGA)

GbE#1, GbE#2, GbE#3, 10GbE#1

PCIe slot #8 Modular RAID

PCIe slot #1 PCI Riser


CPU#3 x x x #3 or #4

#2 PCIe slot #5 PCI Riser; Gen2x4

CPU#4 x x x PCIe slot #3 PCI Riser




CPU#5 x x #5 or #6

#3 PCIe slot #9 hot-plug

CPU#6 x x PCIe slot #10 hot-plug

PCIe slot #11 hot-plug

PCIe slot #12 hot-plug

CPU#7 x #7 or #8

#4 PCIe slot #17 Gen2x4

CPU#8 x PCIe slot #13

PCIe slot #14

PCIe slot #15

PCIe slot #16

Table 7: Mapping of PCIe slots to I/O Hubs and CPUs


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

3.4.3 Expansion card configuration table

Expansion cards require I/O space for Legacy Booting and for OS running. Due to the limitation of I/O space, see "I/O space limitation" on page 42, the basic rules for installing expansion cards are:

1. Begin with the installation of the SAS and SAS RAID controllers in order of I/O space assignment order.

2. Continue with the installation of the expansion cards for LAN and FC into the PCIe slots.

Chipset IOH#1 IOH#2

Bus scan order 3 2 1 5 6 4 7 8

I/O space assignment order within the I/O Hub (IOH)

1 2 3 4 3 2 1

Hot-plug n n n n n n n n

Gen2 width x8 x8 x8 x8 x8 x4 x8 x8

Slot # 1 2 8 3 4 5 6 7

Vendor Type PCIe Max #

LSI SAS 3Gb Gen1 x4 3 x x x x x x

SAS 6Gb Geb 2 x6 3 x x x x x x

Cougar2 Gen 2 x8 1 x

SAS RAID (WASAT)

Gen 2 x8 2 x (*1)

x (*1)

x (*1)

x (*1)

x (*1)

x (*1)

Emulex 8G-FCx1 Gen 2 x8 8 x x x x x x

8G-FCx2 Gen 2 x8 8 x x x x x x

FCoE (*2) Gen 2 x8 8 x x x x x x

Qlogic 8G-FCx1 Gen 2 x8 6 x x x x x x

8G-FCx2 Gen 2 x8 6 x x x x x x

Intel GbEx1 Gen 1 x4 8 x x x x x x x

FTS GbEx2 (D2735)

Gen1 x4 8 x x x x x x x

GbEx4 (D2745)

Gen2 x4 8 x x x x x x

10GbEx2 (D2755)

Gen2 x8 8 x x x x x x

Table 8: Expansion card installation recommendation I/O Hubs IOH#1 and IOH#2


Features

*1) The SAS RAID controller shall not be installed in PCIe slot #1 to #7 on the PCI Riser when a BBU is connected because of limited length of the BBU cable.*2) No mix with FC controller possible.*3) Capacity: 160 GB, 320 GB, 640 GB*4) Capacity: 320 GB, 640 GB, 1280 GB

Mellanox IB-QDRx1 Gen2 x8 2 x x x x x x

IB-QDRx2 Gen2 x8 2 x x x x x x

Fusion-IO

ioDRIVE (*3)


ioDRIVE (*4)


Chipset IOH#3 IOH#4

Bus scan order 12 11 10 9 17 16 15 14 13


1 2 1 2

Hot-plug y y y y n n n n n

Gen2 width x8 x8 x8 x8 x8 x8 x8 x8 x4

Slot # 9 10 11 12 13 14 15 16 17


LSI SAS 3Gb Gen1 x4 3 x x x x

SAS 6Gb Geb 2 x6 3 x x x x

Cougar2 Gen 2 x8 1

SAS RAID (WASAT)

Gen 2 x8 2 1 1 x x


Chipset IOH#1 IOH#2

Bus scan order 3 2 1 5 6 4 7 8


1 2 3 4 3 2 1

Hot-plug n n n n n n n n

Gen2 width x8 x8 x8 x8 x8 x4 x8 x8

Slot # 1 2 8 3 4 5 6 7




Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

*1) The SAS RAID controller shall not be installed in PCIe slot #1 to #7 on the PCI Riser when a BBU is connected because of limited length of the BBU cable.*2) No mix with FC controller possible.*3) Capacity: 160 GB, 320 GB, 640 GB*4) Capacity: 320 GB, 640 GB, 1280 GB

Emulex 8G-FCx1 Gen 2 x8 8 2 2 2 2 x x x x

8G-FCx2 Gen 2 x8 8 2 2 2 2 x x x x

FCoE Gen 2 x8 8 2 2 2 2 x x x x

Qlogic 8G-FCx1 Gen 2 x8 8 2 2 2 2 x x x x

8G-FCx2 Gen 2 x8 8 2 2 2 2 x x x x

Intel GbEx1 Gen 1 x4 8 x x x x x x x x x

FTS GbEx2 (D2735)

Gen1 x4 8 x x x x x x x x x

GbEx4 (D2745)

Gen2 x4 8 x x x x x x x x

10GbEx2 (D2755)

Gen2 x8 8 x x x x x x x x

Mellanox IB-QDRx1 Gen2 x8 2 x x x x x x x x

IB-QDRx2 Gen2 x8 2 x x x x x x x x

Fusion-IO

ioDRIVE (*3)


ioDRIVE Duo (*4)


Chipset IOH#3 IOH#4

Bus scan order 12 11 10 9 17 16 15 14 13


1 2 1 2

Hot-plug y y y y n n n n n

Gen2 width x8 x8 x8 x8 x8 x8 x8 x8 x4

Slot # 9 10 11 12 13 14 15 16 17




Features

3.5 I/O space

I/O space is required for the hardware itself, Legacy OPROM, EFI driver, and OS driver, see figure 10.

Figure 10: I/O space requirement

3.5.1 I/O space requirements

Figure 10 lists the I/O space requirements where "x" indicates that I/O space is required. Please note that I/O space assignment is only needed if Legacy LAN Boot(PXE, iSCSI) and an LSI SAS/SAS_RAID card is used.

PCIe card

Firmware

OptionROM(Legacy)

EFIDriver

BIOS

LegacyOS

EFI-awareOS

CSM

Firmware


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

*1) WinPE does not require I/O space but some drivers require I/O space for loading properly.

*2) When both Legacy OPROM and EFI driver are available, the BIOS loads the EFI driver and the Legacy service is provided by the CSM module, that is the Legacy OPROM is not loaded.

3.5.2 I/O space limitation

The I/O space is limited to 64 KB. Each PCI Bridge assigns its I/O space in several blocks of 4 KB. ICH10 requires 1-block for Legacy support.

Auto and Disabled can be selected in the BIOS setup menu to assign I/O space for onboard devices and for PCIe slots including slot #8 for the Modular RAID controller. Auto is set for all onboard devices and PCIe slots by default.

Device Hardware Operating systemI/O space is requested in config. space

Option ROM (legacy)

EFI driver (EFI native)

DOS Win PE (*1)

Linux 5.4

Windows 32&64bit

VM ware

Intel GbE

x x -- -- -- -- -- --

Intel 10GbE

x x -- -- -- -- -- --

LSI SAS

x x x x x -- x x

LSI SAS RAID

x x x x -- x -- x

Emulex FC

x x (*2) -- -- -- -- -- --

Table 10: I/O space requirement


Features

The I/O space resource distribution to each I/O Hub depends on the number of recognized I/O Hubs:

● 4 CPUMEMRs

There is no limitation for assigning I/O space.

● 6 or 8 CPUMEMRs

If many expansion cards are installed which require I/O space, the BIOS may not assign I/O space for all devices. You have to enter the BIOS Setup menu and to disable the assignment of I/O space for devices that do not need I/O space under OS operation.

Table 11 lists the I/O space assignment for individual CPU configurations. In the BIOS Setup menu, you can choose from the following settings for the device per slot:

● Auto: I/O space is assigned when the device requests for it.

● Disabled: I/O space is not assigned even if the device requests for it.

If you specify auto, the I/O space is assigned in descending order of the PCI-bus number within each I/O Hub until no I/O space is left.


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

Device / slot

I/O Hub

I/O space block (4 KB each) Remark

4 CPUs 6 CPUs 8 CPUs Assign order

ICH (Legacy, SATA), VGA

#1 x x x 1 This block is mandatory for Legacy Support.

GbE#1 x x x 8 Do not disable I/O space assignment to this device if you use PXE-boot from the Onboard-LAN controller as the Intel Boot Agent ROM requires I/O space.

GbE#2 x x x 7

Modular RAID controller slot #8

x x x 6 Cougar2 controller requires I/O space for operation.

GbE#3 x x x 5 Do not disabled I/O space assignment to this device if you use PXE-boot from Onboard-LAN as the Intel Boot Agent ROM requires I/O space.

10GbE#1 x x x 4

PCIe slot #2

x x x 3

PCIe slot #1

x x x 2

Table 11: I/O space assignment for 4, 6, and 8 CPU configurations


Features

PCIe slot #3

#2 Max 8 Max 4 Max 4 4

PCIe slot #4

3

PCIe slot #5

5

PCIe slot #6

2

PCIe slot #7

1

PCIe slot #9

#3 IOH #3 not recognized

Max 4 Max 2 1

PCIe slot #10

2

PCIe slot #11

3

PCIe slot #12

4

PCIe slot #13

#4 IOH #4 not recognized

IOH #4 not recognized

Max 2 1

PCIe slot #14

2

PCIe slot #15

3

PCIe slot #16

4

PCIe slot #17

5

Sum Max 16 Max 16 Max 16

Device / slot

I/O Hub

I/O space block (4 KB each) Remark

4 CPUs 6 CPUs 8 CPUs Assign order

Table 11: I/O space assignment for 4, 6, and 8 CPU configurations


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

3.6 Screen resolutions

The screen resolution that applies to the internal graphics controller depends on the operating system in use, see table 12.

3.7 Temperature / System monitoring

Temperature and system monitoring aims to reliably protect the hardware from overheating and to provide information on the system state. It also prevents unnecessary noise by reducing the fan speed.

Temperature and system monitoring is controlled by an onboard controller that supports the following functions:

Temperature monitoring

Measuring the processor temperature and ambient temperature by way of a temperature sensor.

Fan monitoring

The fans are monitored. Fans that are blocked, sluggish or no longer available are detected.

Screen resolution (pixel)

Refresh rate (Hz) No. of bits per pixel

32x400 85 4 bpp, 8 bpp640x480 85 4 bpp, 8 bpp, 16 bpp, 32 bpp800x600 85 4 bpp, 8 bpp, 16 bpp, 32 bpp1024x768 75 8 bpp, 16 bpp, 32 bpp1152x864 60 8 bpp, 16 bpp1280x1024 60/70/75/85 16 bpp1280x1024 60 24 bpp1600x1200 60 8 bpp, 16 bpp

Table 12: Screen resolution of the internal graphic controller


Features

Fan control

The speed of the fans is regulated according to temperature. Maximum speed will only be achieved when switching the server on and in case of a fan test.

Sensor monitoring

Any fault in, or the removal of, a temperature sensor is detected. Should this happen, all fans monitored by this sensor run at maximum speed to ensure the highest possible protection of the hardware.

Voltage monitoring

When the voltage exceeds or falls below the warning levels, an alert is issued.

System Event Log (SEL)

All monitored events of the system board are signalized via the Global Error LED or CSS LED and recorded in the System Event Log. They can be retrieved in the iRMC S2’s Web interface or via the ServerView Operations Manager.

3.8 Connectors and indicators

This section provides an overview of:

● "Onboard connectors and indicators" on page 48

● "External connectors and indicators" on page 55


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

3.8.1 Onboard connectors and indicators

3.8.1.1 Baseboard D3144

Figure 11 shows the onboard connectors on the baseboard where the individual connectors are named in table 13.

Figure 11: Onboard connectors baseboard D3144

D

A B C

D

E

F

G

HIJ

K

L

M


Features

Figure 12 shows the indicators on the baseboard where the individual indicators are named in table 14.

Pos. Connector Pos. ConnectorA PCI Riser H Chassis ID boardB I/O board I Operator panelC PCIe slots on the baseboard J Power distribution boardD Trusted Platform Module

(TPM)K Front video connector

E USB Flash Module (UFM) L Front USB connectorF FAN M SATA optical disk driveG CPU Memory Riser

Table 13: Onboard connectors baseboard D3144


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

Figure 12: Indicators on the baseboard D3144

All the indicators are only visible after the cover has been removed.

If the server has been powered off (power-plugs must be disconnected), it is possible to indicate the faulty component by pressing the indicate CSS button located on the PCI Riser.

Pos. Indicator Pos. IndicatorA BMC heart beat C PCI hot-plug power/faultB POST code D PCIe CSS

Table 14: Connectors on the baseboard D3144

A

BC

D


Features

3.8.1.2 CPU Memory Riser (CPUMEMR) D3145

Figure 13 shows the onboard connectors and indicators on the CPUMEMR where the individual connectors and indicators are named in table 15.

Figure 13: Onboard connectors and indicators CPUMEMR D3145

Pos. Connector Pos. IndicatorA CPU C DIMM, CPU, CPUMEMR

CSSB CPUMEMR

Table 15: Onboard connectors and indicators CPUMEMR D3145

B1C

1D

1E

1F

1G

1H

1A

1B

B2C

2D

2E

2F

2G

2H

2A

2B

CPU1B1D

1A1C

2B2D

2A2C

1E1G

2F2H

1F1H

2E2G

CPUMEMR

A

B

C


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

3.8.1.3 PCI Riser (PCIR) D3146

Figure 14 shows the onboard connectors and indicators on the PCIR where the individual connectors and indicators are named in table 16.

Figure 14: Onboard connectors and indicators PCIR D3146

Pos. Connector Pos. IndicatorA PCIe slots C PCIe CSSB PCIR D CSS button

Table 16: Onboard connectors and indicators PCIR D3146

123467 5

A

B

C D


Features

3.8.1.4 I/O board

Figure 15 shows the onboard connector on the I/O board.

Figure 15: Onboard connector on the I/0 board

I For external connectors, please refer to "I/O board" on page 59.

Pos. ConnectorA 1x USB connector

A


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

3.8.2 Onboard DIP switch

Figure 16 shows the position of the DIP switch where the DIP switch settings are named in table 17.

I Default setting: Switches 1 to 4 are set to Off.

Figure 16: Position of the DIP switches

I Password skip

Switch 2 is used to define whether the boot password is skipped at system startup, if Password On Boot is set to First Boot or Every Boot in the BIOS Setup security menu.

No. Function Setting Description1-1 Not used1-2 Password SKIP On Password skip enabled

Off Password skip disabled1-3 BIOS Write

ProtectOn BIOS can only be readOff BIOS can both be read and

written to1-4 CMOS clear On BIOS should clear CMOS area

Table 17: DIP switch settings

ON 1-

11-

21-

31-

4


Features

3.8.3 External connectors and indicators

3.8.3.1 Baseboard D3144

Figure 17 shows the external connectors and indicators on the baseboard where the individual connectors and indicators are named in table 18. The LAN indicators are explained in table 19.

Figure 17: External connectors of the baseboard D3144

Pos. Port Nr.

Connector Pos. Indicator

A -- 1x 100 MB Management LAN controller

(indicator description table 19)

C Global error indicator (orange), CSS indicator (yellow), ID indicator (blue)

A #1 1x GB Shared LAN controller


A #2 - #6 5x GB System LAN controller


Table 18: External connectors on the baseboard D3144

A B C


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

The system is equipped with one Dual 10 GB Ethernet Controller type Intel 82599. In addition, three Dual 1 GB Ethernet Controller type Intel 82576 NS (system LAN) are installed supporting transmission rates of 10 Mbit/s, 100 Mbit/s, and (only for system LAN) 1 Gbit/s.

The 1Gb Ethernet LAN controllers support WoL functionality by means of Magic Packet™. It is also possible to boot a system without its own boot hard disk via LAN. PXE is supported for this.

The separate management LAN connector is used as a management interface (iRMC S2) and is prepared for operation with the Remote Management.

Figure 18 shows the LAN indicators and table 19 explains how the LAN indicators indicate the state of the connection and the transfer rate.

Figure 18: LAN indicators

B #7 - #8 2x 10 GB System LAN controller


Pos. Port Nr.

Connector Pos. Indicator

Table 18: External connectors on the baseboard D3144

F C

C

DG E

A B A B A B

A BA B


Features

Pos. Indicator Function DescriptionA Activity LAN link/transfer

(1 GB system LAN)Steady green signal: LAN connection exists.

Remains dark: no LAN connection exists.

Flashes green: LAN transfer takes place.

B 10/100/1000 LAN speed (1 GB system LAN)

Steady yellow signal: LAN transfer rate of 1 Gbit/s.

Steady green signal: LAN transfer rate of 100 Mbit/s.

Remains dark: LAN transfer rate of 10 Mbit/s.

C 10G LAN link/transfer (10 GB system LAN)

Steady green signal: LAN connection exists.



D 10/100/1000 LAN speed (shared LAN)

Steady yellow signal: LAN transfer rate of 1 Gbit/s.



E Activity LAN link/transfer (shared LAN)




Table 19: LAN indicators


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm

F 10/100 LAN speed (management LAN)



G Activity LAN link/transfer (management LAN)




Pos. Indicator Function Description

Table 19: LAN indicators


Features

3.8.3.2 I/O board

Figure 19 shows the external connectors on the I/O board where the individual connectors and indicators are named in table 20.

Figure 19: External connectors on the I/O board

The serial connector COM1 can be used as default interface or to communicate with the iRMC S2.

Pos. ConnectorA 4x USBB Serial COM1C Video

Table 20: External connectors on the I/O board

A

C B


Features

© c

ogni

tas.

Ges

ells

chft

für T

echn

ik-D

okum

enta

tion

mbH

201

1 P

fad:

C:\P

rogr

amm

e\FC

T\tim

_app

\tim

_loc

al\w

ork\

WA

LTE

R\O

BJ_

DO

KU

-120

26-0

01.fm