external interface specification using ipmi...2 intel® intelligent power node manager 1.5 external...

88
Document Number: 322999-001 Intel ® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI December 2009

Upload: others

Post on 27-Jan-2021

4 views

Category:

Documents


0 download

TRANSCRIPT

  • Document Number: 322999-001

    Intel® Intelligent Power Node Manager 1.5External Interface Specification Using IPMI

    December 2009

  • 2 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    Legal Lines and DisclaimersINFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL® PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. Intel products are not intended for use in medical, life saving, life sustaining, critical control or safety systems, or in nuclear facility applications.

    Intel may make changes to specifications and product descriptions at any time, without notice.

    Designers must not rely on the absence or characteristics of any features or instructions marked “reserved” or “undefined.” Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them.

    The Intel® Intelligent Power Node Manager 1.5 may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.

    Intel software products are copyrighted by and shall remain the property of Intel Corporation. Use, duplication, or disclosure is subject to restrictions stated in Intel's Software License Agreement, or in the case of software delivered to the government, in accordance with the software license agreement as defined in FAR 52.227-7013.

    Code names presented in this document are only for use by Intel to identify a product, technology, or service in development, that has not been made commercially available to the public, i.e., announced, launched or shipped. It is not a "commercial" name for products or services and is not intended to function as a trademark.Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.

    I2C is a two-wire communications bus/protocol developed by Philips. SMBus is a subset of the I2C bus/protocol and was developed by Intel. Implementations of the I2C bus/protocol may require licenses from various entities, including Philips Electronics N.V. and North American Philips Corporation.

    Copies of documents which have an order number and are referenced in this document, or other Intel literature may be obtained by calling 1-800-548-4725 or by visiting Intel's website at http://www.intel.com.

    Intel,Intel® Intelligent Power Node Manager, Xeon and the Intel logo are trademarks of Intel Corporation in the United States and other countries.

    *Other names and brands may be claimed as the property of others.

    Copyright © 2009, Intel Corporation. All Rights Reserved.

    http://www.intel.com

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 3

    Contents

    1 Introduction ..............................................................................................................71.1 Scope ................................................................................................................7

    1.1.1 System States and Power Management .......................................................71.2 Reference Documents ..........................................................................................71.3 Overview ...........................................................................................................7

    1.3.1 Use Cases ...............................................................................................81.3.2 Intel® Intelligent Power Node Manager Use Cases........................................91.3.3 Features ............................................................................................... 17

    2 Requirements on Platform Components................................................................... 212.1 Intel® Intelligent Power Node Manager IPMI OEM Commands ................................. 222.2 IPMI Sensors .................................................................................................... 372.3 IPMI Events...................................................................................................... 372.4 Alerts .............................................................................................................. 422.5 Command Passing via BMC................................................................................. 422.6 Intel® Intelligent Power Node Manager Discovery.................................................. 442.7 Error Conditions ................................................................................................ 452.8 Management IPMI Interface................................................................................ 452.9 SEL Device Commands....................................................................................... 452.10 IPMI Device “Global” Commands ......................................................................... 462.11 Sensor Device Command.................................................................................... 482.12 IPMI OEM Device Commands .............................................................................. 492.13 External Intel® Intelligent Power Node Manager Configuration and Control Commands...

    ...................................................................................................................... 522.14 BMC Requirements for Intel® Intelligent Power Node Manager Discovery.................. 552.15 Local Platform Intel® Intelligent Power Node Manager Configuration and Control

    Commands....................................................................................................... 562.16 Intel Management Engine Firmware Update IPMI Commands................................... 612.17 Online Update Flow............................................................................................ 652.18 IPMI Commands Supported in Recovery Mode....................................................... 662.19 IPMI Sensors Implemented by Platform Services FW .............................................. 67

    2.19.1 CPU Temperature Sensors ....................................................................... 702.19.2 Memory Throttling Status Sensors ............................................................ 712.19.3 ICH9 Fan Speed Sensors ......................................................................... 712.19.4 ICH9 On-Die Temperature Sensors ........................................................... 712.19.5 Intel Management Engine Power State Sensor............................................ 722.19.6 Dynamic Power Intel® Intelligent Power Node Manager Event Sensor ........... 722.19.7 Dynamic Power Intel® Intelligent Power Node Manager Health Sensor .......... 722.19.8 Dynamic Power Intel® Intelligent Power Node Manager Operational Capabilities

    sensor .................................................................................................. 722.19.9 Server Platform Services Firmware Health Sensor....................................... 722.19.10IPMI Platform Event Messages Generated by Platform Services FW ............... 732.19.11Generic Event/ Reading Type Codes.......................................................... 742.19.12Event Messages Definition ....................................................................... 75

    2.20 Event Generation Control ................................................................................... 782.20.1 Server Platform Services Debug Event ...................................................... 792.20.2 Debug SEL Entry Definition (External) ....................................................... 792.20.3 Debug SEL Entry Definition (Internal) ....................................................... 80

    2.21 Completion Codes ............................................................................................. 81

    3 BMC IPMI Interface ................................................................................................. 833.1 IPMI Device “Global” Commands ......................................................................... 833.2 Sensor Device Commands .................................................................................. 83

  • 4 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    3.3 Alert Immediate Commands................................................................................843.4 OEM Commands Implemented by BMC .................................................................84

    3.4.1 Power Consumption Readings...................................................................843.4.2 Inlet Air Temperature Readings ................................................................853.4.3 ICC_TDC Reading from PECI ....................................................................863.4.4 OEM ME Power State Change....................................................................863.4.5 OEM Command Definition ........................................................................873.4.6 Summary of Options ...............................................................................883.4.7 IPMI Command Bridging ..........................................................................883.4.8 IPMB Reset Scenarios..............................................................................88

    Figures2-1 Example IPMI Command Bridging from LAN ..........................................................432-2 Intel® ME Online Update Flow.............................................................................662-3 Temperature Sensor Data Format ........................................................................712-4 16-Bit PECI Reading to 8-Bit Mapping...................................................................71

    Tables1-1 Terminology....................................................................................................... 71-2 Intel® Intelligent Power Node Manager 1.5 Usage Models and Use Cases ................... 81-3 Policy Management Feature.................................................................................171-4 Monitoring and Querying Feature .........................................................................171-5 Alerts and Notifications Feature ...........................................................................181-6 External Interface ..............................................................................................181-7 Supported System Architectures ..........................................................................192-1 Intel® Intelligent Power Node Manager Interfaces to Platform Components ...............212-2 Platform Ingredients for Intel® Intelligent Power Node Manager Support ..................212-3 Intel® Intelligent Power Node Manager-specific OEM commands (IPMI) ....................222-4 Intel® Intelligent Power Node Manager IPMI Sensors .............................................372-5 Intel® Intelligent Power Node Manager IPMI Events ...............................................372-6 Intel® Intelligent Power Node Manager OEM SDR – Record Body .............................442-7 SEL Device Commands .......................................................................................452-8 Intel® Intelligent Power Node Manager OEM SDR – Record Body .............................55

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 5

    Revision History

    §

    Revision Number Description Date

    322999-001 • Initial Public release of document. December 2009

  • 6 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 7

    Introduction

    1 Introduction

    1.1 ScopeThis document contains the mapping of external interface specification using Intel® Intelligent Power Node Manager version 1.5 over IPMI.

    1.1.1 System States and Power Management

    1.2 Reference Documents

    1.3 OverviewIntel® Intelligent Power Node Manager version 1.5 is a platform resident technology that enforces power and thermal policies for the platform. These policies are applied by exploiting subsystem knobs (such as processor P and T states) that can be used to

    Table 1-1. Terminology

    Acronym or Term Definition

    S0 A system state where power is applied to all HW devices and system is running normally.

    S1, S2, S3 A system state where the host CPU is not running however power is connected to the memory system.

    S4 A system state where the host CPU and memory is not active.

    S5 A system state where all power to the host system is off however power cord is still connected.

    Sx All S states that are different than S0.

    OS Hibernate OS state where the OS state is saved on the hard drive.

    Standby OS state where the OS state is saved on memory and resumed from the memory when mouse/keyboard is clicked.

    Shut Down All power is off for the host machine however the power cord is still connected.

    M0 FW power state where all HW power planes are activated host power state is S0.

    M1 FW power state where all HW power planes are activated however the host power state is different than S0 (Some host power planes are not activated). Host PCIe* interface are unavailable to the host SW.

    M-Off No power is applied to the management processor subsystem. FW is shut down.

    Ref Document Name File/Location

    [IPMI] Intelligent Platform Management Interface Specification, version 2.0, 2004.

    http://www.intel.com/design/servers/ipmi/spec.htm

    [IPMB] Intelligent Platform Management Bus Specification, version 1.0, 1999.

    http://www.intel.com/design/servers/ipmi/spec.htm

    [PET] IPMI Platform Event Trap Format Specification, version 1.0, 1999.

    http://www.intel.com/design/servers/ipmi/spec.htm

  • Introduction

    8 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    control power consumption. Intel® Intelligent Power Node Manager enables data center power and thermal management by exposing an external interface to management software through which platform policies can be specified. It also enables specific data center power management usage models such as power limiting.

    The configuration and control commands are used by the external management software or BMC to configure and control the Intel® Intelligent Power Node Manager feature. Since Platform Services firmware does not have any external interface, external commands are first received by the BMC over LAN and then relayed to the Platform Services firmware over IPMB channel. The BMC acts as a relay and the transport conversion device for these commands. For simplicity, the commands from the management console might be encapsulated in a generic CONFIG packet format (config data length, config data blob) to the BMC so that the BMC doesn’t even have to even parse the actual configuration data.

    BMC provides the access point for remote commands from external management SW and generates alerts to them. Intel® Intelligent Power Node ManagerIntel® Intelligent Power Node Manager on Intel® Manageability Engine (Intel® ME)is an IPMI satellite controller. A mechanism needs to exist to forward commands to Intel® ME and send response back to originator. Similarly events from Intel® ME have to be sent as alerts outside of BMC. It is the responsibility of BMC to implement these mechanisms for communication with Intel® Intelligent Power Node Manager.

    The rest of the sections describe the details of these interfaces. The details include list of commands, sensors exposed, alerts exposed the requirement on BMC to support passing the commands and alerts to/from external SW, and about discovery of Intel® Intelligent Power Node Manager Functionality by external SW.

    1.3.1 Use Cases

    In this section we describe the key usage models of Intel® Intelligent Power Node Manager 1.5 (see Table 1-2 below).

    To effectively manage power, we must be able to measure it and then control its usage as necessary. For this reason, Intel® Intelligent Power Node Manager is a power management technology that has two usage models: a) platform power and thermal monitoring, and b) platform power limiting.

    Intel® Intelligent Power Node Manager can be used to monitor platform power and temperature over a period so that it can understand actual power usage patterns and actual thermal behavior of a server, and hence a group of servers in a data center. Once the power usage pattern is understood, appropriate power control policies can be established and specified for Intel® Intelligent Power Node Manager to enforce.

    Intel® Intelligent Power Node Manager controls power by enforcing one or more policies that it has received as inputs in the form of policy directives. Intel® Intelligent Power Node Manager policy directive specifies at least the following:

    Table 1-2. Intel® Intelligent Power Node Manager 1.5 Usage Models and Use Cases

    Usage Model Use Cases

    Power and Thermal Monitoring Platform power monitoring

    Inlet temperature monitoring

    Platform Power Budgeting Power limiting to optimize rack utilization

    Power limiting triggered by inlet temperature threshold

    Power limiting to prevent circuit breaker tripping

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 9

    Introduction

    1. Power Budget: This specifies the power budget allocated to the node in watts.

    2. Correction Time: This specifies the upper time limit that actual power consumption should not exceed the specified budget: this mean that spikes in actual power consumption are allowed as long as they do not exceed the correction time.

    3. Policy Duration: This specifies how long the policy should be maintained. If this is not specified, Intel® Intelligent Power Node Manager assumes that the policy is in effect until it is killed or replaced by another policy.

    1.3.2 Intel® Intelligent Power Node Manager Use Cases

    This section describes the specific use cases for Intel® Intelligent Power Node Manager 1.5 (see Table 1-2)

    1.3.2.1 Platform Power Monitoring

    Mandatory/Optional: Mandatory

    Description: Intel® Intelligent Power Node Manager monitors platform power consumption and hold average power over duration. It can be queried to return actual power at any given instance.

    Usage: External management software periodically queries Intel® Intelligent Power Node Manager for actual wall power consumed in watts.

    Actors: Management software

    Preconditions: Intel® Intelligent Power Node Manager is operational and a management console is also running.

    Basic Flow of Events:

    1. Management software issues a query to Intel® Intelligent Power Node Manager to gather current actual platform power profile

    2. Intel® Intelligent Power Node Manager gets current platform power, average power, min, max, current budget

    3. Intel® Intelligent Power Node Manager returns the data in response to query

    4. Management console uses the query to adjust its behavior (for example, how frequently to poll Intel® Intelligent Power Node Manager, change power directives).

    Alternate Flows (event/alerts):

    5. Instead of querying by polling Intel® Intelligent Power Node Manager, Management console decides to let Intel® Intelligent Power Node Manager notifies when significant thresholds are crossed.

    6. Management console subscribes to one of the threshold-crossing events advertised by Intel® Intelligent Power Node Manager.

    7. Intel® Intelligent Power Node Manager returns subscription Id as an indication of successful subscription.

    8. Intel® Intelligent Power Node Manager sends alert notifications to subscribed management console when specified thresholds are reached.

    9. Management console uses the alert notification to decide to take appropriate action (start polling more frequently, adjust power policy directives, or other action)

  • Introduction

    10 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    10. When management console decides that it no longer needs to be notified of the alerts, it sends an unsubscribe request to Intel® Intelligent Power Node Manager.

    11. Intel® Intelligent Power Node Manager responds with a return code indicating success or failure of the request.

    Exception Paths

    12. None.

    Post conditions: Intel® Intelligent Power Node Manageris operational and management software is also running.

    1.3.2.2 Inlet Temperature Monitoring

    Mandatory/Optional: Mandatory

    Description: Intel® Intelligent Power Node Manager monitors server inlet temperatures periodically. If there is an alert threshold in effect, then Intel® Intelligent Power Node Manager issues an alert when the inlet (room) temperature exceeds the specified value. The threshold value can be set by policy, or it can be pre-configured to a default value such at the ASHRAE limit.

    Usage: This use case allows management software (and hence IT) to monitor room ambient temperature and detect hotspots and cooling system malfunctioning. For instance, if inlet temperature does exceeds that temperature that the cooling system is supposed to deliver to the server, this could be an indication that the cooling system is not functioning correctly and an alert from Intel® Intelligent Power Node Manager allows IT to become aware of this situation and decide on an appropriate corrective action.

    Actors: Management software

    Preconditions: Intel® Intelligent Power Node Manager is operational and a management console is also running.

    Basic Flow of Events:

    1. Management software monitors Intel® Intelligent Power Node Manager for alerts to gather information about platform inlet thermal state so that it can determine thermal condition of room.

    — Intel® Intelligent Power Node Manager reads inlet temperature.

    2. Intel® Intelligent Power Node Manager returns an alert and temperature values when the inlet sensor exceeds a user-settable limit or the ASHRAE limit

    Alternate Flows (event/alerts):

    Exception Paths

    None.

    Post conditions: Intel® Intelligent Power Node Manager is operational and a management software is also running.

    1.3.2.3 Platform Power Limiting (Abstract)

    Description: This use case is the platform power control capability provided by Intel® Intelligent Power Node Manager 1.5.

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 11

    Introduction

    The expected usage of this capability is to allow external management software to address key IT issues by setting a power budget for each server. For example, if there is a physical limit on the power available in a room, then IT can decide to allocate power to different servers based on their usage – servers running critical systems can be allowed more power than servers that are running less critical workload.

    Specifically, power limits can be set to address the following data center scenarios:

    1. Power limiting to increase rack population (see below)

    2. Power limiting triggered to maintain inlet temperature threshold (see below)

    3. Power limiting to prevent circuit breaker tripping (see below)

    4. Power limiting on OS failure (see below)

    5. Power limiting at boot time (see below)

    Although these scenarios are variations of the power limiting use case described in this section, there are key differences that influence how they are implemented; for this reason, each of them have been described as separate use cases below.

    Intel® Intelligent Power Node Manager maintains platform power budget that has been specified as a policy directive input. Intel® Intelligent Power Node Manager enforces the power limit by ensuring that the limit is not exceeded beyond the grace period at any time during the duration for which the policy is in effect.

    In maintaining the power cap, the optimal implementation of this policy will achieve the best performance at any given power level. This means that Intel® Intelligent Power Node Manager should be intelligent in its power capping method and mechanisms.

    Usage:

    External management software has made the decision to set a power cap for a node. This power budget is specified to Intel® Intelligent Power Node Manager as a policy directive input. Intel® Intelligent Power Node Manager then maintains the budget by enforcing the specified policy.

    If Intel® Intelligent Power Node Manager is unable to maintain the budget by reducing the power consumption of the platform, it will notify the external management software with an alert. In this situation, the management software can take a more drastic corrective action such as, reallocating power budget for this node, bringing additional servers on line, migrating applications, or even powering down the server.

    Actors: Management software

    Preconditions: Node is up and running an ACPI compliant operating system and power consumption is within limits.

    Basic Flow of Events:

    1. Management software sets a power budget to the node.

    2. Intel® Intelligent Power Node Manager validates the policy

    3. Intel® Intelligent Power Node Manager executes a closed loop control:i. Intel® Intelligent Power Node Manager monitors the power consumption of

    the entire node.ii. When power consumption reaches the allocated limit, Intel® Intelligent

    Power Node Manager runs its decision algorithm to determine the appropriate action to bring the power level to within the limit.

  • Introduction

    12 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    iii. Intel® Intelligent Power Node Manager, executes the action determined by the decision algorithm

    4. In Intel® Intelligent Power Node Manager cannot maintain the budget, it send an alert to the management software.

    Alternate Flows:3a. If in step 3, Intel® Intelligent Power Node Manager determines that power

    consumption is below the lower threshold, Intel® Intelligent Power Node Manager will relax the limits on performance states.

    Exception Paths3b. If in step 3, Intel® Intelligent Power Node Manager determines that there

    are no performance states or throttling states available that will reduce the power consumption of the platform, it will notify the external management console about this exception.

    Post conditions: Power consumption of node stays within limits.

    1.3.2.4 Platform Power Limiting – Typical Usage

    Mandatory/Optional: Mandatory

    Description: This use case is a specialization of the abstract platform power limiting use case.

    This is the typical usage of platform power capping. The power limit may be set for the following reasons:

    a. To increase number of servers per rack. The datacenter manager (or management software) has determined a power budget to allocate to each node in the rack so that he can maximize the number of severs in the rack.

    b. To set an upper boundary for node power consumption. In this case, after monitoring node power usage patter for a period, the datacenter manager has identified the min, max and average power consumption of the node. Data center manager can then set a value between the min and max as desired to achieve his goal.

    c. To allocate power to different nodes in a power constrained environment. This allows the datacenter manager to allocate different power budgets to different servers based on the criticality of the workload they are running, or the time of day etc.

    The power budget assigned to this node is the specified as a policy directive. The following values must be specified in the policy directive input:

    a. Power limit: specifies power budget allocated to node

    b. Grace period: specifies upper time limit before corrective action must be taken. This is expected to be a few seconds

    c. Trigger: typical usage

    Usage: External management software has made the decision to set a power cap for a node. This power budget is specified to Intel® Intelligent Power Node Manager as a policy directive input. Intel® Intelligent Power Node Manager then maintains the budget by enforcing the specified policy.

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 13

    Introduction

    If Intel® Intelligent Power Node Manager is unable to maintain the budget by reducing the power consumption of the platform, it will notify the external management software with an alert. In this situation, the management software can take a more drastic corrective action such as, reallocating power budget for this node, bringing additional servers on line, migrating applications, or even powering down the server.

    Actors: Management software

    Preconditions: Node is up and running an ACPI compliant operating system and power consumption is within limits.

    Basic Flow of Events:

    1. Management software sets a power budget to the node.

    2. Intel® Intelligent Power Node Manager validates the policy

    3. Intel® Intelligent Power Node Manager executes a closed loop control:

    a. Intel® Intelligent Power Node Manager periodically monitors the power consumption of the entire node.

    b. When power reaches the limit Intel® Intelligent Power Node Manager runs its decision algorithm to determine the appropriate action to bring the power level to within the limit.

    c. Intel® Intelligent Power Node Manager, executes the action determined by the decision algorithm

    4. If Intel® Intelligent Power Node Manager cannot maintain the budget, it sends an alert to the management software.

    Alternate Flows:

    Exception Paths

    Post conditions: Power consumption of node stays within limits.

    1.3.2.5 Platform Power Limiting to Maintain Inlet Temperature Threshold

    Mandatory/Optional: Mandatory

    Description: This use case is a specialization of platform power limiting use case.

    In this case the power limit is triggered when the room temperature reaches a threshold that has been specified to Intel® Intelligent Power Node Manager by a policy directive threshold. The policy directive input must include the following values:

    1. Power limit: the allocated power budget

    2. Grace period: specifies upper time limit before corrective action must be taken. This is expected to be a few seconds

    3. Inlet temperature threshold: specifies the temperature value that triggers the power throttling corrective action

    4. Trigger: inlet temperature threshold

    Usage: External management software has made the decision to set a power cap for a node. This power budget is specified to Intel® Intelligent Power Node Manager as a policy directive input. Intel® Intelligent Power Node Manager then maintains the budget by enforcing the specified policy.

  • Introduction

    14 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    If Intel® Intelligent Power Node Manager is unable to maintain the budget by reducing the power consumption of the platform, it will notify the external management software with an alert. In this situation, the management software can take a more drastic corrective action such as, reallocating power budget for this node, bringing additional servers on line, migrating applications, or even powering down the server.

    Actors: Management software

    Preconditions: Node is up and running an ACPI compliant operating system and power consumption is within limits.

    Basic Flow of Events:

    1. Management console software sets a temperature inlet limit for the node.

    2. Node monitors Tin.

    3. If Tin reaches an allocated limit, Intel® Intelligent Power Node Manager evaluates its algorithm to determine which action to take

    a. If Tin > user settable value, Intel® Intelligent Power Node Manager evaluates which components can be placed in lower performance state and attempts to limit power to a

    b. If Tin > user settable value, (for example, 32C (ASHRAE allowable limit), Intel® Intelligent Power Node Manager will place the platform in it’s lowest power envelope (or Minimum Power)

    c. If Tin > user settable value, (for example, 35C (Intel Spec), Intel® Intelligent Power Node Manager will direct the BMC to execute a system shutdown. This feature may be disabled by the end-user.

    4. Intel® Intelligent Power Node Manager with the help of OS (OSPM), places the components in appropriate performance states or throttling states to maintain platform within the defined power envelope per step 3.

    5. Power consumption stays within the allocation or platform is shutdown.

    Alternate Flows:

    1. If in flow above, Intel® Intelligent Power Node Manager determines that Tin is back below the set thresholds, Intel® Intelligent Power Node Manager will relax the limits on performance states.

    Exception Paths

    2. If in step 3, Intel® Intelligent Power Node Manager determines that there are no performance states or throttling states available that will reduce the power consumption of the platform, it will notify the external management console about this exception. It is expected that the management console will take extraordinary actions (like reallocating power budget for this node, bringing additional servers on line, migrating applications) to mitigate the power budget violation.

    Post conditions: Power consumption of node stays reduced due to thermal challenge.

    Notes: Node limit includes user-settable value as well as ASHRAE guideline (32C), a user settable power values applies to the user settable temperature. At the 32C, power levels will be set to Minimum Power as determined by Intel® Intelligent Power Node Manager. As a minimum the ASHRAE guideline is in effect if this feature is turned on.

    1.3.2.6 Power Limiting to Prevent Circuit Breaker Tripping

    Mandatory/Optional: Optional

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 15

    Introduction

    Description: This use case is a specialization of platform power limiting use case.

    In this case the power limit is set to prevent a circuit breaker from tripping. In this case, the following values must be specified in the policy directive input:

    • Power limit: specifies a value at or below the maximum power that the circuit supports.

    • Grace period: specifies the upper time limit before the circuit breaker trips when power is drawn above its limit

    • Inlet temperature threshold: specifies the upper temperature value that indicates a higher room temperature than expected.

    • Trigger: circuit breaker

    Usage: External management software has made the decision to set a power cap for a node. This power budget is specified to Intel® Intelligent Power Node Manager as a policy directive input. Intel® Intelligent Power Node Manager then maintains the budget by enforcing the specified policy.

    If Intel® Intelligent Power Node Manager is unable to maintain the budget by reducing the power consumption of the platform, it will notify the external management software with an alert. In this situation, the management software can take a more drastic corrective action such as, reallocating power budget for this node, bringing additional servers on line, migrating applications, or even powering down the server.

    Actors: Management software

    Preconditions: Node is up and running an ACPI compliant operating system and power consumption is within limits.

    Basic Flow of Events:

    1. Management software sets a power budget to the node.

    2. Intel® Intelligent Power Node Manager validates the policy

    3. Intel® Intelligent Power Node Manager executes a closed loop control:

    • Intel® Intelligent Power Node Manager monitors the power consumption of the entire node.

    • When power consumption reaches the allocated limit, Intel® Intelligent Power Node Manager runs its decision algorithm to determine the appropriate action to bring the power level to within the limit.

    • Intel® Intelligent Power Node Manager, executes the action determined by the decision algorithm

    4. Intel® Intelligent Power Node Manager cannot maintain the budget, it send an alert to the management software.

    Alternate Flows:

    5. If in step 3, Intel® Intelligent Power Node Manager determines that power consumption is below the lower threshold, Intel® Intelligent Power Node Manager will relax the limits on performance states.

    Exception Paths

    6. If in step 3, Intel® Intelligent Power Node Manager determines that there are no performance states or throttling states available that will reduce the power consumption of the platform, it will notify the external management console about this exception.

  • Introduction

    16 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    Post conditions: Power consumption of node stays within limits.

    1.3.2.7 Blade Power Limiting (Maintain allocated power per blade)

    Mandatory/Optional: Mandatory

    Description: To limit chassis-level power consumption with support from enterprise blades. Currently blade servers do not dynamically maintain power consumption to an allocated level. (They statically determine a performance level and stay there; this results in a performance loss that could be avoided by dynamically maintaining power using Intel® Intelligent Power Node Manager technology).

    Usage: External management software will allocate chassis level power budget and communicate it to Chassis Management Module (CMM). The CMM will divide the power budget among the blades that are in the chassis and allocate power per blade based on the needs/configuration of each blade. The blade will then maintain the power budget allocated to it and communicate exceptions to the CMM.

    Actors: CMM, Management software

    Preconditions: Blade is inserted into the state and is in M2, M3, M4 or M5 state.

    Basic Flow of Events:

    1. Management software sets a power budget to the Chassis. The CMM is aware of this limit.

    2. Blade reports its power requirement to CMM during power negotiation at boot time (M2/M3 blade operational state).

    3. Blade gets its power allocation from CMM before going to active (M4 blade operational) state.

    4. Blade monitors power consumed by sensing the current and voltage at the connection to the backplane

    5. When power consumed exceeds allocated limit, blade will limit its power consumption by limiting processor power consumption by using a combination of power limiting knobs.

    Alternate Flows: None

    Exception Paths

    Blade does not get any of the power it requested – blade stays/goes to inactive (M1 blade operational) state

    Unable to maintain limit: When power is exceeded despite lowest P-state, power renegotiation with CMM happens (Note: The request to allocate more power may or may not be granted).

    Post conditions: Power consumption by the blade is within the allocated limits.

    Notes:

    • Power supply is only at chassis-level

    • Presence of current/voltage sensors at the backplane connection

    • Presence of thermal sensor at air inlet to the blade

    • Blade has a microcontroller on which to run NPTM policy engine

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 17

    Introduction

    1.3.3 Features

    The features of Intel® Intelligent Power Node Manager 1.5 are grouped into the following categories: policy management, monitoring and querying, alerts and notifications, external interface protocol. The policy management features implement specific IT goals that can be specified as policy directives for a management software. Monitoring and querying are the features that enable tracking of power consumption. Alerts and notifications provide the foundation for automation of power management in the data center management stack. The external interface specifies the protocols that must be supported in this version of Intel® Intelligent Power Node Manager. The rest of this section describes these features and identify those that are optional in version 1.5; all others are mandatory.

    1.3.3.1 Policy Management

    1.3.3.2 Monitoring and Querying

    Table 1-3. Policy Management Feature

    Category Feature Description

    Policy Management

    Maintain platform power budget

    This is used when IT wants to allocate a power budget to a node for a given duration. See specific use cases above.

    Maintain policy-free period This allows IT to specify a period during which no policy should be applied. For the duration specified, Intel® Intelligent Power Node Manager does not execute any policy.

    Maintain priority: performance or power.

    This feature allows IT to specify the desired optimization priority for the platform. This directive allows Intel® Intelligent Power Node Manager to resolve conflicts between multiple policies. Whichever priority is set takes precedence over the others.

    Maintain concurrent policy in a single domain

    This means that multiple power control policies can be in effect at the same time, and Intel® Intelligent Power Node Manager will be able to enforce them concurrently without violating any of them.

    Table 1-4. Monitoring and Querying Feature

    Category Feature Description

    Monitoring and Querying

    Platform power consumption: point in time or average over an interval

    This allows IT to monitor actual power consumption either at a given point in time or as an average over a time duration. Intel® Intelligent Power Node Manager reads, calculates and returns the appropriate value in Watts. Intel® Intelligent Power Node Manager will also be able to return the Min and Max values over the time interval.

    Current policy directives This is used to read the policy directives that are currently active within Intel® Intelligent Power Node Manager.

  • Introduction

    18 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    1.3.3.3 Alerts and Notifications

    1.3.3.4 External Interface

    Power management capability This is used to read the power capability available for this platform.

    Current inlet temperature This is used to gauge room inlet air temperature cooling this platform and to provide outlet temperatures being returned to the data center. (Note that the outlet temperatures are only useful at a rack aggregate level, and this can only be determined by averaging airflow weighted average rack temperatures)

    Platform airflow (optional) This allows IT to monitor airflow in the server for planning and to ensure adequate cooling exists in the data center area of the server installation.

    Table 1-5. Alerts and Notifications Feature

    Category Feature Description

    Alerting Budget out of range Returned by Intel® Intelligent Power Node Manager when the power limit value is not with in the allowed minimum and maximum levels

    Power at threshold The user is allowed to set a power threshold so they can be warned when power consumption reaches this level. This alert is sent by Intel® Intelligent Power Node Manager when that threshold is reached.

    Started aggressive power throttling

    This notification is sent when Intel® Intelligent Power Node Manager enters or exits aggressive mode – this is when it starts to use T states (clock gating) to force processor power consumption to a lower level.

    Inlet temperature above user settable limits

    Returned by Intel® Intelligent Power Node Manager when the room exceeds IT set limits (including potential to use ASHRAE limits)

    Platform Thermal Alerts (Optional)

    Returned by Intel® Intelligent Power Node Manager when the platform experiences any of the thermal events or interrupts; potentially including events such as a) Processor hits PROCHOT, b) Processor hits THERMTRIP, c) Memory hits MEMHOT, d) Fan failure or fault

    Table 1-6. External Interface

    Category Feature Description

    External Interface Intel® Intelligent Power Node Manager discovery

    This allows the external management software to discover the presence of , its capabilities, and parameters necessary to communicate with it further.

    Queries This set of commands provide an interface for management software to query Intel® Intelligent Power Node Manager about the statistics of power consumption, thermal data, and policies currently stored in Intel® Intelligent Power Node Manager.

    Set policies/parameters This set of commands provide an interface for management software to set Intel® Intelligent Power Node Manager policies.

    Alerts These allow the Intel® Intelligent Power Node Manager to notify external management SW about exception conditions or programmed threshold violations.

    Table 1-4. Monitoring and Querying Feature

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 19

    Introduction

    1.3.3.5 System Architectures Supported

    §

    Table 1-7. Supported System Architectures

    Category Feature Description

    System Architecture Support

    Rackmount/pedestal servers These servers have individual power supplies and the base Intel® Intelligent Power Node Manager architecture applies

    Blade servers (optional) Blade servers share a common power supply and the base architecture must be modified to support blades

  • Introduction

    20 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    Requirements on Platform Components

    2 Requirements on Platform Components

    The following table depicts the dependencies Intel® Intelligent Power Node Manager has on the various platform components (ingredients):

    Table 2-1. Intel® Intelligent Power Node Manager Interfaces to Platform Components

    Mechanism Discover Monitor Control

    Discover platform information from BIOS

    BIOS and IPMI/HECI

    P System information at boot timeP Maximum platform airflow

    Power supply PMBus P Power consumption information from power supply

    Processor ACPI and IPMI/HECI

    P Modifying maximum processor P-state and T-state using in-band agent running on BIOS

    Table 2-2. Platform Ingredients for Intel® Intelligent Power Node Manager Support

    Platform Components Dependency Readiness

    CPU Support of P, T states Yes Supported. Minimum and Maximum # of number of P state for Intel® Xeon® 5500 Platform available.

    BMC/Intel® ME F/W SMI support Supported

    SCI support Supported

    BMC/Intel ME-BIOS comm. Protocol Supported

    Thermal monitoring (Inlet Temps, Thermal Alerts, Fan Status, Airflow)

    Supported

    BIOS SMI Handler Supported

    SCI Handler Supported

    BMC/Intel ME-BIOS comm. Protocol Supported

    Store MaxPower and MaxAirflow data Requires storage registers

    OS ACPI 2.0 compliant Implemented in Windows & Linux

    Platform Power Supply Power supply with PMBus Yes for new power supplies

    VMM Depends on VMM power mgmt arch. Supported

  • Requirements on Platform Components

    22 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    2.1 Intel® Intelligent Power Node Manager IPMI OEM CommandsThe following table describes the IPMI commands which support Intel® Intelligent Power Node Manager. These commands a used to discover and configure Intel® Intelligent Power Node Manager and collect statistics. The commands use the IPMI Network Function Code of 2Eh, which signifies that these commands are defined by a OEM or a group other than the IPMI group.

    The Intel® Intelligent Power Node Manager configuration shall be non-volatile and survive a cold-boot of the system.

    There are also additional standard commands related to sensor thresholds and enabling events (listed below). These are not elaborated here since they are not Intel® Intelligent Power Node Manager-specific.

    • Set Sensor Thresholds

    • Get Sensor Thresholds

    • Set Sensor Event Enable

    • Get Sensor Event Enable

    • Re-arm Sensor Events

    • Get Sensor Event Status

    • Get Sensor Reading

    • Platform Event Message

    • Alert Immediate

    The following table specifies the content of the command request and response for the commands in Table 1. In the table, all reserved bits will return 0 for a Get command and should be set to 0 for a Set command.

    Table 2-3. Intel® Intelligent Power Node Manager-specific OEM commands (IPMI)

    Command NetFn CMD M/E1Min

    PrivilegeLevel

    Enable/Disable Node Manager Policy Control 2Eh C0h M Admin

    Set Node Manager Policy 2Eh C1h M Admin

    Get Node Manager Policy 2Eh C2h M Admin

    Set Node Manager Alert Thresholds 2Eh C3h M Admin

    Get Node Manager Alert Thresholds 2Eh C4h M Admin

    Set Node Manager Policy Suspend Periods 2Eh C5h M Admin

    Get Node Manager Policy Suspend Periods 2Eh C6h M Admin

    Reset Node Manager Statistics 2Eh C7h M Admin

    Get Node Manager Statistics 2Eh C8h M Admin

    Get Node Manager Capabilities 2Eh C9h M Admin

    Get Node Manager Version 2Eh CAh M Admin

    Set Node Manager Power Draw Range 2Eh CBh M Admin

    Set Node Manager Alert Destination 2Eh CEh M Admin

    Get Node Manager Alert Destination 2Eh CFh M Admin

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 23

    Requirements on Platform Components

    Because NetFn = 2Eh, the first three data bytes in a request and the first three data bytes after completion code in response structures have the value 000157h, which is the IANA code of the Intel Corporation.

    Net Function = 2Eh, LUN = 00b

    Code Command Request, Response Data Description

    C0h Enable/Disable Node Manager Policy Control

    RequestByte 1:3 = Intel manufacturers ID – 0x000157, LS byte firstByte 4 – Flags[0:2] – Policy Enable/Disable=0x00 – Global Disable Node Manager policy control=0x01 – Global Enable Node Manager policy control =0x02 – Per Domain Disable Node Manager policies for the domain given by Byte 5=0x03 – Per Domain Enable Node Manager policies for the domain given by Byte 5=0x04 – Per Policy Disable Node Manager policy given by Byte 6 within Domain given by Byte 5=0x05 – Per Policy Enable Node Manager policy given by Byte 6 within Domain given by Byte 5[3:7] – ReservedByte 5 – Domain Id[0:3] = Domain Id (Currently, supports only one domain, Domain 0). This field is valid if Per Policy Enable/Disable is set or Per Domain Enable/Disable is set.[4:7] - ReservedByte 6 – Policy IdThis field is valid if Per Policy Enable/Disable is set.

    Enable or Disable the Node Manager policy control feature.

    Global enable/disable affects all policies for all domains.

    Per Domain enable/disable affects all policies of the specified domain.

    Per Policy enable/disable affects only the policy for the specified domain/policy combination.

    ResponseByte 1 – completion code=00h – Success=80h – Invalid Policy Id=81h – Invalid Domain IdByte 2:4 = Intel manufacturers ID – 0x000157, LS byte first

  • Requirements on Platform Components

    24 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    C1h Set Node Manager Policy

    RequestByte 1:3 = Intel manufacturers ID – 0x000157, LS byte first

    Byte 4 – Domain Id | Reserved[0:3] = Domain Id (Identifies the domain that this Node Manager policy applies to. Currently, supports only one domain, Domain 0.)[4] = Policy Enabled (set to 1 if policy should be enabled by default during policy creation/modification). Policy will be enforced (enabled and evaluated in runtime) if the corresponding Per Domain control as well as Global control is already enabled see C0h command.[5:7] = Reserved (should be set to 0)

    Byte 5 – Policy Id

    Byte 6 – Policy Configuration Action| Policy Trigger Type[0:3] – Policy Trigger Type=0 – No Policy Trigger, (In that case Policy Trigger Limit should be ignored)=1 – Inlet Temperature Limit Policy Trigger in [Celsius] [4:7] – Policy Configuration Action=0 - Policy Pointed by Policy Id shall be removed (remaining bytes shall be ignored on read). Corresponding (with the same Policy Id) Alert Thresholds and Suspend Periods will be removed as well.=1 - Add Power Policy. This command creates/modifies policy of type that will maintain Power limit

    Byte 7 – Policy Exception Actions(if maintained policy power limit given by bytes 8-9 is exceeded over Correction Time Limit)[0] – send alert[1] – shutdown system (hard shutdown via BMC)[2:7] – reserved

    Bytes 8-9 – Power Limit. This field contains power to be maintained in [Watts]

    Bytes 10:13 – Correction Time Limit -The max time in ms, in which the Node Manager must take corrective actions in order to bring the platform back within the specified power limit before taking the action specified in the “Policy Exception Action” parameter.

    The time is counted from the moment when the average power consumption exceeds the power limit. The average power is calculated as arithmetic moving average with the time period equal to the half of Correction Time Limit. It means that Node Manager may take the exception action after the time period equal to 1.5 of Correction Time Limit parameter starting from the moment when instantaneous power crossed the power limit.

    Bytes 14:15 – Policy Trigger Limit If Byte 6 bits [0:3] is:0 – Policy Trigger Value will be ignored1 – Policy Trigger Value should define the Inlet temperature in Celsius. The value (inlet temperature of the system) as specified by trigger type will be compared against this limit and if exceeded, cause a trigger to start enforcing the Power Limit specified (Power limit will not be enforced until the trigger happens).Bytes 16:17 – Statistics Reporting Period in seconds. The number of seconds that the measured power will be averaged over for the purpose of reporting statistics to external management SW. This is a moving window. Note that this value is different from the period that Node Manager uses for maintaining an average for the purpose of power control.

    User can specify any valid PolicyId. If already existing, this command will overwrite/modify the parameters for the existing policy, otherwise a new policy will be created with this policy Id. Modification is possible only if that policy for the specified PolicyId is disabled.Note: The operator may define a special kind of Inlet Air Temperature policy called Minimum Power Consumption policy with the Power Limit set to 0. The policy does not have the power limit defined. When the inlet air temperature raises above the trigger value defined in the policy, the SPS firmware reduces the power consumption to minimum by requesting OSPM or SMM to set minimum P-state and T-state. The Minimum Power Consumption policy does not allow setting the correction action to “System Shutdown”, but the operator can specify whether the SPS firmware shall minimize power consumption by only reducing P-state or the firmware shall use both P-state and T-state

    Net Function = 2Eh, LUN = 00b

    Code Command Request, Response Data Description

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 25

    Requirements on Platform Components

    ResponseByte 1 – completion code=00h – Success =80h – Invalid Policy Id=81h – Invalid Domain Id =82h – unknown or unsupported Policy Trigger Type=83h – unknown or unsupported Policy Configuration Action=84h – Power Limit out of range=85h – Correction Time out of range=86h – Policy Trigger value out of range=89h – Statistics Reporting Period out of range=D5h – Policy could not be updated since PolicyId already exists and is enabled.Byte 2:4 = Intel manufacturers ID – 0x000157, LS byte first

    Net Function = 2Eh, LUN = 00b

    Code Command Request, Response Data Description

  • Requirements on Platform Components

    26 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    C2h Get Node Manager Policy

    RequestByte 1:3 = Intel manufacturers ID – 0x000157, LS byte firstByte 4 – Domain Id [0:3] = Domain Id (Currently, supports only one domain, Domain 0)[4:7] = Reserved (should be set to 0)Byte 5 – Policy Id

    Gets the Node Manager policy parameters.

    ResponseByte 1 -- completion code=00h – Success =80h – Invalid Policy Id=81h – Invalid Domain IdByte 2:4 = Intel manufacturers ID – 0x000157, LS byte firstByte 5 – Domain Id | Policy state | Reserved[0:3] = Domain Id (Identifies the domain that this Node Manager policy applies to. Default is ‘0000b’. Currently, supports only one domain, Domain 0.)[4] = Policy enabled[5] = per Domain Node Manager policy control enabled[6] = Global Node Manager policy control enabled[7] = Reserved (should be set to 0)Byte 6 – Policy Type | Policy Trigger Type[0:3] – Policy Trigger Type=0 – No Policy Trigger, Policy will maintain Power limit (In that case Policy Trigger Value will be equal to the Power Limit)=1 – Inlet Temperature Limit Policy Trigger in [Celsius][4:7] – Policy Type=1 - Power Control PolicyByte 7 – Policy Exception Actions(if maintained policy power limit given by bytes 8-9 is exceeded over Correction Time Limit)[1] – shutdown system[0] – send alert [2:7] – reserved

    Bytes 8-9 – Power Limit. This field contains power to be maintained in [Watts]Bytes 10:13 – Correction Time Limit - the max time in ms, in which Node Manager must take corrective actions in order to bring the platform back within the specified power limit before taking the action specified in the “Policy Exception Action” parameter.The time is counted from the moment when the average power consumption exceeds the power limit. The average power is calculated as arithmetic moving average with the time period equal to the half of Correction Time Limit. It means that Node Manager may take the exception action after the time period equal to 1.5 of Correction Time Limit parameter starting from the moment when instantaneous power crossed the power limit.Bytes 14:15 – Policy Trigger Limit The value (inlet temperature of the system) as specified by trigger type will be compared against this limit and if exceeded, cause a trigger to start enforcing the Power Limit specified (Power limit will not be enforced until the trigger happens).If Byte 6 bits [0:3] is 0 this field contains the same value as Power Limit (that is, it does not contain the trigger value passed to Node Manager using Set Node Manager Policy).

    Bytes 16:17 – Statistics Reporting Period The number of seconds that the measured power will be averaged over for the purpose of reporting statistics to external management SW. This is a moving window. Note that this value is different from the period that Node Manager uses for maintaining an average for the purpose of power control.

    Net Function = 2Eh, LUN = 00b

    Code Command Request, Response Data Description

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 27

    Requirements on Platform Components

    C3h Set Node Manager Alert Thresholds

    RequestByte 1:3 = Intel manufacturers ID – 0x000157, LS byte firstByte 4 – Domain Id[0:3] = Domain Id (Currently, supports only one domain, Domain 0.)[4:7] = Reserved (should be set to 0)Byte 5 – Policy IdByte 6 – Number of alert thresholdsBytes 7:N – Alert threshold array (the array length is based on the number of thresholds given in the byte 6). Node Manager will generate the event if the average power or temperature based on the policy trigger (computed over an averaging period derived based on correction time limit) exceeds any of the configured alert thresholds. (assert for exceeding (going high) and desertion for going low). The hysteresis value for avoiding jitters around the threshold will be OEM configurable using factory-preset values.Note: Max 3 alert thresholds are supported per policy. Each alert threshold is 2 bytes in length (LSB first). If number of alert thresholds is 0 then the previously set alert thresholds (if present) are removed from the policy. Alert thresholds shall be provided in units defined for trigger in given Policy Id [Celsius] (or [Watts] for policy without trigger)

    Sets the Node Manager alert thresholds. This is part of the Node Manager Policy described earlier and applies to the same policy as specified by PolicyId.

    ResponseByte 1 — completion code=00h – Success =80h – Invalid Policy Id=81h – Invalid Domain Id=84h – Limit in one of thresholds is invalid =87h – Invalid Number of Policy Thresholds=D5h – Alert thresholds can not be changed for enabled policy, disable it firstByte 2:4 = Intel manufacturers ID – 0x000157, LS byte first

    Net Function = 2Eh, LUN = 00b

    Code Command Request, Response Data Description

  • Requirements on Platform Components

    28 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    C4h Get Node Manager Alert Thresholds

    RequestByte 1:3 = Intel manufacturers ID – 0x000157, LS byte firstByte 4 – Domain Id[0:3] = Domain Id (Currently, supports only one domain, Domain 0)[4:7] = Reserved (should be set to 0)Byte 5 – Policy Id

    Gets the Node Manager alert thresholds

    ResponseByte 1 – completion code=00h – Success =80h – Invalid Policy Id=81h – Invalid Domain Id Byte 2:4 = Intel manufacturers ID – 0x000157, LS byte firstByte 5 – Number of alert thresholdsBytes 6:N – Alert threshold array (the array length is based on the number of threshold given in the byte 5)). If number of alert thresholds is 0 then the array length is 0 bytes.Note: Max 3 alert thresholds are supported per policy. Each alert threshold is 2 bytes in length (LSB first). Alert thresholds are provided in units defined for trigger in given Policy Id [Celsius] (or [Watts] for policy without trigger).

    Net Function = 2Eh, LUN = 00b

    Code Command Request, Response Data Description

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 29

    Requirements on Platform Components

    C5h Set Node Manager Policy Suspend Periods

    Request:Byte 1:3 = Intel manufacturers ID – 0x000157, LS byte firstByte 4 – Domain Id[0:3] = Domain Id (Currently, supports only one domain, Domain 0)[4:7] = Reserved (should be set to 0)Byte 5 – Policy IdByte 6 – Number of policy suspend periods. This value should be specified as 0 if all the suspend periods are to be removed (if previously set).Bytes 7:N – array of policy suspend periods (following information is repeated for each suspend period). Each suspend period is defined by 3 bytes:1st byte – Policy suspend start time =0 – 239 number of minutes from mid-night divided by 6= 240 – 255 reserved2nd byte – Policy suspend stop time = 0 reserved=1 – 240 number of minutes from mid-night divided by 6= 241 – 255 reserved3rd byte – Suspend period recurrence pattern:[0] – repeat the suspend period every Monday[1] – repeat the suspend period every Tuesday[2] – repeat the suspend period every Wednesday[3] – repeat the suspend period every Thursday[4] – repeat the suspend period every Friday[5] – repeat the suspend period every Saturday[6] – repeat the suspend period every Sunday[7] – reserved. Write as 0bNote: Policy suspend start and stop time is 1 byte in length each. Max 5 suspend periods can be specified per policy. If the number of policy suspend period (that is, byte 6) is 0 then the rest of the bytes in the request message are not required and previously configured suspend periods are removed from the system for the specified policy Id.The suspend periods are specified as an array. For example, if policy suspend start time is in byte 7 then byte 8 will contain the policy suspend stop time and byte 9 will contain suspend period recurrence pattern. Similarly, if the 2nd set of suspend periods are to be specified then they will be present in bytes 10:12.The suspend times are encoded on one byte each as number of minutes from mid-night divided by 6 to fit into one byte. If there is a need to specify an end-time that is beyond midnight, use two suspend periods, one ending at midnight (suspend stop time byte set to 240) and one from midnight (suspend start time set to 0) until the necessary end-time of the next day.

    Sets the Node Manager policy suspend period (during which no platform power policy control will be enforced)

    ResponseByte 1 – completion code=00h – Success =80h – Invalid Policy Id=81h – Invalid Domain Id =85h – One of periods in the table is inconsistent. Start time is greater than or equal to stop time or stop time sets time beyond 1 day=87h – Invalid Number of policy suspend periods =D5h – Suspend periods can not be changed for enabled policy, disable it first.Byte 2:4 = Intel manufacturers ID – 0x000157, LS byte first

    Net Function = 2Eh, LUN = 00b

    Code Command Request, Response Data Description

  • Requirements on Platform Components

    30 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    C6h Get Node Manager Policy Suspend Periods

    RequestByte 1:3 = Intel manufacturers ID – 0x000157, LS byte firstByte 4 – Domain Id[0:3] = Domain Id (Currently, supports only one domain, Domain 0)[4:7] = Reserved (should be set to 0)Byte 5 – Policy Id

    Get the Node Manager suspend periods

    ResponseByte 1 – completion code=00h – Success=80h – Invalid Policy Id=81h – Invalid Domain IdByte 2:4 = Intel manufacturers ID – 0x000157, LS byte firstByte 5 – Number of policy suspend periodsBytes 6:N – array of suspend periodsEach suspend period is defined by 3 bytes:1st byte – Policy suspend start time encoded as a number of minutes from mid-night divided by 62nd byte – Policy suspend stop time encoded as a number of minutes from mid-night divided by 63rd byte – Suspend period recurrence pattern:[7] – reserved. Write as 0b[6] – repeat the suspend period every Sunday[5] – repeat the suspend period every Saturday[4] – repeat the suspend period every Friday[3] – repeat the suspend period every Thursday[2] – repeat the suspend period every Wednesday[1] – repeat the suspend period every Tuesday[0] – repeat the suspend period every MondayNote: If byte 5 is 0x00 then no subsequent bytes will be present in the response. This means there are no suspend periods configured for the specified policy Id.Note: The suspend periods are specified as an array. For example, if 1st suspend period is in bytes 6:8 then bytes 9:11 will contain the 2nd policy suspend period.

    C7h Reset Node Manager Statistics

    RequestByte 1:3 = Intel manufacturers ID – 0x000157, LS byte firstByte 4 – Mode[0:4] -Mode=0x00 – reset global statistics including power statistics and inlet temperature statistics=0x01 – per policy statistics including power and trigger statistics [5:7]– ReservedByte 5 – Domain Id[0:3] = Domain Id (Currently, FW supports only one domain, Domain 0)[4:7] = Reserved (should be set to 0)Byte 6 – Policy Id (ignored if in Byte 4 field Mode is set to 0x00)

    Set the Node Manager Power Statistics

    ResponseByte 1 – Completion code=00h – Success=80h – Invalid Policy Id=81h – Invalid Domain Id=88h – Invalid ModeByte 2:4 = Intel manufacturers ID – 0x000157, LS byte first

    Net Function = 2Eh, LUN = 00b

    Code Command Request, Response Data Description

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 31

    Requirements on Platform Components

    C8h Get Node Manager Statistics

    RequestByte 1:3 = Intel manufacturers ID – 0x000157, LS byte firstByte 4 –Mode[0:4] – Mode=0x01 – global power statistics in [Watts]=0x02 – global inlet temperature statistics in [Celsius]=0x03 – Reserved=0x04 – Reserved=0x11 – per policy power statistics in [Watts]=0x12 – per policy trigger statistics in [Celsius][5:7] – ReservedByte 5 – Domain Id[0:3] = Domain Id (Currently, FW supports only one domain, Domain 0)[4:7] = Reserved (should be set to 0)Byte 6 – Policy Id (not ignored if Byte 1 == 0x11 or Byte 1 == 0x12)

    Get the Node Manager Power StatisticsNote that the average values provided here may be different from the averaged values used by Node Manager for taking corrective action or triggering alerts based a ‘Set Node Manager Alert Threshold’ because the averaging period for the two could be different.

    Net Function = 2Eh, LUN = 00b

    Code Command Request, Response Data Description

  • Requirements on Platform Components

    32 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    ResponseByte 1 – Completion code=00h – Success =80h – Invalid Policy Id=81h – Invalid Domain Id=88h – Invalid Mode Byte 2:4 = Intel manufacturers ID – 0x000157, LS byte firstBytes 5:6 – CurrentBytes 7:8 – MinimumBytes 9:10 – MaximumBytes 11:12 – AverageBytes 13:16 – Timestamp as defined by the IPMI v2.0 specification. Indicates the time when the response to the commands was sent. If NM cannot obtain valid time, the timestamp is set to 0xFFFFFFFF as defined in the IPMI v2.0 specification.Bytes 17:20 – Statistics Reporting Period (the timeframe in seconds, over which the firmware collects statistics)Byte 21 – Domain Id | Policy State | Reserved[0:3] = Domain Id (Currently, supports only one domain, Domain 0)[4] = Policy/Global Administrative state if request Byte 1 == 0x11 or request Byte 1 == 0x12 state returns= 1 – If policy is enabled by user and Node Manager is Globally Enabled (see C0h command) and Node Manager Domain control is also Enabled (see C0h command).=0 - Otherwise.if request Byte 1 == 0x01 or request Byte 1 == 0x02= 1 if Node Manager is Globally Enabled (see C0h command).= 0 – Otherwise.[5] = Policy Operational state= 1 – Policy is actively monitoring defined trigger (power or thermal) and will start limiting target if defined trigger is exceeded.= 0 – Policy is suspended so it cannot actively limit defined target. It may happen if one of the defined below events happens. – suspend period is enforced; – there is a problem with trigger readings;– there is a host communication problem; – host is in Sx state;– host did not send End Of POST notification;– policy is administratively disabled [6] = Measurements state= 1 - Measurements in progress (host CPU is in S0 state and there are no problems with the readings reported to the remote console). = 0 - Measurements are suspended (host CPU in Sx state or in the S0 state problem with the readings reported to the remote console)[7] = Policy activation state=1 Policy is triggered and is actively limiting target. Note: This bit is set for thermal policies even if there are no power readings available.= 0 – Policy is not triggered. Note: If Field Mode == 0x01 in the request message then the response contains power values obtained from Power Budget Control.

    If Field Mode == 0x02 in the request message then the response contains inlet temperature values expressed in [Celsius]

    If Field Mode == 0x11 in the request message then the response contains power statistics values expressed in [Watts] and calculated according to given policy. . A request in this mode can be issued only for policies with no trigger defined. Otherwise, an error is returned.

    If Field Mode == 0x12 in the request message then the response contains policy trigger statistics values expressed in trigger units [Watts] or [Celsius] and calculated according to given policy. A request in this mode can be issued only for policies with thermal trigger defined. Otherwise, an error is returned.

    Net Function = 2Eh, LUN = 00b

    Code Command Request, Response Data Description

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 33

    Requirements on Platform Components

    C9h Get NodeManager Capabilities

    RequestByte 1:3 = Intel manufacturers ID – 0x000157, LS byte firstByte 4 – Domain Id[0:3] = Domain Id (Currently, supports only one domain, Domain 0)[4:7] = Reserved (should be set to 0)Byte 5 –Policy Type | Policy Trigger Type[0:3] – Policy Trigger Type =0 – No Policy Trigger,=1 – Inlet Temperature Policy Trigger value in [Celsius]=others – reserved [4:7] – Policy Type=1 - Power Control Policy

    ResponseByte 1 – completion code=00h – Success=81h – Invalid Domain Id=82h – unknown Policy Trigger Type=83h – unknown Policy TypeByte 2:4 = Intel manufacturers ID – 0x000157, LS byte firstByte 5 – Max Concurrent Settings – number of policies supported for the given policy trigger type and policy type.Bytes 6:7 – Max Power/Thermal value to be settable as trigger (in units defined in query) or max Power Limit to be maintained if Policy Trigger Type is equal to 0Bytes 8:9 – Min Power/Thermal value to be settable as trigger (in units defined in query) or min Power Limit to be maintained if Policy Trigger Type is equal to 0Bytes 10:13 – Min Correction Time settable in milli-seconds (ms)Bytes 14:17 – Max Correction Time settable in milli-seconds (ms)Bytes 18-19 – Min Statistics Reporting Period in secondsBytes 20-21 – Max Statistics Reporting Period in secondsByte 22 – Domain limiting scope[0:6] Limiting type=0 – platform power limiting=1 – CPU power limiting=2..127 - reserved for future use[7] – Limiting based on=0 – Wall input power. PSU input power =1 – DC power – PSU output power or bladed system

    Net Function = 2Eh, LUN = 00b

    Code Command Request, Response Data Description

  • Requirements on Platform Components

    34 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    CAh Get Node Manager Version

    RequestByte 1:3 = Intel manufacturers ID – 0x000157, LS byte first

    Get Node Manager and firmware version numbers.Major Firmware revision and Minor Firmware revision unambiguously identify firmware release. For every release, at least one of these numbers changes.

    ResponseByte 1 – completion code=00h – Success Byte 2:4 = Intel manufacturers ID – 0x000157, LS byte firstByte 5 – Node Manager version=01h – supported Node Manager 1.0 =02h – supported Node Manager 1.5=03h..FFh – reserved for future useNote: For support of each trigger type of the platform and/or number of supported policies per trigger type utilize the “Get Node Manager Capabilities” commandByte 6 – IPMI interface version=01h Node Manager IPMI version 1.0. Byte 7 – Patch version (binary encoded).Note: Change on this byte does not impact IPMI interface (Byte 6) nor Node Manager version (Byte 5). Should be set to 0x00 if patch version is not used by the firmware.Byte 8 – Major Firmware revision (binary encoded) – identifies current build of the code –and should contain the same value as the ‘Get Device Id’ byte 4 Major firmware revision. Note: Change on this byte does not impact IPMI interface (Byte 6) nor Node Manager version (Byte 5).Byte 9 – Minor Firmware revision (BCD encoded) – identifies current build of the code and should contain the same value as the ‘Get Device Id’ byte 5 Minor firmware revision. Note: Change on this byte does not impact IPMI interface (Byte 6) nor Node Manager version (Byte 5).

    CBh Set Node Manager Power DrawRange

    RequestByte 1:3 = Intel manufacturers ID – 0x000157, LS byte firstByte 4 – Domain Id[0:3] = Domain Id (Identifies the scope of processors. Currently, FW supports only one domain, Domain 0)[4:7] = Reserved. Write as 0000b.Byte 5:6 – Minimum Power Draw in [Watts]. If set to 0 the minimum power draw value will be invalidated and no validation of policy parameters against minimum power consumption will be performed. Byte 7:8 – MaximumPowerDraw in [Watts]. If set to 0 the maximum power draw value will be invalidated and no validation of policy parameters against maximum power consumption will be performed.

    Set the Min/Max power consumption rangesThis information is preserved in the persistent storage.After receiving the request NM validates whether there are any policies with limit not fitting into the new power consumption range. If NM detects such policies, it sends NM Health Event with Policy Misconfiguration flag set. Additionally Node Manager disables all the policies with power limit below the Minimum Power Draw. The policies are disabled permanently – NM does not enable them until it receives a Set Node Manager Policy IPMI request.The same action is taken, when Node Manager receives a new power draw range from BIOS at POST.

    ResponseByte 1 – completion code=00h – Success (Remaining standard completion codes are shown in completion code section.)=81h – Invalid Domain IdByte 2:4 = Intel manufacturers ID – 0x000157, LS byte first

    Net Function = 2Eh, LUN = 00b

    Code Command Request, Response Data Description

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 35

    Requirements on Platform Components

    CEh Set Node Manager Alert Destination

    RequestByte 1:3 = Intel manufacturers ID – 0x000157, LS byte firstByte 4 – Channel number[0:3] – BMC channel number over which to send the alert from BMC to management console. Alerts can be sent to only one console.[4:6] -reserved[7] – destination information operation0 – register alert receiver 1 – unregister alert receiver. Use this bit to invalidate the current destination configuration. Alerts will be blocked.

    Byte 5 – Destination InformationFor channel medium = IPMB[0] – reserved [1:7] - 7-bit I2C Slave Address For channel medium = 802.3 LANDestination Selector/ Operation[0:3] - destination selector. Selects which alert destination should go to.0h = use volatile destination info. 1h-Fh = non-volatile destination.Destination Selector definition is the same as in the « Set/Get LAN Configuration Parameters » command.[4:7] – reserved

    Byte 6 - Alert String SelectorSelects which Alert String, if any, to use with the alert.[0:6] - string selector.0000_0000b = use volatile Alert String.01h-7Fh = non-volatile string selector.Alert String Selector definition is the same as in the « Set/Get PEF Configuration Parameters » command.[7] -0b = don’t send an Alert String1b = send Alert String identified by following string selector.

    Provide alert destination information for Intel® Intelligent Power Node Manager to send direct alerts that bypass the BMC SEL.

    Destination Selector/Operation and Alert String Selector fields correspond to the associated LAN configuration parameters applicable to the BMC channel number over which to send the alert

    Note that the NM will determine the channel medium type in order to resolve the destination. If NM is implemented in an entity separate from the BMC, then this is done by querying the BMC using the Get Channel Info IPMI command.

    ResponseByte 1 – completion code=00h – Success Byte 2:4 = Intel manufacturers ID – 0x000157, LS byte first

    Net Function = 2Eh, LUN = 00b

    Code Command Request, Response Data Description

  • Requirements on Platform Components

    36 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    CFh Get Node Manager Alert Destination

    RequestByte 1:3 = Intel manufacturers ID – 0x000157, LS byte first

    Provides alert destination information that is used to send alerts from Intel® Intelligent Power Node Manager.

    ResponseByte 1 – completion code=00h – Success Byte 2:4 = Intel manufacturers ID – 0x000157, LS byte firstByte 5 – Channel number[0:3] – BMC channel number over which alert from BMC to management console will be sent[4:6] -–reserved[7] –Destination information operation0 – configuration valid. Alert receiver registered.1- configuration invalid. Alert receiver not registered. Alerts are blocked.

    Byte 6 - Destination Selector/ Operation[0:3] -destination selector. Selects which alert destination should go to.0h = use volatile destination info. 1h-Fh = non-volatile destination.

    [4:7] - reserved

    Byte 7 - Alert String SelectorSelects which Alert String, if any, to use with the alert.[0:6] - string selector.0000_0000b = use volatile Alert String. 01h-7Fh = non-volatile string selector.[7] -0b = don’t send an Alert String1b = send Alert String identified by following string selector.

    Net Function = 2Eh, LUN = 00b

    Code Command Request, Response Data Description

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 37

    Requirements on Platform Components

    2.2 IPMI SensorsThe following IPMI sensors are supported by Intel® Intelligent Power Node Manager. The values are encoded as described by the special encoding rules described in the Appendix.

    2.3 IPMI EventsThe following are Intel® Intelligent Power Node Manager-specific events that are exposed via IPMI.

    Table 2-4. Intel® Intelligent Power Node Manager IPMI Sensors

    Description Sensor NumberReading Availability Notes

    Node manager exception event sensor

    See SDR Section 2.6

    Event-only OEM Event only sensor used to send events when Intel® Intelligent Power Node Manager detects that power budget could not be maintained. “Command illegal for specified sensor or record type (CDh)” error code is returned in response to the following commands: Get Sensor Reading, Set/Get Sensor Thresholds, Re-Arm Sensor Events, Set/Get Sensor Event Enable.

    Node Manager Health event sensor

    See SDR Section 2.6

    Event-only OEM Event only sensor used to send events about integrity of Intel® Intelligent Power Node Manager policy or necessary sensor readings.“Command illegal for specified sensor or record type (CDh)” error code is returned in response to the following commands: Get Sensor Reading, Set/Get Sensor Thresholds, Re-Arm Sensor Events, Set/Get Sensor Event Enable.

    Node Manager Operational Capabilities sensor

    See SDR Section 2.6

    Read, Event OEM sensor, whose value will indicate the operational capabilities of the sensor. Whenever the sensor value changes, an immediate alert is also sent. Please see the event description for the description of the values of the sensor.

    Node Manager Alert Threshold Exceeded sensor

    See SDR Section 2.6

    Event-only OEM Event only sensor used to send events when Intel® Intelligent Power Node Manager detects that a specified alert threshold for one of the policies is exceeded. “Command illegal for specified sensor or record type (CDh)” error code is returned in response to the following commands: Get Sensor Reading, Set/Get Sensor Thresholds, Re-Arm Sensor Events, Set/Get Sensor Event Enable.

    Table 2-5. Intel® Intelligent Power Node Manager IPMI Events

    Event Sensor Type Event Dir Event Type ImmediateAlert

    Node Manager Exception Event DCh – OEM 0 – assertion 72h – OEM No

    Node Manager Health Event DCh – OEM 0 – assertion 73h – OEM Yes

    Node Manager Operational Capabilities Change Event

    DCh – OEM 0 – assertion1 – deassertion

    74h – OEM Yes

    Node Manager Alert Threshold Exceeded DCh – OEM 0 – assertion

    1 – deassertion

    72h – OEM Yes

  • Requirements on Platform Components

    38 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    Net Function = S/E (0x4), LUN = 00b

    Code Command Request, Response Data Description

    02h Platform Event MessageNode Manager Exception Event

    Request Byte 1 - EvMRev=04h (IPMI2.0 format)Byte 2 – Sensor Type=DCh (OEM)Byte 3 – Sensor Number= See SDR Section 2.6 – Node Manager event SensorByte 4 – Event Dir | Event Type[0:6] – Event Type=72h (OEM) [7] – Event Dir=0 Assertion Event=1 Deassertion Event

    Byte 5 – Event Data 1[0:1] – Return as 00b[2] – Reserved[3] = Node Manager Policy event0 – Reserved 1 – Policy Correction Time Exceeded – policy did not meet the contract for the defined policy. The policy will continue to limit the power or shutdown the platform based on the defined policy action.[4:5]=10b – OEM code in byte 3[6:7]=10b – OEM code in byte 2Byte 6 – Event Data 2[0:3] – Domain Id (Currently, supports only one domain, Domain 0)[4:7] – ReservedByte 7 – Event Data 3=.

    Event will be sent each time when maintained policy power limit is exceeded over Correction Time Limit. First occurrence of not acknowledged event will be retransmitted no faster

    than every 300 milliseconds

    ResponseByte 1 completion code=00h – Successothers – Error

  • Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI 39

    Requirements on Platform Components

    02h Alert Immediate MessageNode Manager Health Event

    Request Byte 1 - EvMRev=04h (IPMI2.0 format)Byte 2 – Sensor Type=DCh (OEM)Byte 3 – Sensor Number= See SDR Section 2.6 – Node Manager Health sensorByte 4 – Event Dir | Event Type[0:6] – Event Type= 73h (OEM)[7] – Event Dir=0 Assertion Event

    Byte 5 – Event Data 1[0:3] – Health Event Type=02h – Sensor Node Manager[4:5]=10b – OEM code in byte 3[6:7]=10b – OEM code in byte 2

    Byte 6 – Event Data 2[0:3] – Domain Id (Currently, supports only one domain, Domain 0)[4:7] – Error type=0-9 - Reserved=10 – Policy Misconfiguration=11 – Power Sensor Reading Failure=12 – Inlet Temperature Reading Failure=13 – Host Communication error=14 – Real-time clock synchronization failure=15 – Platform shutdown initiated by NM policy due to execution of action defined by Policy Exception Action see “Set NM Policy” command Byte 7 bit [1]=16 – Reserved Byte 7 – Event Data 3if error indication = 10 or 15 if error indication = 11 if error indication = 12 Otherwise set to 0.

    This message provides a run-time error indication about Intel® Intelligent Power Node Manager’s health. Types of service that can send an error are defined as follows:Misconfigured policy Error reading power data Error reading inlet temp.Note: Misconfigured policy can happen if the max/min power consumption of the platform exceeds the values in policy due to hardware reconfiguration.First occurrence of not acknowledged event will be retransmitted no faster than every 300 milliseconds.Real-time clock synchronization failure alert is sent when NM is enabled and capable of limiting power, but within 10 minutes the firmware can not obtain valid calendar time from the host side, so NM can not handle suspend periods.

    ResponseByte 1 completion code=00h – Successothers – Error

    Net Function = S/E (0x4), LUN = 00b

    Code Command Request, Response Data Description

  • Requirements on Platform Components

    40 Intel® Intelligent Power Node Manager 1.5 External Interface Specification Using IPMI

    Net Function = S/E (0x4),