vmware virtualized sap hana with dell emc storage · pdf filerequires an sap registration id.)...

VMWARE VIRTUALIZED SAP HANA WITH DELL EMC STORAGE

Storage configuration and connectivity

Scale-up and scale-out deployments

June 2017

ABSTRACT

This solution guide describes best practices for deploying an SAP HANA system on

Dell EMC storage arrays in VMware vSphere virtualized environments.

H14721.2

This document is not intended for audiences in China, Hong Kong, Taiwan,

and Macao.

SOLUTION GUIDE

Copyright

2 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity Solution Guide

The information in this publication is provided as is. Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any software described in this publication requires an applicable software license.

Copyright © 2017 Dell Inc. or its subsidiaries. Intel, the Intel logo, the Intel Inside logo, and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. All Rights Reserved. Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be the property of their respective owners. Published in the USA 06/17 Solution Guide H14721.2

Dell Inc. believes the information in this document is accurate as of its publication date. The information is subject to change without notice.

Contents

3 VMware Virtualized SAP HANA with Dell EMC Storage Storage configuration and connectivity

Solution Guide

Contents

Executive summary ........................................................................................................................ 4

Deploying VMware vSphere virtualized SAP HANA on Dell EMC storage ................................. 6

SAP HANA storage devices in virtualized environments .......................................................... 11

Connecting an ESXi host to Dell EMC storage arrays ............................................................... 17

Creating data and log devices on vSphere datastores .............................................................. 19

Setting up an SAP HANA virtual machine in vSphere ............................................................... 21

Conclusion ..................................................................................................................................... 23

References ..................................................................................................................................... 24

Executive summary


Executive summary

Since the initial release of the SAP HANA in-memory, real-time platform as a physical

appliance, updates to the platform have provided more flexible deployment and scalability

for customers while reducing the total cost of ownership (TCO). The enhancements

include:

SAP HANA scale-outBreaks the RAM barrier of a single server by functioning as a

multinode cluster.

SAP HANA Tailored Datacenter Integration (TDI)Broadens the appliance model

toward shared enterprise storage arrays.

Virtualized SAP HANAEnables the installation of SAP HANA single-node (scale-

up) and multinode (scale-out) clusters in VMware virtualized environments. SAP

Note 1788665SAP HANA Support for virtualized/partitioned (multitenant)

environments provides more information. (Access to the note requires an SAP

registration ID.)

While these updates make the SAP HANA deployment easier and help customers with

the integration into their existing data center, businesses must still meet strict SAP HANA

requirements relating to their compute, network, and storage environments.

This VMware Virtualized SAP HANA with Dell EMC Storage solution offers SAP

customers a data center platform that provides agility, high availability, cost savings, and

easy provisioning. The solution enables SAP customers to provision SAP HANA

instances more quickly in virtual machines.

Using the SAP HANA platform with the VMware vSphere virtualization infrastructure

provides the best environment for achieving a unique, cost-effective solution. This solution

is enabled by VMware vSphere vMotion, VMware vSphere Distributed Resource

Scheduler (DRS), and VMware vSphere High Availability (HA), ensuring the best

operational performance and availability.

VMware provides best practices for deploying SAP HANA scale-up and scale-out systems

under vSphere. In vSphere virtualized environments, this solution guide can be used as a

supplement to the VMware documents and other Dell EMC documents that describe

storage configuration best practices for SAP HANA in physical TDI environments. See

References on page 24 for the titles of these documents.

VMware virtualized SAP HANA scale-up and scale-out systems can be deployed in TDI

environments on certified Dell EMC storage arrays. When running an SAP HANA

production system, the environment must meet SAP hardware and performance

requirements. You can use the SAP HANA hardware configuration check tool (hwcct) to

validate that the environment is correctly configured and that the storage performance

meets the SAP key performance indicators (KPIs). For more information, see SAP Note

1943937Hardware Configuration Check Tool - Central Note (Access to the note

requires an SAP registration ID.)

Note: The SAP performance requirements apply to production systems only.

Business case

Solution

overview

https://launchpad.support.sap.com/#/notes/1943937

Executive summary


Solution Guide

This guide describes best practices for implementing SAP HANA in vSphere virtualized

environments when using the following Dell EMC storage systems in TDI deployments:

VNXTM

series unified and block storage arrays

VMAXTM

, VMAX3™, and VMAX All FlashTM

XtremIOTM

VPLEXTM

ScaleIOTM

UnityTM

SC SeriesTM

This guide addresses vSphere-specific storage connectivity and configuration procedures

to follow when connecting vSphere virtualized SAP HANA systems on ESXi hosts with

certified Dell EMC enterprise storage arrays. This document is a supplement to the Dell

EMC best practice guides for SAP HANA TDI deployments that are listed in References.

Use it with the following VMware guide for virtualized SAP HANA scale-up and scale-out

deployments: Architecture Guidelines and Best Practices for Deployments of SAP HANA

on VMware vSphere

This guide is intended for administrators who are responsible for implementing,

maintaining, and protecting robust and virtualized SAP HANA in-memory databases and

storage systems on vSphere. It assumes that you have some familiarity with SAP HANA

in-memory databases, Dell EMC storage hardware and software, and vSphere virtual

infrastructures.

Table 1 provides explanations for some abbreviations used in this solution guide.

Table 1. Terminology

Term Explanation

RDM Raw Device Mapping

RHEL Red Hat Enterprise Linux

SLES SUSE Linux Enterprise Service

VMDK Virtual Machine Disk

VMFS Virtual Machine File System

Dell EMC and the authors of this document welcome your feedback on the solution and

the solution documentation. Contact [email protected] with your

comments.

Authors: Werner Katzenberger, Aighne Kearney

Document

purpose

Audience

Terminology

We value your

feedback

http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/whitepaper/sap_hana_on_vmware_vsphere_best_practices_guide-white-paper.pdf


mailto:[email protected]?subject=Feedback:%20VMware%20Virtualized%20HANA%20with%20Dell%20EMC%20Storage%20Solution%20Guide%20H14721.2

Deploying VMware vSphere virtualized SAP HANA on Dell EMC storage



Following SAP’s enhancement of the SAP HANA deployment options with the TDI model,

Dell EMC certified the VMAX, VMAX3, VMAX All Flash, VNX, XtremIO, Unity, VPLEX,

and SC Series block storage platforms for production use in TDI deployments.

Tests with Dell EMC SAP HANA-certified block storage systems confirmed that the arrays

can be used in VMware virtualized environments with only a minor impact on performance

when you follow the guidelines in this document.

As in physical environments, you can install an SAP HANA system in virtualized

environments as a scale-up single-node system or as a scale-out multinode system. The

number of SAP HANA virtual machines that you can install on a single physical server

depends on several factors, including the number of CPU sockets, the server RAM, the

SAP HANA installation use case (production or non-production), and the maximum

scalability supported by SAP and VMware.

Consult the corresponding VMware best practices and recommendations when you create

an SAP HANA virtual machine and need to size and configure memory, CPU,

hyperthreading, and non-uniform memory access (NUMA) nodes for the machine.

Scale-up (single-node) systems

Scale-up systems are primarily used for transactional SAP applications such as the SAP

Business Suite. The SAP HANA database is stored in the RAM of a single virtual

machine, while the SAP HANA persistence is stored on a data and log file system

belonging to this virtual machine.

Figure 1 shows the virtual disk devices used in SAP HANA virtual machines. The virtual

disk that holds the operating system is created during the initial machine configuration and

can reside on any available vSphere datastore. Installing the SAP HANA software

requires a /hana/shared mount point. In scale-up installations, this mount point is

private to the virtual machine and can be a directory under the operating system root

device (if the size meets the requirements), or a separate virtual disk mounted under

/hana/shared. This virtual disk does not have specific performance requirements and

can reside on any multipurpose datastore.

Overview

SAP HANA

deployment

options



Solution Guide

Figure 1. Virtual disks in an SAP HANA scale-up scenario

Scale-out (multinode) systems

Scale-out systems are used primarily for analytic applications such as SAP Business

Warehouse. The SAP HANA database is distributed across multiple virtual machines,

either on a single physical server or across a vSphere cluster consisting of multiple

physical servers. The system uses a “shared-nothing cluster architecture,” in which every

node in a scale-out deployment stores its database persistence on its own data and log

file systems. In a scale-out deployment, the /hana/shared file system must be shared

across all SAP HANA nodes that are part of the scale-out cluster. See SAP HANA shared

file system for configuration guidelines.

High Availability in a vSphere cluster

In a vSphere cluster, you must enable the vSphere HA feature. HA handles the restart of virtual machines on another host that has enough free resources if one of the hosts in the cluster fails. To ensure that SAP HANA starts automatically with the virtual machine, enable the SAP HANA service autostart feature, either at the time of the SAP HANA

installation or by setting the autostart option to 1 in the following file:

/hana/shared/<SID>/profile/<SID>_HDB<InstNo>_<hostname>

Figure 2 shows the minimum required vSphere HA settings.

High Availability

considerations



Figure 2. vSphere HA Settings

Using vSphere host groups, VM groups, and affinity rules, the administrator can control which hosts in the vSphere cluster are allowed to run SAP HANA virtual machines.

HA for SAP HANA virtual machines

The SAP HANA service auto-restart watchdog function monitors the SAP HANA

application and the associated services within a virtual machine. The watchdog function

automatically detects a failure and restarts the corresponding SAP HANA process

(nameserver, indexserver, and so on).

The VMware HA virtual machine monitoring feature (Guest not heartbeating) handles

operating system (OS) failures. The monitoring feature restarts the guest OS of the virtual

machine as well as SAP HANA on the same host. (The SAP HANA autostart option

must be enabled.) Enable the monitoring feature when you enable vSphere HA, as shown

in Figure 2. Dell EMC recommends setting Heartbeat monitoring sensitivity to High, as

shown in Figure 3.



Solution Guide

Figure 3. Enabling VMware HA VM monitoring

To enable heartbeat monitoring, install and run the VMware tools in the virtual machine.

These tools are installed either as part of the OS (open-vm-tools) or by using the

vSphere Web client and the virtual machine’s context menu (select Guest OS > Install

VMware Tools).

In addition to heartbeat polling, VM Monitoring monitors the virtual machine’s I/O activity.

When heartbeats are not received and no disk or network activity has occurred over the

last 120 seconds, the virtual machine is reset by default. The administrator can change

the advanced setting das.iostatsInterval to modify this 120-second interval. Dell

EMC recommends aligning das.iostatsInterval with the failure interval you

selected in the vSphere HA VM Monitoring section in the vSphere Web client.

SAP HANA high availability in scale-out installations

In an SAP HANA scale-out installation on physical servers, you can deploy a standby

server to enable the SAP HANA host auto-failover functionality. If one of the active SAP

HANA servers (workers) fails, the SAP HANA nameserver triggers a failover to the

standby host.

In VMware virtualized environments, an SAP HANA standby host is not required and does

not work with the default SAP HANA storage connector. Instead, the VMware HA feature

restarts the SAP HANA virtual machines on another host in the cluster when a physical

host fails, or on the same host if just the virtual machine OS fails.

SAP HANA split-brain condition and host isolation

In physical SAP HANA scale-out installations with shared storage, the first installed node

becomes the master nameserver, Master 1, and the second and the third installed nodes

become master candidates Master 2 and Master 3. SAP HANA experiences a split-brain

situation if multiple hosts try to become the master nameserver or indexserver and to

access the same data (persistence) from disk. Such a situation might occur if the host of

the master nameserver becomes isolated from the network but can still access the shared

disk storage and a master candidate tries to take over the master role. Physical SAP



HANA installations use techniques such as SCSI-3 persistent reservation to prevent this

situation.

The standby master nameserver candidates are not required in VMware virtualized

environments. Because VMware HA restarts the existing master nameserver quickly

enough, promoting a new node to the master role is not necessary. To prevent SAP

HANA from using Master 2 and Master 3 candidates in scale-out installations, set the

following entries in the global.ini file:

[communication]

listeninterface = .global

[persistence]

basepath_shared = no

When installing SAP HANA using hdbinst, add these parameters to the

/usr/sap/<SID>/SYS/global/hdb/custom/config/global.ini file after the first

node has been installed.

When using hdblcm for a SAP HANA scale-out installation, create a default

global.ini file with the above entries in a temporary directory. The

--custom_cfg installation parameter must point to this temporary directory.

In existing SAP HANA installations that already have multiple master candidates (Master

2 and Master 3), disable the failover mechanism by removing the nameserver roles

Master 2 and Master 3:

1. Click SAP HANA Studio > Administration-Perspective > Landscape > Hosts.

2. Select Configure Hosts for Failover Situation and change the master role to

Slave.

If a network failure occurs whereby a physical ESXi host becomes disconnected from the

network but all virtual machines on the host are still running, the VMware HA Host

Isolation rule and the associated response are applied. The default response leaves all

virtual machines powered on, as shown in Figure 2. Because a virtualized SAP HANA

scale-out cluster does not have a standby node to take over the function (master

nameserver) of the unresponsive node on the isolated host, a split-brain situation cannot

occur. The default response allows the administrator to investigate and fix the network

issue, or to power off the isolated host, which then causes VMware HA to restart the

virtual machines on another host.

It is also possible to set the Host Isolation response to Power off and restart VMs. In this

case, the host is powered off and the virtual machines are restarted automatically on

another host that still has network connectivity.

With only one SAP HANA master nameserver candidate and the VMware HA Host

Isolation rules, a master nameserver can exist only once and therefore a split-brain

condition cannot occur.

SAP HANA storage devices in virtualized environments


Solution Guide


SAP HANA installations on VMware vSphere require the following storage devices:

OS disk (root device) ‒ approximately 10 GB

The disk is created when the virtual machine is deployed. The virtual disk file

(VMDK) can reside on any datastore that meets the installation requirements, that

is, has vMotion support.

SAP home directory: /usr/sap ‒ approximately 50 GB

The /usr/sap directory can be a mount point under the OS root directory. Adjust

the size of the root device correspondingly. Alternatively, you can use and mount a

separate virtual disk under /usr/sap/<SID>. Dell EMC recommends this method

for storage-based cloning and when using SAP instance cloning tools such as SAP

Landscape Management (LaMa).

SAP HANA shared file system: /hana/shared

The /hana/shared file system stores the SAP HANA installation binaries,

configuration files, and trace files. The size of the file system depends on the size of

the SAP HANA installation (database size and number of nodes). See the SAP

HANA white paper SAP HANA Storage Requirements for guidelines on capacity

sizing. The SAP HANA shared file system section of this guide provides

configuration details.

SAP HANA database persistence (data and log)

Each active SAP HANA node (scale-up node or worker node in scale-out

deployments) requires two devices to persist the in-memory database to disk and to

write the transaction log. The SAP performance requirements for the storage

devices apply only to these two file systems, so it is important to consider storage-

configuration best practices when you set up an SAP HANA production

environment. The size of the data and log devices depends on the actual database

size. See the SAP HANA Storage Requirements white paper for more information.

SAP HANA scale-out scenarios require a shared file system that is mounted on every

SAP HANA node within the cluster. The file system stores the installation binaries,

configuration files, and trace files. In physical environments, this file system is generally

provided as a network file system (NFS) share by storage systems with network-attached

storage (NAS) capabilities, such as VNX series unified, Unity, SC Series storage, and

VMAX with eNAS.

If a NAS array is not available, vSphere with native Linux functionality provides a viable

alternative. A Linux virtual machine (non-SAP HANA node) running an NFS server

process provides the NFS share. This process exports a file system that is mounted on all

the SAP HANA cluster nodes. Reliability is achieved using vSphere Fault Tolerance.

Overview

SAP HANA

shared file

system

http://go.sap.com/documents/2015/03/74cdb554-5a7c-0010-82c7-eda71af511fa.html




Figure 4. A /hana/shared file system using Linux NFS and vSphere fault tolerance

Setting up the NFS server

To set up the NFS server:

1. Create a Linux virtual machine (SLES or RHEL) and install the NFS server

package. Refer to the OS documentation for instructions. The Linux virtual

machine is not required to meet the high CPU and memory requirements of an

SAP HANA node.

2. Add a new VMDK to the virtual machine. The VMDK can reside on any vSphere

datastore. The size of the disk must be the required size of the shared file system

for the SAP HANA cluster.

3. Format the disk using the XFS file system.

4. Create a mount point in the following format: /hana_<sid>. Then mount the disk

under the new mount point, replacing <sid> with the system ID of the SAP

HANA cluster that is using the export as the shared file system.

5. Open the /etc/exports file. For each SAP HANA node (NFS client) in the

cluster, add the following line:

/hana_<sid> <fully qualified file server client host

name>(rw,sync,no_root_squash,no_subtree_check)

6. Restart the NFS server process to activate the changes

Note: The virtual machine running the NFS server can easily provide multiple SAP HANA

share exports for different SAP HANA installations.

Setting up the NFS client on the SAP HANA nodes

To set up the NFS client:

1. Ensure that the Linux NFS client is installed and running on each SAP HANA node.

2. Open the /etc/fstab file in each SAP HANA node and add the following line:

<fully qualified file server host name>:/hana_<sid>

/hana/shared nfs hard,intr,nolock,proto=tcp 0 0



Solution Guide

3. Create the local mount point on each SAP HANA node using the following command:

$ mkdir -p /hana/shared

4. Mount the shared file system on each node using the following command:

$ mount -a

Using fault tolerance for the NFS server virtual machine in vSphere

Because the SAP HANA nodes depend on the shared file system, Dell EMC recommends

that you turn on the fault-tolerance function for the NFS server virtual machine. To do this,

go to the vSphere Web client, highlight the NFS Server virtual machine, and select and

enable Fault Tolerance in the context menu.

Note: The fault-tolerance feature may require additional licenses in your vSphere environment and

it presents some limitations on the number of virtual CPUs per protected virtual machine.

vSphere fault tolerance provides continuous availability for the SAP HANA share during

an NFS server failure. vSphere creates a live-shadow virtual-machine instance that is

always up to date with the primary virtual machine. If a hardware outage occurs, vSphere

automatically triggers failover, ensuring zero downtime for the SAP HANA share and

preventing data loss. After failover, vSphere creates a secondary virtual machine to

deliver continuous protection for the SAP HANA share.

SAP HANA persists the in-memory database to the /hana/data file system at regular

intervals and writes the transaction log to the /hana/log file system. These two file

systems are the SAP HANA database persistence. The storage performance

requirements for SAP HANA TDI certification apply to the SAP HANA persistence for

production systems only. Dedicate the underlying storage device (RDM or a VMware

VMFS datastore) to the persistence and then configure it using the best practices for SAP

HANA on that storage array. For more information, see the relevant document in the

following list:

Storage Configuration Best Practices for SAP HANA Tailored Datacenter

Integration on EMC VMAX and VMAX3 Storage Systems


Integration on EMC VNX Series Unified Storage Systems


Integration on EMC XtremIO Storage

EMC VPLEX For SAP HANA Solution Guide

Storage Configuration Best Practices for SAP HANA TDI on EMC Unity Storage

Systems Solution Guide

SAP HANA production on vSphere

In production environments, a virtual SAP HANA node requires the same storage

performance and bandwidth as a physical node. The configuration of the LUNs and the

underlying pools or RAID groups must follow the same rules. Because the additional

virtualization layer affects the bandwidth and latency, the scalability of a particular Dell

SAP HANA

database

persistence

http://www.emc.com/collateral/white-paper/h12342-storage-config-best-practices-sap-hana-tdi-vmax-vmax3-wp.pdf


http://www.emc.com/collateral/white-paper/h13047-storage-config-best-practices-sap-hana-tdi-vnx-wp.pdf


http://www.emc.com/collateral/technical-documentation/h13984-storage-config-best-practices-sap-hana-tdi-xtremio-cg.pdf


http://www.emc.com/collateral/white-papers/h14283-vplex-sap-hana-sg.pdf

http://www.emc.com/collateral/technical-documentation/h15220-storage-config-best-practices-sap-hana-tdi-unity-sg.pdf

http://www.emc.com/collateral/technical-documentation/h15220-storage-config-best-practices-sap-hana-tdi-unity-sg.pdf



EMC array for virtualized SAP HANA nodes might differ from, and could be lower than,

the number of SAP HANA nodes supported in a physical environment.

For nonproduction virtualized SAP HANA environments, the storage performance does

not have to meet the SAP KPIs. This means you have more flexibility when you configure

the virtual environment—RAID protection, disk types, pools, RAID groups, and so on. Dell

EMC recommends monitoring the storage performance and making adjustments if the

performance does not meet your requirements.

Datastores and raw device mappings

When creating LUNs for virtualized SAP HANA deployments in production environments,

follow the storage configuration best practices for physical SAP HANA environments

corresponding to the SAP TDI-certified Dell EMC storage array. See Dell EMC

documentation for links to these documents.

When Dell EMC block storage devices (LUNs) are connected to vSphere ESXi hosts, the

SAP HANA virtual machines can use them for the database persistence, either as

datastores formatted using a VMFS cluster or as RDMs. VMFS datastores are preferred

because they are easier to manage and provide more data growth flexibility.

This guide does not address the use of VMware Virtual Volumes (vvols).

Using VMFS datastores

Creating a VMFS datastore on a block storage LUN is the preferred configuration option

for SAP HANA virtual machines. Creating dedicated and isolated datastores for SAP

HANA data and log file systems is a VMware best practice, which means the datastores

must be created on different LUNs in the storage array. Create all data devices (VMDKs)

of a SAP HANA system on one datastore, and all log devices on a second datastore.

When running multiple SAP HANA databases in a vSphere cluster, you can achieve

further workload isolation with dedicated data and log datastores for each SAP HANA

database system ID SID.

Each SAP HANA worker virtual machine has a data and a log file system residing on

VMDK files created on the corresponding VMFS datastore

Figure 5 shows an example of an SAP HANA three-node scale-out deployment on a

vSphere cluster using VMFS datastores.



Solution Guide

Figure 5. Deploying SAP HANA persistence on VMFS datastores

Using raw device mappings

RDMs are an alternative option for the SAP HANA database persistence. With RDMs,

every data and logfile system resides on a dedicated LUN in the storage array.

RDMs enable you to configure the LUNs for virtualized SAP HANA nodes the same way

as for physical environments. They also enable you to use the LUNs for physical SAP

HANA installations when the nodes are migrated to and from a virtualized environment.

For larger SAP HANA deployments, the high number of LUNs might become difficult to

manage, therefore, consider VMFS datastores instead.

Figure 6 shows an example of an SAP HANA 3+1 scale-out deployment on RDMs.



Figure 6. SAP HANA persistence deployed on RDMs

Sizing storage LUNs for SAP HANA persistence

This section provides guidelines on how to correctly size the LUNs used as RDMs or the

storage LUNs for the vSphere VMFS datastores. The SAP document SAP HANA Storage

Requirements describes capacity sizing for SAP HANA devices. Because the

requirements have been updated several times, use the most recent version of the guide

when sizing a new SAP HANA installation.

The examples in this guide use the following capacity sizing:

SAP HANA data file system: 1.2x net data of the SAP HANA node

SAP HANA log file system: 0.5x RAM of the SAP HANA node (maximum 512 GB)

Sizing storage LUNs for VMFS datastores

The LUNs for the VMFS datastores contain the SAP HANA persistence. When sizing

them, provide the following information:

Number of virtualized SAP HANA nodes: scale-up and scale-out

RAM size of the SAP HANA nodes

Figure 5 shows a 3-node SAP HANA scale-out cluster. Assuming each node has 128 GB

RAM and holds approximately 50 GB of data, the capacity calculation is:

Data: 50 GB x 1.2 x 3 nodes = 180 GB

Log: 128 GB x 0.5 x 3 nodes = 194 GB



Connecting an ESXi host to Dell EMC storage arrays


Solution Guide

All VMFS datastores must retain a level of free space to ensure that virtual machines

function correctly. Dell EMC recommends adding 25 percent free space to the previous

calculation. The size of the LUNs should be:

Data: 225 GB LUN size

Log: 242 GB LUN size

These are the minimum capacity requirements for the SAP HANA persistence. They do

not take account of future growth. With virtualized SAP HANA, you can increase the RAM

capacity of the SAP HANA nodes, requiring larger file systems for the database

persistence and therefore more capacity in the VMFS datastores. It is more efficient to

create larger LUNs at the beginning to prepare for future growth. It is also better to avoid

adding another LUN to a VMFS datastore because performance does not scale when

LUNs are concatenated.

Unlike physical environments, the underlying disk configuration of virtual environments

(RAID groups, thin pools, and so on) does not change. You must still adjust the size and

configuration of the LUNs to meet the capacity requirements for datastores.

Sizing storage LUNs for RDMs

When using RDMs for the SAP HANA persistence, you can size the storage LUNs in

accordance with the SAP minimum requirements described in SAP HANA Storage

Requirements.

For the example shown in Figure 6, we created the following LUNs:

Three data LUNs: 50 GB x 1.2 = 60 GB for each LUN

Three log LUNs: 128 GB x 0.5 = 64 GB for each LUN


To connect Dell EMC storage to VMware ESXi servers, follow the best practices

described in the EMC Host Connectivity Guide for VMware ESX Server. When running

virtualized SAP HANA systems on vSphere, consider the additional connectivity options

described in this section.

Host bus adapters

For redundancy reasons, use at least two fibre channel (FC) host bus adapters (HBAs)

connected to the storage array over two switches. If you deploy multiple virtualized SAP

HANA production nodes on a single ESXi host, the storage bandwidth and latency must

fulfill the SAP TDI KPIs for each SAP HANA system1. In TDI deployments, a single SAP

HANA production node requires 400 MB/s maximum bandwidth. With four SAP HANA

production nodes on a single physical ESXi host, the combined bandwidth of all HBAs on

the host must be at least 4 x 400 MB/s (1600 MB/s)—for example, a single 8 Gb/s FC

HBA provides approximately 750 MB/s bandwidth.

1 See SAP and VMWare documentation for information on how many SAP HANA production

systems can be deployed per physical host.

Introduction

ESXi

considerations

http://www.emc.com/collateral/TechnicalDocument/docu5265.pdf



ESXi HBA queue depth

Dell EMC recommends increasing the HBA queue depth on the ESXi host to 128:

1. Run the following command on the ESXi host:

# esxcli system module parameters set -p

lpfc_lun_queue_depth=128 -m lpfc

2. To display the HBA queue depth setting, use the following command:

# esxcli system module parameters list -m lpfc|grep

lpfc_lun_queue_depth

3. Restart the ESXi host to activate the changes.

vSphere Native Multipathing

If the ESXi host is using vSphere Native Multipathing (NMP), Dell EMC recommends

round-robin for the multipath setting for VMAX, VMAX3, VMAX All Flash, VNX, Unity, SC

Series, and XtremIO storage arrays. For best performance, set the multipath round-robin

policy for the I/O operation limit to 1. For the SC Series, the I/O operation limit should be

set to 3. The current VMware default value for this setting is 1000.

To change the default policy setting from fixed to round-robin and the host I/O operation

limit to 1:

1. Open the vSphere CLI (recommended) or the service console.

2. Run the following command:

For VMAX, VMAX3, and VMAX All Flash: # esxcli storage nmp satp set --default-psp=VMW_PSP_RR --

satp=VMW_SATP_SYMM

For VNX2 and Unity using the ALUA protocol: # esxcli storage nmp satp set --default-psp=VMW_PSP_RR --

satp=VMW_SATP_ALUA_CX

For XtremIO: # esxcli storage nmp satp rule add -c tpgs_off -e "XtremIO

Active/Active" –M XtremApp -P VMW_PSP_RR -O iops=1 -s

VMW_SATP_DEFAULT_AA –t vendor -V XtremIO

3. To set the host I/O operation limit to “1,” run the following commands:

List all devices connected to the ESXi host:

# esxcli storage nmp device list

Set the I/O operation limit to “1” for all Dell EMC storage devices used for the

SAP HANA persistence (either as datastores or as RDMs):

# esxcli storage nmp psp roundrobin deviceconfig set -I 1 -t

iops -d <naa.deviceid>

Verify the device settings:

# esxcli storage nmp psp roundrobin deviceconfig get -d

<naa.deviceid>

Creating data and log devices on vSphere datastores


Solution Guide

Dell EMC PowerPath/VE

Dell EMC PowerPathTM

/VE for VMware vSphere can be used as an alternative to, or in

combination with, vSphere NMP. PowerPath/VE is a multipathing extensions module for

vSphere, providing software that works with SAN storage to intelligently manage FC,

iSCSI, and FC over Ethernet (FCoE) I/O paths. PowerPath/VE is installed on the vSphere

host and scales to the maximum number of virtual machines on the host, thus improving

I/O performance. The virtual machines do not have PowerPath/VE installed and are not

aware that PowerPath/VE is managing I/O to storage. PowerPath/VE dynamically load-

balances I/O requests and automatically detects and recovers from path failures.

PowerPath/VE automatically assigns an optimized load-balancing policy for all

PowerPath-managed devices. Dell EMC recommends that you use the default optimized

load-balancing and failover policies that PowerPath/VE assigns to the devices based on

the underlying storage array type.

The number of front-end ports that are used for the ESXi hosts cluster depends on the

number of virtualized SAP HANA production nodes. Treat a virtualized SAP HANA node

like a physical SAP HANA node. For more information, see the storage configuration best

practices document for the corresponding Dell EMC storage array.

With ESXi hosts, use single-initiator zoning or single-initiator-single-target zoning. Single-

initiator-single-target zoning is preferred.

Note: Dell EMC does not support multi-initiator zones in a VMware ESX Server fabric

environment.

To perform zoning on the fabric, create zone sets that contain the initiator and the targets.


VMDKs on VMFS datastores are the preferred storage for the SAP HANA persistence on

SAP HANA virtual machines. After you have correctly sized and created the storage

LUNs, as described in Sizing storage LUNs for SAP HANA persistence, and formatted

VMFS datastores on these LUNs, you can create data and log devices for the SAP HANA

virtual machines on the datastores.

Each SAP HANA node uses its own data and log device and has access only to its pair of

devices, as Figure 1 shows. For performance reasons, VMware recommends provisioning

the SAP HANA data and log devices as “thick provision eager zeroed.” VMware also

recommends assigning dedicated SCSI controllers for each data and log device. See

vSphere SCSI controller for more information.

For information on creating data and log VMDKs, see Setting up an SAP HANA virtual

machine in vSphere.

Front-end ports

for storage

arrays

Fabric zoning

Introduction

Scale-up and

scale-out



VMware recommends using multiple vSphere SCSI controllers for each SAP HANA virtual

machine. Assign SCSI controller 0 to the operating system disk, SCSI controller 1 to the

SAP HANA data devices, and SCSI controller 2 to the SAP HANA log devices.

vSphere offers several different types of SCSI controller to connect disks to a virtual

machine:

LSI Logic Parallel

LSI Logic SAS

VMware Paravirtual

BusLogic Parallel (this does not work with Dell EMC storage)

For best performance, VMware recommends using one VMware Paravirtual SCSI

controller for all data devices and one for all log devices.

With the VMWare Paravirtual SCSI controller used for the SAP HANA persistence, set the

following kernel parameters for the virtual machine running SAP HANA:

elevator=noop

vmw_pvscsi.cmd_per_lun=254

vmw_pvscsi.ring_pages=32

For information on how to set these kernel parameters, see the following knowledge base

articles:

Red Hat document 5428 (applies also to SUSE)

Novell document 7009616 (applies also to SLES versions other than those

mentioned)

vSphere SCSI

controller

Prepare an SAP

HANA virtual

machine in Linux

https://access.redhat.com/solutions/5427

http://www.novell.com/support/kb/doc.php?id=7009616

Setting up an SAP HANA virtual machine in vSphere


Solution Guide


This section describes the configuration steps you must complete when creating an SAP

HANA virtual machine with data and log persistence on Dell EMC storage as one of the

following:

Single-node scale-up system

Multinode scale-out system without a standby host

When deploying a new SAP HANA virtual machine, ensure that the OS version is

supported by SAP and that you follow SAP and VMware best practices when configuring

the memory and CPU of the virtual machine.

Operating system virtual disk

The virtual disk on which the OS is installed can reside on any datastore that meets the

functional requirements, namely, support for vSphere vMotion. This device does not have

to meet the SAP HANA performance requirements.

vSphere compatibility mode

It is important to use the correct vSphere compatibility mode (virtual-machine version). We

tested and validated compatibility mode ESXi 5.5 and later versions (virtual-machine

version 10).

To configure the disk:

1. In the vSphere Web client, edit the settings in the SAP HANA virtual machine.

2. Add two additional VMware Paravirtual SCSI controllers to the virtual machine.

Set SCSI Bus Sharing to None.

The virtual machine should now have three SCSI controllers assigned:

SCSI controller 0Used for the system device

SCSI controller 1To be used for the SAP HANA data devices

SCSI controller 2To be used for the SAP HANA log devices

3. Add a new SAP HANA data or log disk.

The SAP HANA persistence can be a vSphere VMDK residing on a VMFS

datastore, or an RDM pointing directly to the underlying storage LUN.

VMDK—Add a new hard disk to the virtual machine on the desired datastore

using the thick provision eager zeroed option. Assign an SAP HANA data

device to SCSI controller 1 and an SAP HANA log device to SCSI controller 2,

as shown in Figure 7 and Figure 8 respectively.

Overview

Deploying an SAP

HANA virtual

machine

Assigning the

persistence to

the SAP HANA

virtual machine



Figure 7. Creating a new data hard disk

The virtual-machine data device is assigned to SCSI device 1:0

and the log device is assigned to SCSI device 2:0.

Figure 8. Creating a new hard disk log

Conclusion


Solution Guide

RDM—Add the data and log RDM disks and assign them to SCSI controller 1

(data) and SCSI controller 2 (log), as shown in Figure 9.

Figure 9. Assigning an RDM to a virtual machine

Conclusion

Virtualizing SAP HANA scale-up and scale-out systems enables an organization to benefit

from all supported VMware virtualization solutions and options (such as HA and live

migration using vSphere vMotion) to increase service level agreements and lower the

TCO. We verified that virtualized SAP HANA installations on Dell EMC storage arrays

meet the SAP performance KPIs for SAP HANA production deployments when we follow

the storage configuration best practices and supplemental configuration guidelines

described in this document.

Summary

References


References

The following documentation on EMC.com or EMC Online Support provides additional

relevant information. Access to these documents depends on your login credentials. If you

do not have access to a document, contact your Dell EMC representative.

Storage Configuration Best Practices for SAP HANA Tailored Datacenter Integration on EMC VMAX and VMAX3 Storage Systems

Storage Configuration Best Practices for SAP HANA Tailored Datacenter Integration on EMC VNX Series Unified Storage Systems

Storage Configuration Best Practices for SAP HANA Tailored Data Center Integration on EMC Unity Storage Systems

Storage Configuration Best Practices for SAP HANA Tailored Datacenter Integration on EMC XtremIO Storage

EMC VPLEX for SAP HANA Solution Guide

EMC Host Connectivity Guide for VMware ESX Server

Dell EMC SC Series Best Practices with VMware vSphere 5.x-6.x

The following documentation on the VMware website provides additional relevant

information:

Architecture Guidelines and Best Practices for Deployments of SAP HANA on VMware vSphere

The following documentation on the SAP website provides additional relevant information:

SAP HANA Installation Guide

SAP HANA Server Installation and Update Guide

SAP HANA Administration Guide

SAP HANA Storage Requirements

SAP HANA Certified Hardware Directory – Enterprise Storage

SAP Note 1788665SAP HANA Support for virtualized/partitioned (multitenant) environments

SAP Note 800326 Certified EMC solutions for Linux environments

SAP Note 1943937Hardware Configuration Check Tool - Central Note

SAP Community WIKI: SAP HANA on VMware vSphere

Dell EMC

documentation

VMware

documentation

SAP

documentation

http://www.emc.com/

https://support.emc.com/







http://www.emc.com/collateral/white-papers/h14283-vplex-sap-hana-sg.pdf

http://www.emc.com/collateral/TechnicalDocument/docu5265.pdf

http://en.community.dell.com/techcenter/extras/m/white_papers/20441056/download

http://www.vmware.com/



http://help.sap.com/hana/

http://help.sap.com/hana/sap_hana_server_installation_guide_en.pdf

http://help.sap.com/saphelp_hanaplatform/helpdata/en/7e/b0167eb35e4e2885415205b8383584/frameset.htm

http://help.sap.com/saphelp_hanaplatform/helpdata/en/33/0e5550b09d4f0f8b6cceb14a64cd22/frameset.htm

http://scn.sap.com/docs/DOC-62595

http://global.sap.com/community/ebook/2014-09-02-hana-hardware/enEN/enterprise-storage.html


https://launchpad.support.sap.com/#/notes/800326/E

http://service.sap.com/sap/support/notes/800326


http://scn.sap.com/docs/DOC-60470

vmware virtualized sap hana with dell emc storage · pdf filerequires an sap registration id.)...

Documents