vmware virtual san for esxi 6.0 with vmware horizon vie vsan 6.2 for es… · vmware virtual san...

A Dell Technical White Paper

VMware Virtual SAN for ESXi 6.0 with VMware Horizon View Options for utilizing local storage space by creating a virtual SAN

Dell Wyse Solutions Engineering July 2015

2 VMware Virtual SAN for ESXi 6.0 with VMware Horizon View | v.6.7

Revisions

Date Description

July 2015 Initial release v.6.7

THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND

TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF

ANY KIND. Copyright © 2015 Dell Inc. All rights reserved. Dell and the Dell logo are trademarks of Dell Inc. in the

United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their

respective companies.


Table of contents Revisions ............................................................................................................................................................................................. 2

Executive summary .......................................................................................................................................................................... 4

1 Using Virtual SAN as a solution ............................................................................................................................................... 5

1.1 Solution overview............................................................................................................................................................ 5

1.2 Virtual SAN configuration .............................................................................................................................................. 7

1.2.1 Networking ....................................................................................................................................................................... 7

1.2.2 Disk configuration ........................................................................................................................................................... 8

1.2.3 VM Storage Policies for Virtual SAN ............................................................................................................................. 9

1.2.4 VMware Virtual SAN Capacity ...................................................................................................................................... 11

2 Solution implementations ...................................................................................................................................................... 12

2.1 Dell Wyse Datacenter Virtual SAN solution .............................................................................................................. 12

2.2 VDI Virtual SAN Low, Medium, and High-Capacity hardware configurations ................................................... 12

2.3 Virtual SAN infrastructure ............................................................................................................................................ 13

2.4 Software configurations ............................................................................................................................................... 14

2.5 Performance and test results ...................................................................................................................................... 14

2.5.1 200 Standard users on a single Virtual SAN node ................................................................................................... 14

2.5.2 VMware Virtual SAN Observer test results ................................................................................................................ 16

2.5.3 375 Enhanced users on a 3 node cluster .................................................................................................................. 17

2.6 Storage controller ......................................................................................................................................................... 18

3 Conclusion ................................................................................................................................................................................ 19

4 Acknowledgements ................................................................................................................................................................. 20

A Dell H730P PERC configuration for Virtual SAN ................................................................................................................ 21


Executive summary

Shared storage is basically a set of storage devices that are connected to and used by two or more

systems. The most commonly used term in IT departments for this shared storage is SAN (Storage area

network). Typically a storage area network is part of the overall network of computing resources for an

enterprise and it would contain separate storage units connected to the ESXi host(s) via Fibre Channel,

iSCSI or configured as a NFS. In a VMware ESXi environment the storage is divided into volumes and some

or all of these volumes are then presented to ESXi hosts simultaneously. The VMware administrator can

then decide whether to create VMFS data stores on these volumes or present them as raw device

mappings to individual virtual machines. Typically the storage devices come with their own configuration,

administration and/or monitor software that can be used in addition to VMware tools already available in a

VMware virtual environment.

VMware Virtual SAN is a software defined storage solution introduced by VMware in 2012 that allows you

to create a clustered data store from the storage (SSDs and HDDs, or all-flash using SSDs and PCIeSSDs)

that is present in the ESXi hosts. The Virtual SAN solution simplifies storage management through object-

based storage systems and fully supports vSphere enterprise features such as HA, DRS and vMotion. The

Virtual SAN storage cluster must be made up of at least three ESXi servers. VMware Virtual SAN is built into

the ESXi 6.0 hypervisor and can be used with ESXi hosts that are configured with PERC RAID controllers.

To be able to use Virtual SAN in a hybrid configuration, which is the context of this document, you will

need at least one SSD and one HDD in each of the servers participating in the Virtual SAN cluster and it’s

important to note that the SSD doesn’t contribute to the storage capacity. The SSDs are used for read

cache and write buffering whereas the HDD’s are there to offer persistent storage. Virtual SAN is highly

available as it’s based on the distributed object-based RAIN (redundant array of independent nodes)

architecture. Virtual SAN is fully integrated with vSphere. It aims to simplify storage placement decisions

for vSphere administrators and its goal is to provide both high availability as well as scale out storage

functionality.

VMware Virtual SAN offers an easy to configure, highly redundant, high performance storage system

without the need to purchase any additional hardware. Performance should mimic single hop iSCSI with

very high throughput and no bottlenecks.


1 Using Virtual SAN as a solution

1.1 Solution overview VMware Virtual SAN is software-defined storage solution fully integrated into vSphere. Once enabled on a

cluster, all the magnetic hard disks present in the hosts are pooled together to create a shared data store

that will be accessible by all hosts in the Virtual SAN cluster. Virtual machines can then be created and a

storage policy can be assigned to them. The storage policy will dictate availability / performance and

sizing.

Figure 1 Virtual SAN Clustered Datastore (Artwork copyright VMware 2014)

From a hardware perspective, at least three ESXi hosts are required for the Virtual SAN cluster. Each host

will need at least one SSD and one HDD. The SSD acts as a read cache and a write buffer. The read cache

keeps a list of commonly accessed disk blocks and the write cache behaves as a non-volatile write buffer.

It is essential to the performance of the Virtual SAN as all I/O goes to the SSD first. The higher the

performance of the disks then the better the performance of your virtual machines. It’s important to

determine the number of simultaneous write operations that a particular SSD is capable of sustaining in

order to achieve adequate performance.

All virtual machines deployed to Virtual SAN have an availability policy setting that ensures at least one

additional copy of the virtual machine data is available; this includes the write cache contents. When a

write is initiated by the VM then it is sent to both the local write cache on the owning host and also to the

write cache on the remote hosts. This ensures we have a copy of the in cache data in the event of a host

failure and no data will get corrupted. If a block is requested and not found in the read cache, the request

is directed to the HDD.


Figure 2 Virtual SAN Datastore for Performance

Magnetic hard disk drives (referred to as HDDs from here on) have two roles in Virtual SAN. They make up

the capacity of the Virtual SAN data store as well as making up components for a stripe width. SAS, NL-SAS

and SATA drives are supported

VMware recommends configuring 10% of projected HDD space as SSD storage on the hosts. If a higher

ratio is required then multiple disk groups will have to be created as there is a limit of 1 SSD per disk group.

Virtual SAN implements a distributed RAID concept across all hosts in the cluster, so if a host or a

component within a host (e.g. an HDD or SSD) fails then virtual machines still have a full complement of

data objects available and can continue to run. This availability is defined on a per-VM basis through the

use of VM storage policies.

Setting up Virtual SAN is very straight forward and simple. The following diagram shows the setup

workflow.

Figure 3 Virtual SAN Workflow


1.2 Virtual SAN configuration

1.2.1 Networking Virtual SAN cluster networking includes at least two vmkernel ports, one for management traffic and one

for Virtual SAN traffic. If vMotion, Storage vMotion or High Availability functionality is required in addition, a

third vmkernel port is to be configured for this.

Virtual SAN traffic can be used on 1Gb networks as well as 10Gb networks. Standard switch configuration

can be used for Proof of Concept, while VMware distributed virtual switch configuration is highly

recommended for production versions.

Network vmkernel adapter configuration for the host management traffic using a 1Gb network with

standard switch. It is recommended that the network configuration for the Virtual SAN storage is a 10Gb

network with distributed switch configuration. The following screenshot shows the VMkernel adapter for

the management network (vmk0) on a standard switch and vSANNetwork (vmk1) on a distributed switch.

Figure 4 Virtual SAN Storage network adapters

The distributed switch configuration is the same on all three Virtual SAN storage hosts. It is recommended

to have at least 2 uplinks for each host to provide load balancing and fail back redundancy.

Figure 5 Virtual SAN network distributed switch uplink port group


1.2.2 Disk configuration To enable Virtual SAN, simply select the Datacenter in vSphere, go to menu Manage, Settings and General.

Click Edit button and select Turn ON Virtual SAN. There are two modes when configuring Virtual SAN:

automatic and manual. If you setup Virtual SAN in automatic mode then all empty local disks will be used

in the creation of the shared data store. If you configure it in manual mode then you can manually select

which disks to use in the creation of the data store.

Figure 6 Virtual SAN disk configuration settings

The Virtual SAN is configured from the menu provided for a cluster. In this case, this is the “Virtual SAN”

cluster containing the 3 ESXi hosts for virtual SAN storage:

Figure 7 Virtual SAN storage general view


The size of the newly created data store will be an aggregate of all the disks you have selected for use in

the Virtual SAN (HDD’s only, SSD’s will not be used in this calculation). For example if you have four hosts

each containing four HDD’s that are 10 GB each then the data store will be 160 GB.

Figure 8 Virtual disk group view

Adding extra storage to the cluster is as simple as adding the new host into the cluster, configuring the

Virtual SAN vmkernel network and claiming the disks either automatically or manually. The Virtual SAN

data store will increase in size automatically.

1.2.3 VM Storage Policies for Virtual SAN Storage policy plays a major role for VMware Virtual SAN strategy and performances. After data store

creation you can create VM storage policies to meet VM availability, sizing and performance requirements.

The policies are applied down to the Virtual SAN layer when a VM is created. The VM virtual disk is

distributed across the Virtual SAN datastore per policy definition to meet the requirements.

VMware Horizon View 6.0 has a built in storage policy for Virtual SAN. When creating a desktop pool in

View, select Use VMware Virtual SAN option for Storage Policy Management. See below:


Figure 9 Storage Policy Management options in View 6.1

A set of storage policies will be created in vSphere vCenter for the pool by View.

Figure 10 vSphere VM storage policy created by Horizon View

Each storage policy defines the storage capabilities which will be applied to the VMs in the pool.

Figure 11 vSphere storage policy rule set definition


Details and definitions of each policy and rule can be found in the Appendix. VMs that have storage

policies successfully defined and applied are shown as compliant in vSphere.

Figure 12 VM Storage Policy compliant status

1.2.4 VMware Virtual SAN Capacity The following table shows the current minimum and maximum capacity for each virtual SAN storage

cluster.

Virtual SAN Limits Minimum Maximum

Number of supported ESXi hosts per Virtual SAN cluster 3 64

Number of supported VMs per host n/a 200*

Disk groups per host 1 5

HDDs per disk group 1 7

SSDs per disk group 1 1

Components per host n/a 9000

Components per object n/a 64

Table 1 Virtual SAN Capacity *Current VMware Supported Soft Limit

As a general guidance, the number of SSDs required is determined by the number of disk groups

configured on the Virtual SAN storage cluster. The size of the SSDs is determined by the projected storage

capacity required HDDs on that cluster.

For detailed design and sizing guidance, please see the VMware Virtual SAN 6.0 Design and Sizing Guide

located here: https://www.vmware.com/files/pdf/products/vsan/VSAN_Design_and_Sizing_Guide.pdf

https://www.vmware.com/files/pdf/products/vsan/VSAN_Design_and_Sizing_Guide.pdf


2 Solution implementations

2.1 Dell Wyse Datacenter Virtual SAN solution To implement Virtual SAN, Dell Wyse Datacenter recommends below base configuration:

Host: Dell PowerEdge R730 server with 2 Intel E5-2697v3 2.60GHz

Cluster: min 3 ESXi 6.0 hosts, max 64

RAM: 384 GB (24 x 16 GB RDIMMs)

Storage adapter: Dell PERC H730P RAID controller, 2GB NV Cache

NIC (10GB for Virtual SAN network): QLogic 57800 and 57810 10 Gigabit Ethernet

NIC (1GB for management and VDI): Broadcom 5720 Gigabit Ethernet

Network switch (1Gigabit): Dell Networking S55 or S60

Network switch (10 Gigabit): Dell Networking S4810

For Dell PowerEdge 13G server R730 configurations please refer to Appendix C.

2.2 VDI Virtual SAN Low, Medium, and High-Capacity hardware

configurations There are 3 configurations based on the above baseline:

Low: Each host with up to 3 200 GB SSD SATA Mix Use MLC with up to 12 HDD

Medium: Each host with up to 3 200 GB SSD SATA Mix Use MLC with up to 12 HDD

High-Capacity: Each host with up to 3 800 GB SSD SATA Mix Use MLC with up to 12 HDD

If the HDD used in the configuration has capacity of 300GB each, the total datastore capacity in both

configurations with a 3 host cluster is 10.8TB (Value Config with 12 HDDs).

Low Medium High

Component Description Qty. Description Qty. Description Qty.

Server PowerEdge R730 1 PowerEdge R730 1 PowerEdge R730 1

Processor Intel E5-2697v3 2.60GHz

2 Intel E5-2697v3 2.60GHz

2 Intel E5-2697v3 2.60GHz

2

RAM 16 GB RDIMMs 24 16 GB RDIMMs 24 16 GB RDIMMs 24

Storage Controller Dell PERC H730P 1 Dell PERC H730P 1 Dell PERC H730P 1

NIC Broadcom 5720 Gigabit Ethernet

4 Broadcom 5720 Gigabit Ethernet

4 Broadcom 5720 Gigabit Ethernet

4

QLogic 57810 10 Gigabit Ethernet

1 QLogic 57800 10 Gigabit Ethernet


1

QLogic 57800 10 Gigabit Ethernet



1

SSD 200 GB SSD SATA 11 200 GB SSD SATA 22 800 GB SSD SATA 1


Mix Use MLC (Up to 3 with 1 per disk group)



HDD

300 GB 15K 2.5 Inch 6Gbps SAS (Up to 12 when using 2 or 3 disk groups)

6

300 GB 15K 2.5 Inch 6Gbps SAS (Up to 12 when using 2 or 3 disk groups)

12

1.2 TB 10k 2.5 Inch 6Gbps SAS (Up to 12 when using 2 or 3 disk groups)

12

Table 2 Optimized Hardware list for the Low, Medium, and High-Capacity (two disk group) Virtual SAN

configurations

2.3 Virtual SAN infrastructure Figure 15 below shows Dell Wyse Datacenter Virtual SAN implementation of 3 host cluster.

1000

=OR

G10

0=G

RN

10=O

FF

AC

T/L

NK

AC

T/L

NK

4/105/11

6/12

STACK ID

3/9 Unit 7-121/7 2/8MASTER

PWRRPS

FAN

DIAG

TEMP

RESET

LNK/ACT LNK/ACT LNK/ACT LNK/ACT

49 50 51 52

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46 48

47

FDX

/HDX

LNK

/ACT1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

QSFP+

MASTERSYS FAN PSU

RS-232

ETHERNETLNK ACT4644424038363432302826242220181614121086420 48 56

52 60

SFP+

Force10 S4810P

1000

=OR

G10

0=G

RN

10=O

FF

AC

T/L

NK

AC

T/L

NK

1000

=OR

G10

0=G

RN

10=O

FF

AC

T/L

NK

AC

T/L

NK

VMware Management traffic

Virtual SAN traffic

R720 Management

R720 Compute 1

R720 Compute 2

R720 Compute 3

S55/S60 1Gb Switch

S4810 10Gb Switch

Figure 13 Virtual SAN infrastructure

The S4810 switch carries Virtual SAN traffic and the management traffic is carried through PC6248 switch.

Three R730 hosts joined together to create Virtual SAN cluster and another R730 hosts all the

management VM (vSphere vCenter, Horizon View Connection Server, SQL and fileshare etc.).

R730 Management

R730 Compute 1

R730 Compute 2

R730 Compute 3


2.4 Software configurations VMware ESXi 6.0.0 build 2494585

vSphere vCenter 6.0.0 build 2502243

VMware View 6.0 build 1782638

VM pool Windows 8.1 virtual machines

2.5 Performance and test results Tests were carried out based on Dell Wyse Datacenter Virtual SAN solution standard and high-capacity

configurations. This section includes the test results for 375 Enhanced users and a single host running 200

Standard user desktops. Performance Metrics from both the Compute hosts and Virtual SAN are included.

At the time of this document, the performance stats of Virtual SAN storage are not available in vSphere

client (web client or standalone client). The only way to monitor Virtual SAN performance is using Ruby

vSphere Console (RVC)’s built-in Virtual SAN observer function. Virtual SAN Observer provides a deep dive

view into Virtual SAN such as storage IO and latency, disk layer and cache hit rate etc.

2.5.1 200 Standard users on a single Virtual SAN node For this test a VMware View Pool of 200 desktops were deployed and rebalanced to place all VMs on Node

A in the cluster. All 200 sessions logged in successfully and subsequently logged off. The Login VSI score

for this test showed that additional desktops could be added based on resource utilization.

The processors used for testing were the Intel 2697v3 14 Core models. At peak, the test CPU on the

compute host reached 90%. It’s important to point out that Turbo Boost was enabled by default on the

node providing an additional 30% CPU processing capability.

0

10

20

30

40

50

60

70

80

90

100

CPU Utilization (%)


Peak CPU was recorded at 88% of steady state for 200 users.

Next we look at the memory utilization for the host. As you can see from the chart Active Memory was

extremely low remaining under 30% throughout the test. Consumed memory as expected was close to

the maximum capacity of the host which was 384GB.

Below is the Network Utilization chart for this test. Network for the compute host was extremely low as

expected.

0

50000000

100000000

150000000

200000000

250000000

300000000

350000000

400000000

450000000

Memory Utilization (KB)

Active Memory Utilization Consumed Memory Utilization

0

10000

20000

30000

40000

50000

60000

Network Utilization (KBps)


2.5.2 VMware Virtual SAN Observer test results In addition to the compute host metrics, a screen capture was taken from Virtual SAN Observer which

provides performance metrics for Virtual SAN. Below is a summary graph taken when all users were

logged in. The main takeaway here is the low latency value of 2ms and the Datastore not exceeding 2

outstanding I/O’s.

A closer look at this capture shows the breakdown of the latency record which is recorded at an average

of 2ms.

At peak of steady state the recorded IOPS for the standard user on the Virtual SAN datastore is 670 IOPS.

This equates to 3.35 IOPS per user.


2.5.3 375 Enhanced users on a 3 node cluster This section outlines the results from a 375 Enhanced user workload test. The CPU did not exceed 94% at

the peak of testing. Factoring Turbo boost savings this would be well within the capability of the

processors.

Memory utilization for each host in the cluster shows near identical consumption with active memory also

showing similar performance. Each node had a total of 384GB of physical memory and active memory

utilization was less than 30%.

0

20

40

60

80

100

120

CPU Utilization(%)

Compute Host A Compute Host B Compute Host C

0

50000000

100000000

150000000

200000000

250000000

300000000

350000000

400000000

Memory Utilization (KB)

Compute Host A Consumed Compute Host A Active Compute Host B Consumed

Compute Host B Active Compute Host C Consumed Compute Host C Active


The networking utilization was also very similar in behavior and extremely low as most of the operations

were local to the cluster. Traffic across the physical NICs would be to the core to access the file server and

to authenticate against active directory services.

2.6 Storage controller Virtual SAN supports storage controllers in both pass-through and RAID 0 modes. While the Dell H730P

PERC supports both modes and is the controller of choice for implementing Virtual SAN environments, it

was configured to run in pass-through mode (preferred) for all tests.

In pass-through mode, the vSphere hypervisor is given direct access to the SSDs and HDDs providing

Virtual SAN complete control of the local drives attached to the storage controller.

RAID 0 mode is implemented by creating a single disk RAID 0 volume via the storage controller

configuration and present it to the Virtual SAN disk group as an individual SSD or HDD.

Pass-through and RAID 0 modes have similar performance characteristics in VDI solutions using Virtual

SAN. The main drawbacks for RAID 0 are that it does not support hot plug in operation and each disk

needs to be configured as RAID 0 which increases the configuration complexity. Dell Wyse Datacenter

recommends implementing pass-through mode for virtual SAN.

See appendix A for the Dell H730P controller configuration.

0

10000

20000

30000

40000

50000

60000

Network Utilization (KBps)

Compute Host A Compute Host B Compute Host C


3 Conclusion VMware virtual SAN brings the benefits of software defined storage to vSphere. By clustering member

servers’ SSD and HDD, it creates a flash optimized and highly resilient shared datastore for VDI

environments. The Virtual SAN architecture allows easy scaling through the addition of disks or hosts.

The main finding of this testing supports the claim that 200 Virtual desktops can run on a single Virtual

SAN node. As scaling is linear, a 3 node cluster can support 600 Login VSI Standard user desktops.

Additionally, a 3 node Virtual SAN cluster running the Login VSI Enhanced user workload can support 375

desktop sessions. Performance on the cluster was excellent with no bottlenecks observed. Pool

operations including provisioning, recompose and reboot were very fast. A subjective user test on each

platform showed good performance with no screen drag, delay or jitter. A YouTube HD video played with

no issues further demonstrating the good performance.

Overall, the Virtual SAN performance was excellent. Management was straight forward and all operations

were fast to complete.


4 Acknowledgements Special thanks are given to Wade Holmes, Rahim Ibrahim and the rest of the VMware team for their

collaborative assistance in the development work and the review of this document.

Dell Hypervisor Engineering Team

Kiran Devarapalli, Software Development Staff Engineer

Karan Singh Gandhi, Software Development Engineer

Dell Wyse Datacenter Solution Engineering Team

Yinglong Jiang, Solution Engineer

Kristina Bako, Solution Engineer

Paul Wynne, Solution Engineer

Gus Chavira, Solution Architect

David Hulama, Technical Marketing

Rick Biedler, Solutions Development Manager


A Dell H730P PERC configuration for Virtual SAN

One of the significant differences in 13G servers is the local storage controller. In addition to the RAID

configuration, the new Dell PERC H730P controller offers a pass-through mode which acts similar to a

SAS disk controller. Pass-through mode provides the Virtual SAN direct access to the SSD and HDD and is

the recommended configuration for Virtual SAN.

Below screenshot shows the PERC H730P configured to HBA-Mode under Personality Mode in Ctrl Mgmt

page. This sets the controller into pass-through mode.

Once the PERC H730P is configured to pass-through mode, all disks are marked as “non-RAID”, as shown

in following screenshot.

vmware virtual san for esxi 6.0 with vmware horizon vie vsan 6.2 for es… · vmware virtual san...

Documents