sto2095be vsan readynode and build your own hardware …...rack servers blade servers composable...

61
Bhumik Patel Rakesh Radhakrishnan STO2095BE #VMworld #STO2095BE vSAN ReadyNode and Build Your Own Hardware Guidance VMworld 2017 Content: Not for publication or distribution

Upload: others

Post on 02-Jun-2020

15 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Bhumik Patel Rakesh Radhakrishnan

STO2095BE

#VMworld #STO2095BE

vSAN ReadyNode and Build Your Own Hardware Guidance

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 2: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

• This presentation may contain product features that are currently under development.

• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.

• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.

• Technical feasibility and market demand will affect final delivery.

• Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Disclaimer

2#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 3: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Agenda

1 Overview of VMware vSAN

2 vSAN VCG at a Glance

3 vSAN Hardware Platform Guidance

4vSAN Health Check and Disk Serviceability

5 vSAN Ready Node Sizer / Demo

#STO2095BE CONFIDENTIAL 3

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 4: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Overview of vSAN

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 5: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

HCI is the Fastest Growing Storage Segment

Source: Wikibon Server SAN Research Project, 2016

Switch to efficient,

server economics

Extend simple server

management to storage

Unlock affordable

server-side flash

Hyperscale

HCI(Server SAN)

Traditional

Storage

2012 2026

0

$10B

$20B

$30B

$40B

$50B

Total Storage Market $60B

#STO2095BE CONFIDENTIAL 5

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 6: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Modernization of the Data Center Being Fueled by HCI

Lower total costs

Greater agility and scale

Simplified management

Traditional 3-Tiered Architecture

Complex and Separate Silos

Servers

and Blades

External

Storage

Networking

Hardware

Hyper-ConvergedInfrastructure

Unified Management

VirtualizationCompute | Storage | Network

Server + Storage Network

Built on Industry-Standard

Servers and Switches

Virtualization

1

#STO2095BE CONFIDENTIAL 6

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 7: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

HCI Powered by VMware vSANRuns on any standard

x86 server

Pools SSDs/HDDs into

a shared datastore

Delivers enterprise-

grade security, scale and

performance

Managed through

per-VM storage policies

Deeply integrated with

the VMware stack

vSAN Storage

Managed by vCenter

vSphere vSAN

1

#STO2095BE CONFIDENTIAL 7

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 8: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN Adoption

Q1'14 Q2'14 Q3'14 Q4'14 Q1'15 Q2'15 Q3'15 Q4'15 Q1'16 Q2'16 Q3'16 Q4'16 Q1'18

vSAN Customer Adoption

150% YoY bookings growth

10,000 Customers

Fastest since ESX

At VMworld Las Vegas

#STO2095BE CONFIDENTIAL 8

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 9: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

All-in-one HCI Appliance

Dell EMC best-in-breed data protection

Rapid time-to-value with multiple configurations

Single, pro-active vendor support for software and hardware

Fully Customizable HCI

Choose from 15 different server vendors

Software and support flexibility

Backup agnostic to minimize change

Control HCI Deployments with Choice of DIY or Appliances

Dell EMC VxRail Appliances

VMware vSAN ReadyNodes

1

#STO2095BE CONFIDENTIAL 9

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 10: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Evolution of vSAN Workload Adoption

Business-Critical Apps Virtual Desktops (VDI)

VMware Cloudon AWS (VMC)

DR / DAManagement Clusters

Databases (SQL/Oracle)

Data Center on Wheels Containers

ROBOvSAN

#STO2095BE CONFIDENTIAL 10

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 11: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN VCG at a Glance

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 12: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Hardware Market Segmentation

10-20% vSAN Market; Emerging workloads expected to increase 3X in 2 years80% of vSAN Market

Storage

Dense

Compute

Intensive

Composable

Infrastructure

Archiving, video streaming,

analytics

Web apps, HPC, real time

analytics, In-memory DB

Data warehouses, Search engines

databases, Log aggregation

Rack Servers Blade Servers Composable Infrastructure

HPE – Apollo

Cisco – UCS S-series

Dell – FX2

HPE – Moonshot

Cisco – B-Series

HPE – Synergy

General

Purpose

Use cases BCA, Database, VDI,

ROBO, DR, Test/Dev

Hardware

TypeRack Servers

Supported

Hardware

Examples

Dell – R730XD

HPE – DL 380, 360

Cisco – UCS C-Series

2

#STO2095BE CONFIDENTIAL 12

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 13: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN ReadyNode Partners

308ReadyNodes

116Unique Server

Platforms

15OEM Server

Partners

2250Unique Drives

(NVMe, SSD, HDD)

&

2

#STO2095BE CONFIDENTIAL 13

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 14: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Sample vSAN ReadyNode Configuration

14

Storage IO Subsystem

2

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 15: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN Hardware Platform Guidance

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 16: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN ReadyNode Profile – All Flash

3

#STO2095BE CONFIDENTIAL 16

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 17: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Intel Xeon Scalable Processors Platforms

17

3

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 18: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN Hardware Platform Guidance

A vSAN ReadyNode Customization

B Boot Devices

C Storage Controller

D Flash based Devices (Caching & Capacity)

E Magnetic Disks (Capacity)

F Disk Groups and Compute

3

#STO2095BE CONFIDENTIAL 18

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 19: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

What You Can and Cannot Change in a ReadyNode

19

Compo

nents

Modifia

ble?

On vSAN

HCL?

Guidance

CPU√ ×

• Higher core count with similar or better CPU clock speed is supported.

• Switching between different generation of CPU platform is not supported e.g. Romley vs Purley.

Memory√ × • Adding more memory than what is listed is supported.

Caching

Tier

√ √

• Caching tier device needs to be certified on vSAN HCL

• It can be same or higher endurance and performance class.

• Follow Caching to Capacity ratio guidance discussed in this blog.

• The firmware and driver for the drive should be >= minimum version listed on the vSAN VCG.

Capacity

Tier√ √

• Caching tier device needs to be certified on vSAN HCL

• Capacity tier device needs to be of same or higher endurance and performance class.

• The firmware and driver for the drive should be >= minimum version listed on the vSAN VCG.

Controller× √

• Only tested, certified and listed controller configurations for that vSAN ReadyNode is supported.

• The firmware and driver for the controller should be an exact match as listed on the vSAN VCG.

NIC

√× • You can add more NICs if there are available slots in the server.

• NIC of similar or higher configuration allowed and supported e.g. 10G with 25G or 40G.

• The NIC needs to be IOVP (ESXi) certified.

Boot

Device √ ו Changing boot devices is allowed. Follow guidance in this KB article.

3

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 20: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

20

Items Modifiable Guidance

Disk Group √ • vSAN ReadyNodes are tested and certified with certain number of disk groups. But you can select any number of

disk groups up to 5 (maximum per node).

Disk Group Configuration √ • Change in disk group configuration is allowed e.g. instead of 2 * 1.6 TB, you can choose 4 * 800 TB.

• The changed drives need to belong to same or higher performance and endurance class.

SAS Expander × • SAS Expanders may be required to add more disk drives in a node. Only certified expanders are allowed.

• Refer to the footnote of the ReadyNode BOM to see whether an expander is certified or not.

Caching √ • Higher capacity “caching tier” device is supported than listed in ReadyNode BOM.

Capacity √ • Higher or lower capacity of “capacity tier” device is allowed than listed in ReadyNode BOM.

• Number of capacity drives may be changed. You can add or reduce the number of drives depending on your

overall capacity needs.

• You may need to adjust the caching tier size accordingly. Please follow the guidelines here.

Additional Certified IO

Controller√ • You can add additional certified controller in each node and configure disk groups behind the controller for better

performance.

Change in Storage Device

Protocol× • We do not support changing protocol within vSAN ReadyNode e.g. you can not replace a SAS drive with NVMe or

vice versa having same endurance and performance class.

Separate Boot Device

Controller√ • Boot device and vSAN data store need to be behind separate controllers.

#STO2095BE CONFIDENTIAL

3

Additional Guidance on vSAN ReadyNode Storage Components

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 21: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Device Price Size Endurance

in TBW

Storing

Logs

Storing

Traces

Storing Core

Dumps

Remarks – Pros & Cons for Use as

Boot Device

USB/SD $ Min:4GB

Recommended:

8GB

N/A –

endurance too

low to write

traces or logs

No

Yes, RamDisk

contents are

written only

on reboot

Yes, if ESXi

memory <=512

GB

1. Endurance too low to write logs & traces

2. Use syslog server & net dump collector

SSD $$ Min: 30 GB 512-1024

TBW

(Min: 130

TBW)

Yes,

always

Yes, always Yes, always 1. Endurance Requirements similar to

SATADOM

2. You lose a drive slot for vSAN

consumption

HDD $$ Min: 30 GB NA Yes,

always

Yes, always Yes, always 1. Like SSD, you lose drive slots.

SATADOM $$$ Min:30 GB 512-1024

TBW

(Min: 130

TBW

Yes,

always

Yes, always Yes, always 1. Logs, traces &dumps always stored.

2. Expensive, drives up overall cost

M.2 SSD $$ Min: 30 GB 512-1024

TBW

(Min: 130

TBW)

Yes,

always

Yes, always Yes, always 1. Mirrored M.2 provides redundancy

2. Provides controller separation for vSAN

datastore and boot device.

Boot Devices

#STO2095BE CONFIDENTIAL 21

3

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 22: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Storage Controller

22

Configuration Recommendation

Passthrough Mode vs RAID mode Passthrough mode recommended for operational simplicity

Single Controller vs. Multiple

Controllers per host

Multiple disk groups will provide better performance

Multiple controllers will provide better performance;

SAS Expander Need to use SAS expanders with single controller when number of disks >

physical ports on the controller

Firmware and Drivers Need to EXACTLY match with the listing on the VMware Compatibility Guide

for that specific vSAN release

Queue Depth >512 for All-Flash; >256 for Hybrid

vSAN and Non-vSAN Disks with same

Controller

Do not mix controller mode for vSAN and Non-vSAN disks

VMFS disks only for logging purposes and not for hosting VMs

#STO2095BE CONFIDENTIAL

3

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 23: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

#STO2095BE CONFIDENTIAL 23

# Pass-through (HBA) vs RAID 0 Mode

1Simplicity of Initial Setup – No need to format each disk behind the controller as single RAID

0 disks

2Simplicity of Ongoing Operations – Controller Auto discovers replaced drive. In RAID 0, you

have to manually run esxcli commands to discover the drives

3

No need for controller cache. Better performance with pass through compared to RAID 0

4S.M.A.R.T Data Unavailable through RAID 0. vSAN relies on S.M.A.R.T for predictive

failure analysis of drives

Storage Controller: Pass-through Preferred Over RAID 0 Mode

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 24: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Flash Based Devices

Hybrid

In vSAN hybrid ALL read and write operations always go directly to the Flash tier.

Flash based devices serve two purposes in vSAN hybrid architecture

1. Non-volatile Write Buffer (30%)

– Writes are acknowledged when they enter prepare stage on the flash-based devices.

– Reduces latency for writes

2. Read Cache (70%)

– Cache hits reduces read latency

– Cache miss – retrieve data from the magnetic devices

All Flash

In vSAN all-flash read and write operations always go directly to the Flash devices.

Flash based devices serve two purposes in vSAN All Flash:

1. Cache Tier (write buffer)

– High endurance flash devices.

– Listed on VCG

2. Capacity Tier

– Low endurance flash devices

– Listed on VCG

24#STO2095BE CONFIDENTIAL

3

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 25: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Performance & Endurance Classes

25#STO2095BE CONFIDENTIAL

3

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 26: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Flash Capacity Sizing – Hybrid

• The general recommendation for sizing vSAN's flash capacity is to have 10% of the anticipated consumed storage capacity before the Number of Failures To Tolerate is considered.

• Total flash capacity percentage should be based on use case, capacity and performance requirements.

– 10% is a general recommendation, could be too much or it may not be enough.

Measurement Requirements Values

Projected VM space usage 20GB

Projected number of VMs 1000

Total projected space consumption per VM 20GB x 1000 = 20,000 GB = 20 TB

Target flash capacity percentage 10%

Total flash capacity required 20TB x .10 = 2 TB

#STO2095BE CONFIDENTIAL 26

3

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 27: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Revised Caching Guidelines for All Flash

• Premise for Revised Guideline

– There is a limit to the rate of sustained I/O that can be written to caching tier by vSAN

– We measure max sustained writes per day to caching tier for different workloads.

– Based on this, we calculate the max amount of data written to caching tier per day

– Caching capacity size is calculated based on the max amount of data written per day

• Caching Tier Sizing

– Sustained writes per day (MB/s) = Total IOPS * Block Size * Write ratio * (1.5 Metadata write amplification) * (1.5 increase with RAID5) * Number of Disk groups

– GB written per day = (Throughput in MB/s * Hours per day * 3600) / (1024)

– Caching Tier Size per Host = (GB written per day) / Drive Endurance

– Example: To sustain ~300 MB/s writes (70/30 RW, Random) per day

• Max amount of data written to caching tier = 8TB per day

• Caching tier drive of 10 DWPD needs to be of size 800GB

Note: Caching tier size does NOT depend on usable capacity or capacity tier size

#STO2095BE CONFIDENTIAL 27

3

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 28: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

New All Flash Caching Guidelines – Total Cache per Host

Assumptions

- Fault Tolerance Method = RAID5 / RAID6

- Accounted for 30% future performance increase & impact of resync/rebuild

- While assuming max sustained throughput, IOPS decreases proportionately if block size increases

- Ready Node profile details: https://www.vmware.com/resources/compatibility/vsan_profile.html

- IOPS are assuming 4KB size. Large blocks divide accordingly.

- 2 Disk groups, delivering total Write Cache size so divide by 2 to determine drive size.

Read / Write Profile Workload TypesAF-8

80K IOPS

AF-6

50K IOPS

AF-4

25K IOPS

70/30 Read/Write; RandomRead Intensive,

Standard Workloads800 GB 400 GB 200 GB

>30% Write; RandomMedium Writes,

Mixed Workloads1.2 TB 800 GB 400 GB

100% Write; SequentialHeavy Writes,

Sequential Workloads1.6 TB 1.2 TB 600 GB

#STO2095BE CONFIDENTIAL 28

3

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 29: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Magnetic Disks (HDD)

• SAS/NL-SAS HDDs supported

– 7200 RPM for capacity

– 10000 RPM for performance

– 15000 RPM for additional performance

• Differentiate performance between clusters with SSD selection, and SSD:HDD ratio. Rule of thumb guideline is 10% of anticipated capacity usage

• SATA HDDs no longer supported due to reliability issues and poor performance

– NL SAS will provide higher HDD controller queue depth at same drive rotational speed and similar price point

29#STO2095BE CONFIDENTIAL

3

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 30: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Disk Groups

• Management construct within vSAN

– For Hybrid each disk group consists of maximum of 7 HDDs and 1 flash device

– For All-Flash each disk group consists of maximum of 7 flash devices and 1 flash device

• vSAN supports total of 40 disks per host

– This means maximum of 5 fully populated disk groups with 35 HDDs and 5 flash devices

• Design Recommendations

– Multiple disk groups create smaller failure domains

– Multiple disk groups provide better performance

– Implement Balanced configurations – e.g. do not configure different hosts with different number of disks per hosts and/or different number of disk groups

30#STO2095BE CONFIDENTIAL

3

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 31: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Compute

• Components such as CPU & Memory can vary from Ready Node listings as long as listed on vSphere HCL

• Balanced configuration RECOMMENDED

31

BALANCED vSAN 4 NODE CLUSTER

HOST 1 HOST 2 HOST 3 HOST 4

Same CPU, No. of CoresMemory

Same class, capacity & number of SSDs

Same class, capacity & number of HDDs

Same controller & count Of controllers

#STO2095BE CONFIDENTIAL

3

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 32: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN Network Guidance

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 33: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN Networking Requirements

33

10G+ network for

All-Flash

Virtual Switch &

vmnicUnicast Traffic Isolation & QoS

4

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 34: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN and ESX Networking Communications

#STO2095BE CONFIDENTIAL 34

vSphere Distributed Switch (It’s included!), vSphere Standard Switch and Port Groups

vSAN supports both vSphere Virtual Switches:

– vSphere Standard Switch – provide a de-centralized management interface (per host).

– vSphere Distributed Switch – provide a centralized management interface as well as advanced networking features such as vSphere Network I/O Control, LLDP, Advanced link balencing etc.

vSphere Standard and Distributes switches provide ways to define how connections are made through the switch and to the networks

– Standard and Distributed Port Groups – are used to connect network services through the VSS or VDS. Typically, switches are associated with one or more port groups.

– Used to define how a connection is made through the switch on the network.

– specify port configuration options such as bandwidth limitations and VLAN tagging policies for each member port.

4

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 35: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN and ESX Networking Communications

35

NIC Teaming and Load Balancing Algorithms

Design for Operational Capability

– vSAN only needs a single vmkernel port

– Active/Passive Is simple

– LBT, LACP and other methods supported

vDS required for:

– Route based on Physical NIC Load (LBT)

– Dynamic LACP, and advanced Hash support

– Remember 1+1 doesn’t always equal 2!

– Design for failure

Multi chassis link aggregation capable switches

4

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 36: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Unicast Considerations

• Configuration Options:

– Multicast replaced with unicast in vSAN 6.6. Layer 2/3 supported

– Host discovery brokered by vCenter

– IPv6 Supported

– Still recommend isolating VLAN/Subnet routing for security

– Jumbo Frames not required

36

4

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 37: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN and ESX Networking Communications

37

vSphere Advanced Networking Feature – Network I/O Control

– vSAN vSphere Network I/O Control Recommendations

• Management traffic – 20 shares

• Virtual Machine traffic – 30 shares

• vMotion traffic – 50 shares

• vSAN traffic – 100 shares

– Shared allocation calculation should be based on total available bandwidth and traffic demand per service

– Beware Reservations

– Avoid Limits

– NIOC a better choice than NPAR or Converged Network Adapters. Better visibility, better control.

4

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 38: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN Link Speed

38

Oversubscription concerns and uses

Rebuild speed

• Gets Faster as you add more nodes

• Only as long as you do not have bottlenecks

Things to Consider Carefully

• Cisco FEX or other devices that have no east/west capability.

• Poor throughput between fault domains

• Oversubscribed Line Cards

• Legacy Three Tier Network Designs

4

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 39: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Next Generation Networks

39

Refreshing your network

25G is the new 10Gbps

• Cost is very close to 10Gbps

• 100Gbps replacing 40Gbps

• 100Gbps ports can be broken out into 25Gbps

RDMA/iWARP

Keep an eye on support for NIC’s and Switches

Not supported today for vSAN traffic

4

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 40: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN Health Check and Disk Serviceability

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 41: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN Update Manager Strategy

42

OEM Server Lifecycle

Management Stack

I/O Controller

Disk Drives

Expander Backplane

Network Card

BIOS

Common API-based

Framework using VUMESXi

Firmware Utilities

1. Firmware/driver + software updates for entire stack

2. Common framework for ESXi & vSAN3. ETA: Next 12 – 18 months

Orc

hestr

ation

by V

UM

OEM Management API(SUM, Power Tools, UCSM)

ESXi & vSAN

1. Firmware/driver updates only for controller2. vSAN 6.6 – Q2 20173. Long Term: Shift ALL OEMs to VUM

Orc

he

stra

tion

b

y v

SA

NESXi Firmware Utilities

(StorCLI, PERCCLI)

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 42: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN Config Assist

VMware HCI (vSphere + vSAN)

Overview

• Auto configuration and compliance checks

✓ Both S/W & H/W related checks

✓ Easy remediation

✓ Applicable throughout lifecycle

• Auto update of SCSI controller driver & firmware

✓ Fully UI-driven orchestration: scan, download & upgrade

✓ HCL-aware, free from target version search & find

✓ Integrated with H/W vendor approved process & tool

• Supported controllers from Dell, Lenovo, Fujistu and Supermicro

Benefit

• Simplified HCI production readiness and ongoing hardware updates

5

#STO2095BE CONFIDENTIAL 43

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 43: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN Disk Serviceability

VMware HCI (vSphere + vSAN)

Overview

• Require to service a disk drive

✓ Datacenter admin needs to identify the failed drives

✓ Identified drives need to blink LED

✓ Drive is serviced (pulled and replaced)

• Across Drive and Controller Vendors

✓ Few OEM platforms are supported and planned for others

Benefit

• Simplified failed disk identification for serviceability in HCI environment

5

#STO2095BE CONFIDENTIAL 44

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 44: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN ReadyNode Sizer

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 45: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

vSAN ReadyNode™ Sizer

46

✓ Quick Sizing in 5-6 steps

✓ Advanced Sizing for more customization

✓ Multiple use case scenarios

• General purpose workloads

• Databases

• VDI

✓ Recommendations for hardware and sizing for compute and storage

✓ Best vSAN ReadyNode™ profile match

✓ Sizing for All Flash based on vSAN 6.6

✓ Access Sizer with My VMware or partner central credentials

vSAN Sizing and TCO Tool

https://vsansizer.vmware.com

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 46: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Change VM Profiles, HWIntegrate with APIs

What is better about the NEW vSAN ReadyNode™ Sizer?

Highly Customizable

vSAN

Designed for vSAN

Dedup, Stretched Clusters

Uses vSAN 6.6

Performance Data

Single and Multi VM Profiles

Calculate Compute & Storage

Size for HCI

Cores, RAM, Capacity

APIs enable integration

with other tools

© 2017 VMware Inc. All rights reserved.

Accounts for Limits, Overheads, New Features

Factors Workload IO

Profile

#STO2095BE CONFIDENTIAL 47

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 47: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Recommending the Most Optimized vSAN Cluster Configuration is Simple

48

ReadyNode

Sizer

Server Pack

Server

Configuration

Other

Specifications

• Number of Hosts• Cluster Size• VM Distribution• Server Configuration• Capacity (Raw & Usable)

• Max Memory• CPU• Cores• Disk Group Configuration• Cache/Capacity per host

• Drive Classes• vSAN License Required• vSAN version• Sizing Assumptions

I/O

profile

VM

profile

Server

Details

INPUT OUTPUT

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 48: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Using Design Considerations, Compute the Most Optimized Cluster Size

49

COMPUTE

BOUND

CLUSTER SIZE

IOPS BOUND

CLUSTER SIZE

CAPACITY

BOUND

CLUSTER SIZE

MEMORY

BOUND

CLUSTER SIZE

[MAX]OPTIMIZED

CLUSTER SIZE

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 49: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Design Criteria for Sizing

Step 1. Is the workload Compute Heavy?

– Check: Are there sufficient Cores for Application and vSAN?

– Parameters Considered: VM consumption, Clock Speed, Cores per Socket, CPU headroom and Cores per Guest IOPS

– Pass Criteria: If sufficient, calculate number of hosts. Else, workload is Compute Bound. Calculate additional number of hosts required for extra compute

– Sample Workload: Web Applications

50#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 50: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Design Criteria for Sizing

Step 2. Is the workload IOPS Heavy?

– Check: How much IOPS can be pushed?

– Parameters Considered: IO profile and Max IOPS/Bandwidth per disk group, IO amplification

– Pass Criteria: If sufficient, compute number of disk groups. Else calculate additional number of hosts for extra IOPS

– Sample Workload: Exchange, Financial Applications, Transactional

51#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 51: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Design Criteria for Sizing

Step 3. Is the workload Capacity Heavy?

– Check: How much capacity is required based on drive size specified?

– Parameters Considered: Drive slots, Max drive size used

– Pass Criteria: If Raw capacity required > Max capacity supported per host, logic is capacity bound. Else calculate additional number of hosts for extra capacity

– Sample Workloads: Archival, Video Streaming

52#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 52: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Design Criteria for Sizing

Step 4. Is the workload Memory Heavy?

– Check: Does the system have enough memory to meet user’s requirements?

– Parameters Considered: VM Consumption, Max RAM, IOPS per VM, IO Amplification

– Pass Criteria: If sufficient, compute Cache size, HCI Profile, Drive Classes. Else calculate additional number of hosts for extra memory

– Sample Workloads: Real-Time Analytics

53#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 53: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Sizing Workflow

54

Step 5: Recommendation

Obtain Best

Match vSAN

ReadyNode

Recommendation

Step 2: IOPS Bound

Sufficient IOPS

per Host

Calculate Number

of Disk Groups

Step 3: Capacity Bound

Raw Capacity

Calculate Disk

Layout

Step 4: Memory Bound

Total vRAM <=

Max RAM?

Calculate Cache

Size

Step 1: Compute Bound

Sufficient Cores

for Compute?

Calculate Number

of Hosts

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 54: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

ReadyNode Sizer Demo

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 55: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Wrap Up: vSAN Hardware Guidance

56

Hardware Selection &

Design

Always use certified

ReadyNodes

Pick ReadyNode series

based on expected IOPS,

VM density & capacity

Use a balanced

configuration

Always use latest VMware

HCL certified versions of

firmware & driver for

controllers.

Design for availability &

future growth

Size for Performance

& Capacity

Ensure SSD:HDD ratio is

1:10 of usable for Hybrid

For All Flash, follow new

guidelines based on write

intensity

Follow ReadyNode guidance

to pick right class of drives

(HDD & SSD)

Pick 8-Series RNs for

performance intensive

workloads

Use SAS Expander based

ReadyNodes for capacity

intensive workloads. SAS

expanders are certified on a

per platform basis. Ensure

your ReadyNode is certified

for Expanders

Recommended Best

Practices

SAS or NL-SAS recommended

over SATA for performance &

reliability

Avoid vmfs data stores on boot

devices which are behind the

same controller as vSAN data

store

Ensure controller certified and

on the vSAN VCG.

Follow vSAN ReadyNode What

Can (Can’t) Change Guidance

Blog

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 56: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Call to Action

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 57: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

3 Easy Ways to Learn More about vSAN

58

• Live at VMworld

• Practical learning of vSAN, VxRail and more

• 24x7 availability online– for free!

vSAN Sizer

vSAN Assessment

New vSAN Tools

• StorageHub.vmware.com

• Reference architectures, off-line demos and more

• Easy search function

• And More!

Storage Hub Technical Library Hands-On Lab

Test drive vSAN

for free today!

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 58: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Nerd Out With These Key vSAN Activities at VMworld

#HitRefresh on your current data center and discover the possibilities!

Earn VMware digital badges to

showcase your skills

• New 2017 vSAN Specialist

Badge

• Education & Certification Lounge:

VM Village

• Certification Exam Center:

Jasmine EFG, Level 3

Become a

vSAN Specialist

Learn from self-paced and expert

led hands on labs

• vSAN Getting Started Workshop (Expert led)

• VxRail Getting Started (Self paced)

• Self-Paced lab available online 24x7

Practice with

Hands-on-Labs

Discover how to assess if your IT

is a good fit for HCI

• Four Seasons Willow Room/2nd floor

• Open from 11am – 5pm Sun, Mon, and Tue

• Learn more at Assessing & Sizing in STO1500BU

Visit SDDC

Assessment Lounge

59

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 59: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

Resources

60

VMworld Sessions:

1. Redefine vSAN Deployments with Next Generation Intel® Xeon® processor, Intel® Optane™ and Intel® 3D NAND SSDs [STO2705BU]

2. Turbocharge All Flash vSAN on Next Generation Hardware [STO2524BU]

vSAN VCG:https://www.vmware.com/resources/compatibility/search.php?deviceCategory=vsan

vSAN Hardware Guidance:https://www.vmware.com/resources/compatibility/vsan_profile.html?locale=en

vSAN ReadyNode Customization:https://blogs.vmware.com/virtualblocks/2017/03/14/can-cannot-change-vsan-readynode/

For any questions on vSAN hardware; please reach out to vSAN Hardware PM [email protected]

#STO2095BE CONFIDENTIAL

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 60: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

VMworld 2017 Content: Not fo

r publication or distri

bution

Page 61: STO2095BE vSAN ReadyNode and Build Your Own Hardware …...Rack Servers Blade Servers Composable Infrastructure HPE –Apollo Cisco –UCS S-series Dell –FX2 HPE –Moonshot Cisco

VMworld 2017 Content: Not fo

r publication or distri

bution