8/3/2019 Io Virtualization

1/16

This session may contain product features that are currently under development. This session/overview ofthe new technology represents no commitment from VMware to deliver these features in any generallyavailable product. Features are subject to change and must not be included in contracts, purchase orders,or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricingand packaging for any new technologies or features discussed or presented have not been determined.

TA26

I/O Virtualization (IOV) For

Dummies

Mallik MahalingamStaff Engineer

Rich BrunnerChief Platform Architect

VMware, Inc.

8/3/2019 Io Virtualization

2/16

This session may contain product featuresthat are currently under development.

This session/overview of the new technologyrepresents no commitment from VMware to

deliver these features in any generally

available product.

Features are subject to change and must notbe included in contracts, purchase orders, or

sales agreements of any kind.

Technical feasibility and market demand willaffect final delivery.

Pricing and packaging for any newtechnologies or features discussed orpresented have not been determined.

8/3/2019 Io Virtualization

3/16

This session may contain product features that are currently under development. This session/overview ofthe new technology represents no commitment from VMware to deliver these features in any generallyavailable product. Features are subject to change and must not be included in contracts, purchase orders,or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricingand packaging for any new technologies or features discussed or presented have not been determined.

Motivation

Software based I/O Virtualization provides

I/O Sharing, Consolidation, Security, Isolation, Mobility

Simplified management

Transparent Teaming and Failover

High I/O performance is desirable for enterprise class

applications

Challenge: How to achieve I/O performance withoutcompromising virtualization benefits?

8/3/2019 Io Virtualization

4/16

This session may contain product features that are currently under development. This session/overview ofthe new technology represents no commitment from VMware to deliver these features in any generallyavailable product. Features are subject to change and must not be included in contracts, purchase orders,or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricingand packaging for any new technologies or features discussed or presented have not been determined.

Outline

VMware I/O Virtualization Architecture

Hardware/Software Assists for Virtualized I/O

Passthrough I/O

8/3/2019 Io Virtualization

5/16

This session may contain product features that are currently under development. This session/overview ofthe new technology represents no commitment from VMware to deliver these features in any generallyavailable product. Features are subject to change and must not be included in contracts, purchase orders,or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricingand packaging for any new technologies or features discussed or presented have not been determined.

I/O Virtualization Architecture

I/O Virtualization architectureconsists of

Guest driverVirtual device

Communication mechanism between virtualdevice and virtualization stack

Virtualization I/O stack

Physical device driver

Real device

Guest

Device Driver

PhysicalDevice Driver

I/O Stack

Physical Device

Device Emulation

Guest OS

8/3/2019 Io Virtualization

6/16

This session may contain product features that are currently under development. This session/overview ofthe new technology represents no commitment from VMware to deliver these features in any generallyavailable product. Features are subject to change and must not be included in contracts, purchase orders,or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricingand packaging for any new technologies or features discussed or presented have not been determined.

I/O Virtualization Architecture (contd.)

Virtual device

model a real device

e.g., Intel e1000, LSI mptscsi

model a simple virtualization friendly device

e.g., VMware vmxnet

Guest

Device Driver

PhysicalDevice Driver

I/O Stack

Physical Device

Device Emulation

Guest OS

8/3/2019 Io Virtualization

7/16

This session may contain product features that are currently under development. This session/overview ofthe new technology represents no commitment from VMware to deliver these features in any generallyavailable product. Features are subject to change and must not be included in contracts, purchase orders,or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricingand packaging for any new technologies or features discussed or presented have not been determined.

I/O Virtualization Architecture (contd.)

Virtualization I/O stack

translates guest I/O addresses to host addresses

handles inter VM communicationmultiplexes I/O requests from/to thephysical device

provides enterprise-class I/O features

Guest

Device Driver

PhysicalDevice Driver

I/O Stack

Physical Device

Device Emulation

Guest OS

8/3/2019 Io Virtualization

8/16

This session may contain product features that are currently under development. This session/overview ofthe new technology represents no commitment from VMware to deliver these features in any generallyavailable product. Features are subject to change and must not be included in contracts, purchase orders,or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricingand packaging for any new technologies or features discussed or presented have not been determined.

I/O Virtualization Implementations

PhysicalDevice Driver

I/O Stack

Guest OS

GuestDevice Driver

Virtual

Device

PhysicalDevice Driver

I/O Stack

Guest OS

GuestDevice Driver

VirtualDevice (VMM)

Virtual

Device

Host OS/Dom0/Parent Domain

Hosted or Split Hypervisor DirectPassthrough I/ OPassthrough I/ O

VMware Workstation, VMware Server,VMware ESX ServerMicrosoft Viridian & Virtual Server, Xen

VMware ESX Server(storage and network)

A Future Option

Virtualized I/ OVirtualized I / O

Guest OS

GuestDevice Driver

DeviceManager

8/3/2019 Io Virtualization

9/16

This session may contain product features that are currently under development. This session/overview ofthe new technology represents no commitment from VMware to deliver these features in any generallyavailable product. Features are subject to change and must not be included in contracts, purchase orders,or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricingand packaging for any new technologies or features discussed or presented have not been determined.

Benefits of Virtualized I/O

Emulated I/O decouples virtual HW from real HW.

Transparent move of VM between different physicalmachine.

Suspend/Resume

Live VM Migration

Transparent to VMs (OS, Apps, Virtual Devices)

Guest state is unaffected

Example: VMs open TCP connections to remain intact

VM to Virtual Device linking does not change.

Physical device and Virtual Device association canchange

Can be dynamic e.g Active NIC in a team of NICs

8/3/2019 Io Virtualization

10/16

This session may contain product features that are currently under development. This session/overview ofthe new technology represents no commitment from VMware to deliver these features in any generallyavailable product. Features are subject to change and must not be included in contracts, purchase orders,or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricingand packaging for any new technologies or features discussed or presented have not been determined.

Improving Virtualized I/O

Accelerate guest code

HW support for CPU and MMU virtualization

Support for improving device virtualization

e.g. virtual interrupt delivery

Simplified device models

Para-virtualized drivers

GuestDevice Driver

PhysicalDevice Driver

I/O Stack

Physical Device

Device Emulation

Guest OS

8/3/2019 Io Virtualization

11/16

This session may contain product features that are currently under development. This session/overview ofthe new technology represents no commitment from VMware to deliver these features in any generallyavailable product. Features are subject to change and must not be included in contracts, purchase orders,or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricingand packaging for any new technologies or features discussed or presented have not been determined.

Improving Virtualized I/O (contd.)

Acceleration of virtualization stack

Take advantage of multiple cores

MSI and MSI-X Interrupts

Dedicated DMA engine for memory-memorycopy

pNIC Acceleration

NetQueue: Multiple receive queues (MACaddress/VLAN) and interrupt per queue

Map receive buffers to guest memory -avoids a copy

GuestDevice Driver

PhysicalDevice Driver

I/O Stack

Physical Device

Device Emulation

Guest OS

8/3/2019 Io Virtualization

12/16

This session may contain product features that are currently under development. This session/overview ofthe new technology represents no commitment from VMware to deliver these features in any generallyavailable product. Features are subject to change and must not be included in contracts, purchase orders,or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricingand packaging for any new technologies or features discussed or presented have not been determined.

Passthrough I/O

Guest drives device directly

Use case: I/O Appliances, High performance VMs

I/O MMU

Device

Manager

VF VF VF

PF

PF = Physical Function, VF = Virtual Function

I/O Device

Guest OS

Device Driver

Guest OS

Device Driver

Guest OS

Device Driver

Virtualization

Layer

Requires:

I/O MMU for DMAAddress Translation

and protection (IntelVT-d, AMD I/O MMU)

Partitionable I/O

device for sharing (PCI-SIG IOV SR/MRspecification)

8/3/2019 Io Virtualization

13/16

This session may contain product features that are currently under development. This session/overview ofthe new technology represents no commitment from VMware to deliver these features in any generallyavailable product. Features are subject to change and must not be included in contracts, purchase orders,or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricingand packaging for any new technologies or features discussed or presented have not been determined.

Issues with Passthrough I/O

Guest Transparent VMotion

No simple way to checkpoint Passthrough device state

Hardware independenceNeed a driver for every Passthrough device that VM sees

HW errata/workarounds requires driver upgrade

Memory over-commitment

No visibility of DMAs to guest memory with Passthrough I/O

8/3/2019 Io Virtualization

14/16

This session may contain product features that are currently under development. This session/overview ofthe new technology represents no commitment from VMware to deliver these features in any generallyavailable product. Features are subject to change and must not be included in contracts, purchase orders,or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricingand packaging for any new technologies or features discussed or presented have not been determined.

Solutions for Passthrough I/O

VMotion

Guest Transparent

Virtualization Friendly Passthrough (vmxnet3)

Device provides an hypervisor interface to save and restore stateWorks only for Homogenous HW

Guest Aware

fail-over software combined with device Hot plug mechanism

2 Phase operation for VMotion (Ugly!)

Memory over-commitment

Device and IOMMU co-operation to enable demand paging

Define and develop APIs dynamic pinning/unpinning of DMA guest buffers

Integrate control interface with Virtual SwitchHardware based packet queuing and control via virtual switch

VMware wants to work with IHVs to help define these hardware assists.

8/3/2019 Io Virtualization

15/16

This session may contain product features that are currently under development. This session/overview ofthe new technology represents no commitment from VMware to deliver these features in any generallyavailable product. Features are subject to change and must not be included in contracts, purchase orders,or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricingand packaging for any new technologies or features discussed or presented have not been determined.

Device Class Directions

NIC

Physical Driver Acceleration for Emulated Model

Best candidate for Heterogeneous Pass-thru

iSCSI Software HBASame as NIC

Fiber Channel and SAS HBA

Physical Driver Acceleration for Emulated Model

Still researching Pass-thru solutions

Graphics

Better OpenGL support can help Emulated Model

Poster-child for single-device Fixed Pass-thruHW not yet defined for multiple logical contexts

Long, long way from Heterogeneous Mobile Pass-thru

8/3/2019 Io Virtualization

16/16