1

THAWAN KOOBURATMICHAEL SWIFT

UNIVERSITY OF WISCONSIN - MADISON

The Best of Both Worlds with

On-Demand Virtualization

2

Motivation

Virtualization offers many benefits

Why virtualization has not yet become ubiquitous?

3

Concerns with Virtualization

Virtualization adds an extra layer that introduces overhead all the time

Lack of functionality GPUs, WLAN and etc.

Performance overhead I/O Devices - Trap and emulate Memory Virtualization – Extra level of page tables

4

On-Demand Virtualization

Mechanism Normally execute natively Virtualize on-demand Minimal service disruption

Benefits Native performance and full functionality Pay for virtualization overhead only when used

5

Uses of On-Demand Virtualization

Datacenter Resource Consolidation Debugging

Desktop Environment Migration Checkpointing Introspection

6

Main Ideas

High-level Design Pause OS operation and resume inside VM Set aside VMM partition on disk Virtualize everything

Physical

VMM

VMNative OSGuest OS

7

Problem: Need to transfer OS/Process state from native to virtual

executionSolution:

Hibernate/Resume can capture both OS and process state Snapshot system memory and write image to disk

\\

Capturing OS and Process State

Native Guest VMPhysical Virtualize

VMM

8

Capturing OS and Process States

Physical VMM

VMMPartition

GuestPartition

Native Kernel

HibernateBoot

Guest Kernel

Resume

VMMKernel

Start VM

9

Discovering Devices inside the VM

Problem: Need to switch from physical to virtual devices

Solution: Use Hotplug support in device

Detach/disable original physical devices before hibernate Scan for virtual devices after resume

\\

9

VMPhysical Virtualize

Disk

Native

NIC Disk

Guest

NIC

10

Problem: Need to preserve OS/Application state which ties to

devices Network connections, File systems

Need to transfer device bindings across different models/types of devices

Preserving Device Bindings

VMPhysical Virtualize

Disk

Kernel

NIC Disk

Kernel

NIC

Apps Apps

11

Preserving Device Bindings

Solution: Use logical device to retain states and hide changes

Networks: Bonding Driver Detach physical NIC and attach virtual NIC after resume

Block Devices: Device Mapper Update DM’s table to point to a virtual block device

Applications

Phy VM

e1000e e1000

bond0

KernelApplications

Phy VM

SATA IDE

/dev/mapper/vdisk

KernelNetwork Disk

12

The Cost of Preparation

Set aside VMM partition on disk No need to pre-allocate CPU or memory

Access hardware through logical devices Only for devices whose state must be preserved

Disable hardware that is not emulated by VMM Eg. IOMMU

13

Status

Status Prototype

Linux 2.6.35 (~1200 lines of modification) TuxOnIce (Hibernation patch) KVM

Support one-way conversion from physical to virtual machine

Experience SCP/SSH remains open Take about 90 seconds to convert

Hibernating and rebooting take most of the time

14

Future Work

Hibernate to RAMUse kexec for machine rebootSupport devirtualize

Transfer VM back to its original machine

15

Conclusion

Benefits Add another method of using virtualization Allow more environments to benefit from virtualization

without its overhead all the time

Contributions On-demand virtualization is possible Existing mechanisms in OS can be repurposed

16

Questions

Q/A

17

Other Devices

CPU Current: Offline other CPUs, only one core is left

running Future: CPU hotplug

Memory Current: VM must have equal or more RAM than

physical machine Future: Memory hotplug

Display VGA is supported by all graphic cards Revert to legacy VGA mode before hibernate

18

Dealing with Platform Devices

Platform devices Device on main board – i.e. timers, interrupt routers No hotplug support Kernel only configure them at boot time via ACPI

Solution Boot kernel pass configuration information to resume

kernel so that it can reconfigure its devices

\\VMBoot Kernel

Resumeconfig

\\VMGuest Kernel

config

HibernateImage

19

Conversion Time

Hibernate10 s

Reboot machine50 s

Boot VM9 s

Resume21 s

20

Overall Process

Hibernate and resume the kernel inside a virtual machine

Use hotplugging to detach and scan for new hardware

Reconfigure platform devices Update logical device mappings Resume OS subsystem/Applications