This document is for informational purposes only, and Tekelec reserves the right to change any aspect of the products, features or functionality described in this document without notice. Please contact Tekelec for additional information and updates.

Virtualizing Operating SystemsDr. Dorgham Sisalem

2 |

Agenda

› What is meant with Virtualization?

› Why bother?

› Terminology

› Classical approach? Why do not we use it?

› Virtualization methods Full Virtualization Paravirtualization Hardware-Assisted Virtualization

› Summary

› Slides under http://www.iptel.org/~dor/papers/Sisalem_VM.ppt

Classical Computer System

Machine

Operating System

User

Process

User

Process

User

Process

Physical hardware:

CPU, memory, IO devices

Operating System:

Mediator between user and machine

Distribute resources among processes

Access IO on behalf of processes

Only one OS at a time

Compiled only for a certain ISA

User applications:

Compiled only for certain OS on certain ISA

Each process believes to have access to all HW resources

Instruction Set Architecture

(ISA)

Classical Virtual System

Host Platform

Virtual Machine Monitor

(VMM)

Guest

OS

Guest

OS

Guest

OS

Host:

Physical hardware: CPU, memory, IO devices

Possibly operating system

VMM: Hypervisor

Mediator between guest and host

Distribute resources among guests

Access IO on behalf of guests

Works on a certain ISA

Guest:

Assumes certain ISA (and possibly OS)

Each guest believes to have access to all HW resources

and to be using its native ISA

Instruction Set Architecture

(ISA)

ISA ISAISA

UP

UP

UP

UP

UP

UP

UP

UP

UP

What is a Virtual Machine Monitor?

Classic Definition (Popek and Goldberg ’74) The VMM provides an environment for programs which is

essentially identical with the original machine Programs run in the environment show at worst only minor

decrease in speed The VMM is in control of system resources

Virtualization Benefits

Isolation Fault isolation Performance isolation

Portability Independent of physical hardware Enables migration of live, running VMs

Efficiency Idle resources used by multiple users and services

Improved scalability Add more resources for a customer

A short history of virtualization

Seventies: Save costs

Expensive Hardware

Support time-sharing

Multiple versions of OS

IBM implements virtualization

into its mainframes (VM/370)

Since 1998: Save costs, clouds

Virtualization to save costs by using the

same system for multiple users and OS

Cheap Hardware

High performance

Vmware, Xen, Parallels, Microsoft … offer

virtualization solutions

Eighties, nineties:

Virtualization considered

exotic technology

Moderately expensive Hardware

Moderate performance

Support windows over mac or old games

What‘s in it for the developer?

8 |

› Portable software must be tested on different platforms

› The Different platforms will remain idle most of the time

› Reduce costs, management and space needs

BSD Linux Unix

VMM

BSD Linux Unix

What‘s in it for hosting provider?

9 |

› Data centers with no Virtualization› Multiple customer applications

on the same OS› No isolation› Possible security

problems› Server/customer

› Idle resources› With Virtualization

› Each customer on a virtual machine

› Reduce costs, management and space needs

Customer1 Customer 2 Customer 3

VMM

Cust 1 Cust 2 Cust 3

What‘s in it for users?

10 |

Heterogeneous applications on the user’s host Windows application on mac Old games …

Offer the user a wider range of applications

Visio

(only for windows=

Keynote

(only for mac OS)Atari

Host

Visio Keynote Atari

VM Classification

11 |

VM

Process System

EmulationHigh Level

Language VM

(Java VM)

Hosted

Type 2

Bare HW

(Type 1)

Process vs. System VM

12 |

Host Platform

Virtual Machine Monitor

UP

Instruction Set Architecture

(ISA)

Machine

Operating System (Linux)

User process

(UP)

Windows application

Instruction Set Architecture

(ISA)

Runtime

Linux application

(ISA2)Runtime

Linux

UP

UP

UP

Win XP

UP

UP

UP

Win Vista

UP

UP

Process VM:

Enable a single process to run on the host

System VM:

Enable a complete system (OS + applications) to run on the host

Virtualizing Software

Process VM: Emulation

• Adapt from one OS to another• Adapt from one ISA to another• Virtualize the Application Binary Interface (ABI)

• User ISA• System calls to OS

• Virtualization software intercepts the application’s system calls and translates them into the hosts system calls

• If guest and host ISA are different then translate these as well

• Examples: Wine

13 |

Machine

Operating System

Guest

Instruction Set Architecture

(ISA)

ABI

Host OS

System VM: Hosted vs. Bare HW VMM

14 |

› Hosted VM VMM runs as a user process on top of the host OS Easy to install

• An additional application VMM can use IP device drivers of host OS VMM scheduled as user process

• No control on execution and resources Less efficient Examples: VM Workstation. Parallels ..

Machine

Virtual Machine Monitor

UP

Linux

UP

UP

UP

Win XP

UP

UP

Win Vista

UP

UP

Machine

Virtual Machine Monitor

User Process

(UP)

UP

Win XP

UP

UP

UP

Linux

UP

UP

› Bare HW VM VMM runs directly on the hardware Need to install the machine from scratch VMM offers the IP devices to the guests VMM controls which guest gets which

share and when Examples: VM ESX, Qemu, Xen

Virtualization: What are the Problems?

› Isolation: Make sure that one guest does not harm another Only VMM can use priviliged instructions

› Memory management: Each guest believes it has access to physical memory Map memory locations of guest to actual machine memory

› IO: All guests want access to disc, network, USB .... Schedule between guests and prevent one guest from disturbing

others• Multiple guests share the same NIC but might want to have different IP

addresses

15 |

Computer Architecture: CPU

16 |

Non-Privileged Instruction: User

User ISA: Non-Priviliged

CPU calculations

Memory read-write in user memory space

Privileged Instruction: Kernel Kernel ISA: Priviliged

IO access

Manage user memory space

Create/destroy processes

Context switch

Application

Machine

OS

Instruction Set Architecture

(ISA)

ABI

1. Run calculation on CPU

2. Raise system call for IO access

3. Kernel takes over and issues privileged instruction

1

2

3

Classical Virtualization Approach: CPU

17 |

Even less Privileged Instruction: User

User: Non-Privileged

CPU calculations

Memory read-write in user memory space

Non-Privileged Instructions: Kernel

Kernel: Non-Privileged

IO access

Manage user memory space

Create/destroy processes

Context switch

Privileged Instructions: VMM

VMM: Privileged

IO access

Manage Guest memory space

Create/destroy guests

Context switch between guests

Need at lest three levels of privileges

VMM

Classical Virtualization Approach: CPU

18 |

Application

Machine

Guest OS

ISA

ABI

1. Run calculation on CPU

2. Raise system call for IO access

3. Kernel takes over and issues privileged instruction but in non-privileged mode

4. Command will fail and generate a trap

5. The VMM catches the trap and

1. Checks if the guest is allowed to do the action

2. Issues the command on behalf of the guest

1

2

3

ISA

✗4

5

Computer Architecture: Memory

19 |

Logical page numbers (LPN):Process believes to have access

to all memory

Physical page numbers (PPN):OS maps LPN to a physical location

Process 1 Process 2

LPN

PPN

Classical Virtualization Approach: Memory

20 |

Logical page numbers (LPN)

Physical page numbers (PPN):•Each guest believes to have

access to the entire machine

physical memory

•No longer references physical memory

Machine page numbers (MPN):•Map physical pages to the machine

memory

Process 1 Process 1Process 2 Process 2

Virtual Machine 1 Virtual Machine 2

VMM

MPN

Shadow pages

PPNPPN

Shadow page numbers:•Map logical pages to the machine memory

•Reduces the effort for a two step lookup

•Increases the complexity as the pages must be kept in sync with the guests

Virtualizing X86

› X86 use the IA32 Instruction set

› Offers four levels of priviliges

› Is not virtualizable using the classical approach Some proviliged instructione when run in

non-proviliged mode fail silently• No trap!

21 |

Ring3: User

Ring2

Ring1: Guest OS

Ring0: VMM

Full Virtualization: Processor› Emulate x86 environment to the guest OS

Interpretation• Fetch one instruction at a time• Decode the instruction: get the registers and memory• Execute an Interpreter routine that provides the same functionality• Highly inefficient

1 instruction in OS can cause hundreds of instructions in VMM

• Sometimes used for process vitualization Binary Translation

• Translate source binary program to target binary before/during execution Go through the binary code and replace problematic instructions Generate compiled code

• Higher startup costs: VMM need to understand what is going on• Higher performance: number of additional instructions in the range of tens

Optimization:» Optimize frequently used parts» Cache compiled segments

22 |

Full Virtualitzation: Binary TranslationIntroduction to Virtual Machines, Carl Waldspurger

vEPC mov ebx, eax

cli

and ebx, ~0xfff

mov ebx, cr3

sti

ret

mov ebx, eax

mov [VIF], 0

and ebx, ~0xfff

mov [CO_ARG], ebx

call HANDLE_CR3

mov [VIF], 1

test [INT_PEND], 1

jne

call HANDLE_INTS

jmp HANDLE_RET

start

Guest Code Translation Cache

Full Virtualization: Memory Management

› Use shadow tables for memory management Block memory segments of Guests Guest access to memory segments results in a trap VMM updates shadow tables

› VMM needs to keep shadow and LPM in sync

24 |

Full Virtualization: IO Devices› IO access

Shared devices: Network card• Guest access to a virtual device is intercepted by the VMM• VMM conducts the actual access• Data received on the device cause an interrupt at the guest

OS• VMM needs to implement the device drives for all kinds of

devices Dedicated device: display, keyboard …

• Guest Bypasses VM: No need to virtualize the device• Not really that simple as some instance needs to translate

between guest and IO memory IO devices have their own memory IO devices can access machine memory directly

25 |

OS

Driver

Device

OS

Driver

Virtual device

Device

Driver

Device

Paravirtualization

› VMM offers the Guests a modified ISA Example: XEN

› Guest operating system is modified to use the VMM In Full Virtualization problematic parts are fixed by VMM In Paravirtualization problematic parts are fixed by developer of OS

• Use Hypercalls instead of the native OS calls

› Memory management Guest allocates the pages Read access is still directly without VMM involvement Updates to the pages afterwards is only through hypercalls VMM validates the Guest’s write/delete instructions Guest can apply changes in batches Mapping from logical to physical to machine addresses is the responsibility of the

Guest

26 |

Paravirtualization: IO Access› IO access

Add Paravirtual device drivers to the Guest VMM and Guest domain share an IO interface

• XEN uses a circular buffer • After placing one or more requests in the buffer the Guest

informs the VMM through a hyper call about the data No need for interrupts

27 |

OS

Paravirtual Driver

IO Interface

Driver

Device

Paravirtualization› Advantages

Higher performance than Full virtualization Guest still has access to some real resources (time and

machine addresses)

› Disadvantage Need to modify the Guest OS

Annoying even if <1% Changes are different for each OS

Paravirtual drivers must be implemented for each OS

28 |

Hardware Assisted Virtualization › Add an additional privilige level

Guest OS runs in non-Root level Instructions that required Emulation or

Paravirtualization cause a switch from non-root to root mode

› Hardware takes over the logical-2-physical-to-Machine mapping For every PPN access the hardware checks

also the MPN No need for shadow tables

› Better support for dedicated devices Hardware takes over the mapping of logical

address to device address• No need for virtual device drivers• No need for emulation or paravirtualization

29 |

Ring 2:

Ring 1

Ring 0 (non-Root): Guest OS

Ring 0 (root): VMM

Ring 0 : User

Summary› What are the benefits?

Isolation, portability, scalability

› What are the usage scenarios? Development, data centers, heterogenous applications

› What is the classical way of doing virtualization Trap and emulate

› Why doesn‘t the x86 allow for virtualization in the classical manner Not all insructions of the IA32 cause a trap

› What are the common virtualization techniques Full virtualization: Uses Binary translation, hides the machine but is complex and can be

slow Paravirtualization: Uses Hypercalls, is faster and simpler than full virtualization but requires

changes to the OS Hardware assisted: Remove some of the limitations of Full Virtualization

• Remember which?

30 |

References› Virtual Machines: Versatile Platforms For Systems And Processes, James E. Smith,

Ravi Nair, May 2005, ISBN 1-55860-910-5 “The Architecture of Virtual Machines". Smith, J. E.; Nair, R.(2005) Computer (IEEE Computer

Society)

› “Understanding Full Virtualization, Paravirtualization, and Hardware Assist”

› “A Comparison of Software and Hardware Techniques for x86 Virtualization”, K. Adams and O. Ageson. Proceedings of ASPLOS 2006, October 2006

› “Xen and the Art of Virtualization”, P. Barham et al. Proceedings of the ACM Symposium on Operating Systems Principles (SOSP), October 2003

› "Intel Virtualization Technology: Hardware Support for Efficient Processor Virtualization". G. Neiger et al.; Intel Intel Technology Journal 10 (3); 2006

› “Intel Virtualization Technology for Direct IO”, D. Abramson et al. ; Intel Technology Journal 10 (3); 2006

31 |

32 |

Thank you !!Thank you !!

Questions Questions

Dorgham SisalemDirector Strategic ArchitectureMobile: +49 171 304 2053E-mail: dorgham.sisalem@tekelec.com