os virtualization (1)

8/10/2019 Os Virtualization (1)

1/107

1Ivan Boule

Virtualization

System Virtualizationand

OS Virtual Machines


2/107

2Ivan Boule

Virtualization

Plan

History

Virtualization Usages

Virtualization Taxonomy

Process Level Virtualization

Transparent Hardware Emulation

Transparent Hardware Virtualization

Paravirtualization

Hardware-Assisted Virtualization

Virtualization and Embedded Systems

Evolution of Virtualization


3/107

3Ivan Boule

Virtualization

History of Virtual Machines

VM introduced in the sixties on IBM/370 seriesCo-Designed VM: IBM AS/400

High level ISA including I/Os

Proprietary CISC PowerPCApplication VMs

Sun Java, Microsoft Common Language Infrastructure

OS VMsVMware (Windows/Linux on Intel)

Connectix (Windows/PC emulation on Mac OS)


4/1074Ivan Boule

Virtualization

Plan

History






Paravirtualization





5/1075Ivan Boule

Virtualization

Goals of System Virtualization

Reduction of Total Cost of Ownership (TCO)

Increase utilisation of server resources

Reduction of Total Cost of Functioning

Energy consumption

Cooling

Occupied Space

Hardware ConsolidationReduction of Build Of Material (BOM) for high-volume

low-end products


6/107

Virtualization in high-throughput networkequipments

6


7/1077

Virtualization in Multimedia devices

Reduction of Build Of Material (BOM) for high-volume low-end

products

No need for a General Purpose Processor

~ 20 to 25 % BOM reduction

Run Linux together with OS supporting

Codecs on a single TI DSP

Leverage Linux environment

Reuse existing DSP software

Vi t li ti


8/1078Ivan Boule

Virtualization

Usages of Virtual Machines

Server virtualization

Web sites hosting

OS fault recovery

OS kernel development

Test machine = development host

OS/kernel education & training

Keep backward compatibility of legacy software

Hardware no more available

Run applications not supported by host OS

Virtualization


9/1079Ivan Boule

Virtualization

Recovery Servers

Virtualization


10/10710Ivan Boule

Virtualization

Multi-Core CPU Issues (1)

CPU power gain

No more achieved through Frequency/Speed increase

But obtained with higher density & multi-core chips

Many RTOS designed with mono-processor assumption

Adding multi-processor support is complex & costly

Scaling requires time, at best...

Legacy RT applications also designed for mono-processor

Adaptation to multi-pro even more difficult than RTOS

Virtualization


11/107

11Ivan Boule

Virtualization

Multi-Core CPU Issues (2)

OS virtualization allows to run simultaneously on

a multi-cores CPU multiple [instances of] mono-

processor OS's

Each OS instance is run in a [mono-processor]

Virtual Machine assigned to a single CPU core

No need to change legacy software

Scalability managed at virtualization level

Virtualization


12/107

12Ivan Boule

Virtualization

Plan

History






Paravirtualization




Virtualization


13/107

13Ivan Boule

Virtualization

System Virtualization Principles

Run multiple OS's on the same machine

By design, an OS assumes to have full control

over all physical resources of the machine

Manage sharing/partitioning of machine

resources between Guest OS's

CPU

Physical memory & MMU

I/O devices


14/107

14

Machine Interfaces

Hardware

OS

App App App

System Calls User ISA

System ISAISA

Hardware

OS

App App App

System Calls User ISA

System ISAABI

ISA = Instruction Set Architecture

System level interfaceAll CPU instructions, memory architecture, I/O

ABI = Application Binary InterfaceProcess level interface

User-level non privileged ISA instructions + OS systems calls

Virtualization


15/107

15Ivan Boule


Process level virtualization

Emulation of Operating System ABI

Virtual Servers

System level virtualization

Standalone / Hosted Virtualization

Machine Emulation / Machine Virtualization

Virtualization


16/107

16Ivan Boule

Hosted versus Standalone

Virtualization

Hosted Virtualization

Hosted VM Monitor (VMM) runs on top of native OS

VMware WKS, Microsoft VirtualPC, QEMU, UML

Standalone Virtualization

VMM directly runs on bare hardware

VMware ESX, IBM/VM, Xen, VLX, KVM

OS run in a VM is named a Guest OS

Virtualization


17/107

17Ivan Boule

Hosted Virtualization

Hardware

Native OS

VMM

Guest OS

Applications

VMM

Guest OS

Applications

VMM

Guest OS

Applications

Virtualization


18/107

18Ivan Boule

Example : VMware Workstation

Hosted VM

Unmodified OSes

Specific devicedrivers

X86 only

Guest OS executed

in user mode

Virtualization


19/107

19Ivan Boule

Standalone Virtualization

Hardware

VMM

Guest OS

Applications

Guest OS

Applications

Guest OS

Applications

Virtualization

VM ESX


20/107

20Ivan Boule

VMware ESX

Standalone VM

Supports unmodified OS binaries

Configuration with appropriate device drivers

X86 only

OS hosted

in user mode

Virtualization


21/107

21Ivan Boule

Plan

History






Paravirtualization




Virtualization


22/107

22Ivan Boule

Process level ABI Emulation

Goal: execute binary applications of a given

system Xon the ABI of another system Y

Emulate system XABI on top of system YABIEmulation done by application-level code

System Ymust provide services equivalent to

those of system X(file system, sockets, etc.)

Virtualization


23/107

23Ivan Boule

Process Level (ABI) Emulators

Wine - Windows Emulator on Unix/LinuxWindows API in userland

Adobe Potosop! "oo#le Pi$asa! %%%

&'#winUnix emulation on Windows

POSI( librar'

)as sell * man' Unix $ommands"+U de,elopment tool $ain #$$! #db.

X Window, GNOME, Apache, sshd, ...

Virtualization


24/107

24Ivan Boule

Virtual Ser,ers

Single OS kernel / Multiple resource instancesIsolated kernel execution environments

Root file system

IP tablesProcess for signals

Solaris 10 Containers

Linux-VServer

FreeBSD Jail

Virtualization


25/107

25Ivan Boule

Virtual Ser,ers

Kernel Code

P7

P5

P6

P3 P8 P9

/roots/vm1

10.16.0.0/16

P5 P6

/roots/vm2

10.17.0.0/16

P7

/roots/vm3

10.18.0.0/16

P3

P8 P9

/

74.125.0.0/16

P1 P2

P1

P2

Virtualization


26/107

26Ivan Boule

Virtual Servers

Pro's

CPU independent

Lightweight

Low memory footprint

Low CPU overhead

Scalable

&ons

+o OS etero#eneit' no "POS/01OS $ombination.Sin#le OS binar' instan$e $ommon point o2 2ailure.

Intrusi,e3 must modi2' OS 4 2ollow OS updates

Virtualization

Plan


27/107

27Ivan Boule

Plan

History






Paravirtualization




Virtualization


28/107

28Ivan Boule


0un unmodi2ied OS binaries

Includes emulation of physical devices

Cross ISA EmulationQEMU

Same ISA Emulation

VirtualBox (Intel x86)

Virtualization


29/107

29Ivan Boule


Emulate machine X on top of machine Y

Interpretation

1 instruction of X executed by N instructions of Y

Huge slow down method (1/1000 if X Y)

Dynamic Binary Translation

Convert blocs of X instructions in Y instructions

Application-level emulator runs on a native OS

One VM running a single Guest OS

Virtualization

567U A it t


30/107

30Ivan Boule

3

567U Ar$ite$ture

Solaris (Native OS)

QEMU

PowerPC

RT-OS

Real-Time

Applications

QEMU

PC-x86

Linux

LinuxApplications

Solaris ABI

QEMU

PC-x86

Windows

Windows

Applications

Sun - Sparc

PowerPC ISA x86 ISA x86 ISA

Solaris

Process


31/107

Virtualization

Plan


32/107

32Ivan Boule

Plan

History






Paravirtualization




Virtualization


33/107

33Ivan Boule


Share machine resources among multiple VMs

Execute native/unmodified OS binary images

Provide in each VM a complete simulation of

hardware

Full CPU instruction set

Interrupts, exceptions

Memory access and MMU

I/O devices

Virtualization


34/107

34Ivan Boule

Full CPU Virtualization

Present same functional CPU to all Guest OSes

VMM manages a CPU context for each VM

saved copy of CPU registers

representation of software-emulated CPU context

VMM shares physical CPUs among all VMs

VMM includes a VM scheduler

round-robin

priority-based

Virtualization


35/107

35Ivan Boule

Full CPU Virtualization

Relationships between a VMM and VMs similar to

relationships between native OS and applications

Guarantee mutual isolation between all VMs

Protect VMM from all VMs

Directly execute native binary images of Guest

OS's in non-privileged mode

VMM emulates access to protected resources

performed by Guest OSs

Virtualization

CPU Virtualization


36/107

36Ivan Boule

CPU Virtualization

Run each Guest OS in non-privileged modeEx: ring compression on Intel x86

Applications

Guest OS

Applications

Guest OS

Applications

Guest OS

VM0 VM1 VM2

Virtual Machine Monitor (VMM)Ring 0

Ring 1

Ring 3

Virtualization


37/107

37Ivan Boule

Hardware-Sensitive Instructions

Interact with protected hardware resourcesPrivileged Instructions

Critical Instructions

Cannot be directly executed by Guest OS's

Must be detected and faked by VMM

Dynamic Binary Translation of kernel code

Done once, saved in Translation Cache

Example: Vmware

Virtualization

Privileged Instructions Virtualization


38/107

38Ivan Boule

Privileged Instructions Virtualization

Only allowed in supervisor mode

Ex: cli/stito mask/unmask interrupts on Intel x86

When executed in non-privileged mode

CPU automatically detects a privilege violation

Triggers a privilege-violation exception

Caught by VMM which fakes the expected effect

of the privileged instruction

Ex: cli/sti

VMM does not mask/unmask CPU interrupts

records interrupt mask status in context of VM

Virtualization

Critical Instr ctions Virt ali ation (1)


39/107

39Ivan Boule

Critical Instructions Virtualization (1)Hardware-sensitive instructions

Ex: Intel IA-32pushf/popf

pushf /* save EFLAG reg. to stack */

cli /* mask interrupts => clear EFLAG.IF */

popf /* restore EFLAG reg. => unmask interrupts */

When executed in non-privileged mode

The cliinstruction triggers an exception caught by VMM=> VMM record interrupts masked for current VM

But no exception for popf=> VMM not aware of Guest

OS action (unmask interrupts)

Virtualization

C iti l I t ti Vi t li ti (2)


40/107

40Ivan Boule

Critical Instructions Virtualization (2)

Must be detected and emulated by VMMVMM dynamically analyses Guest OS binary code

to find critical instructions

VMM replaces critical instructions by a trap instruction to enter the VMM

VMM emulates expected effect of critical

instruction, if any.

Ex: pushf/popf combined with cli/stiinstructions

Virtualization

Full Memory Virtualization


41/107

41Ivan Boule

Full Memory Virtualization

CPU include a Memory Management Unit (MMU)Isolated memory addressing spaces

Independant of underlying physical memory layout

Run mutually protected applications in parallelVirtual Memory managed by OS kernel

Provides a virtual address space to each process

4 GB on most 32-bit architectures (Intel x86, PowerPC)

Manages virtual page physical case mappings

Manages swap space to extend physical memory

Virtualization

77U 4 Virtual Address Spa$e


42/107

42Ivan Boule

1ranslationLoo8aside

)u22er

77U 4 Virtual Address Spa$e

Virtual Address Spa$es

P'si$al7emor'

77U

case N

pte4GB

page

case 0

Virtualization

Intel x9: 77U


43/107

43Ivan Boule

Intel x9: 77U

;< >>

DirectoryIndex

Page Table

Index

PageOffset

cr/st

1023

cr/st

0

1023

10 bits 10 bits 12 bitsVirtual

Address

Directory Page

CR3

Directory Address

Physical Memory

32-bit word

cr/st =control & status

Page Table Entry (PTE)cr/st

4KB page

Translation Lookaside Buffer (TLB) = cache for PTEs

0

Virtualization

Memory Virtualization


44/107

44Ivan Boule


Machine Physical Memory

Physical memory available on the machine

Guest OS Physical Memory

Part of machine memory assigned to a VM by VMM

Guest Physical Memory can be > Machine Memory

VMM uses swap space

Guest OS Virtual Memory

Guest OS manages virtual address spaces of its

processes

Virtualization



45/107

45Ivan Boule


Guest OS manages Guest Physical PagesManages MMU with its own page entries

Translates Virtual Addresses into Guest Physical

Addresses (GPA)VMM transparently manages Machine Physical

Pages

Guest Physical AddressMachine Physical Address

VMM dynamically translates Guest Physical Pages

into Machine Physical Pages

Virtualization



46/107

46Ivan Boule


2000

6000

1000

4000

7000

1000

5000

1000

8000

10003000

unmapped virtual page

0

3000

7000

unmapped Guest page

mapped virtual page

machine

memory

Process

virtual

space

VM 2

Guest

Physical

Memorymapped Guest page

VM 1

P1.1 P1.2 P2.1

Machine physical page

Virtualization



47/107

47Ivan Boule


VMM maintains Shadow Page Tables

Copies of Guest OS translation tables

VMM catches updates operations of translation

tables performed by a Guest OS

Write-protect all guest OS page tables

Emulates operation in shadow page table

Updates effective MMU page table entry, if needed

Virtualization



48/107

48Ivan Boule


PTE entries can be tagged with a context ID

Avoids to flush TLB when switching current address

space upon scheduling of a new process

usually PTE tag = OS process identifier

Processes of different Guest OSes can be

assigned the same Process ID

=> VMM must flush TLB when switching VMs

Virtualization



49/107

49Ivan Boule


VMM must respect Guest OS virtual page faults

Not map virtual pages unmapped by Guest OS

When Guest OS unmaps a virtual page:

VMM must delete the associated real-page/physical page

mapping, if any.

Conversely, VMM can transparently:

Introduce & resolve real-page faults for Guest OSes

Share physical pages between Guest OS's

Pages with same content's (e.g. zero-ed pages)

Virtualization



50/107

50Ivan Boule


VMM can swap real pages of a VMon swap space managed by VMM

VMM can dynamically distribute physical memory

among VM'sNeeds a specific support in Guest OS (Linux module)

VMM asks Guest OS to release memory

Guest OS self-allocates [real] pages=> no more available for normal [kernel] allocation service

VMM assigns same amount of physical pages to other VM's

Virtualization

Plan


51/107

51Ivan Boule

History






Paravirtualization




Virtualization

Paravirtualization


52/107

52Ivan Boule

Paravirtualization

OS adaptation to avoid binary translationoverhead

Requires access to OS source code

Include drivers of virtual devices

Examples

Xen

User Mode Linux (UML)

Virtualization

Paravirtualization Principles


53/107

53Ivan Boule

Still run each Guest OS in non-privileged mode

But with minimal virtualization overhead

=> Modified Guest OS kernel

Remove Hardware-Sensitive InstructionsUse fast VMM system calls instead, if needed

Minimise usage of Privileged Instructions

Only affect Machine/CPU dependant part of OSOS portage on new architecture with same CPU

Without system ISA

Virtualization

Paravirtualization (2)


54/107

54Ivan Boule

( )

Guest OS only use Virtual I/O Devices

Front-end driver in Guest OS

Back-end driver in VMM

Data transfer through asynchronous I/O rings

Avoid extra I/O data copies of Full Virtualization

VMM multiplex VM Virtual Devices on physical

devicesVirtual Ethernet

Virtual Disks

Virtualization

Virtual I/O ?e,i$es


55/107

55Ivan Boule

Ethernet NIC

Vdisk

(be)

Applications

Guest OS

Vdisk

(fe)Veth

(fe)

Veth

(be)

Applications

Guest OS

Vdisk

(fe)Veth

(fe)

Applications

Guest OS

Vdisk

(fe)Veth

(fe)

NIC

driver

Disk

driver

Veth

(be)

Veth

(be)

Vdisk

(be)

Vdisk

(be)

Net Bridging

VMM


56/107

Virtualization

Plan


57/107

57Ivan Boule

History

Virtualization UsagesVirtualization Taxonomy




Paravirtualization




Virtualization

Hardware Assisted Virtualization


58/107

58Ivan Boule


Support o2 Virtualization in Hardware

Run unmodified OS binaries

Wit minimal ,irtualization o,eread

Simpli2' V77 de,elopment

6xamples

@V7 Intel-V1! A7?-V.V7ware Intel-V1.

Virtualization



59/107

59Ivan Boule

CPU virtualizationAMD-V

Intel VT-x (x86), Intel VT-i (Itanium) architectures

ARM Cortex-A15MMU virtualization

Intel Extended Page Tables (EPT)

A7? Nested Pa#e Tables +P1.

Virtualization



60/107

60Ivan Boule


DMA virtualizationIO-MMU

I/O Device virtualization

Self-Virtualizing devices

SingleRoot I/OVirtualization and Sharing

Specification (SR-IOV)

Extensions to PCIe (PCI Express) Busstandard

Virtualization

Intel V1-x Ar$ite$ture


61/107

61Ivan Boule

e $ e$ u e

Support unmodi2ied "uest OS wit no need 2orpara,irtualization and/or binar' $ode translation

Simpli2' V77 tas8s 4 impro,e V77 per2orman$es

7inimize V77 memor' 2ootprintSuppress sadowin# o2 "uest OS pa#e tables

6nable "uest OS to dire$tl' mana#e I/O de,i$es

Witout per2orman$e lost

Wile en2or$in# V7 isolation and mutual prote$tion

Virtualization

Intel V1-x Ar$ite$ture O,er,iew


62/107

62Ivan Boule

VMM

VMX root mode

VMX non-root mode

Intel-VT Hardware

VM Exit VM Enter

Guest OS

kernel

Applicationsring 3

ring 0 Guest OS

kernel

Applicationsring 3

ring 0Guest OS

kernel

Applicationsring 3

ring 0

VM 1 VM 3VM 2

rings 0 - 3

Virtualization

Intel VT-x CPU Virtualization


63/107

63Ivan Boule

Virtual Ma$ine eXtension V7(.

1wo new meta-modes o2 &PU operationV7( root mode

)ea,iour similar to IA-=< witout V1

Intended 2or V77 exe$ution

V7( non-root mode

Alternati,e IA-=< exe$ution en,ironment

&ontrolled b' a V77

?esi#ned to run un$an#ed "uest OS in a V7

)ot modes support rin#s ;-= pri,ile#e le,els

Allow V77 to use se,eral pri,ile#e le,els

Virtualization



64/107

64Ivan Boule

1wo additional &PU mode transitions

rom V7( root-mode to V7( non-root mode

+amed VM Enter

rom V7( non-root mode to V7( root mode

+amed VM Exit

V7 entries 4 V7 exits use a new data stru$ture

Virtual Ma$ine Control Stru$ture V7&S. per V7

0e2eren$ed wit a memor' p'si$al addressormat and la'out idden

+ew V1-x instru$tions to a$$ess a V7&S

Virtualization



65/107

65Ivan Boule

"uest State Area

Sa,ed ,alue o2 re#isters loaded b' V7 6xitse%#%! Se#ment 0e#isters! &0=! I?10.

Hidden &PU state e%#%!CPU Interruptibility State.

Host State Area

V7 &ontrol ields

Interrupt Virtualization

6x$eptions bitmaps

I/O bitmaps7odel Spe$i2i$ 0e#ister 0/W bitmaps

6xe$ution ri#ts 2or &PU Pri,ile#ed Instru$tions

Virtualization

Intel VT-x Interrupt Virtualization


66/107

66Ivan Boule

V7&S 6xternal Interrupt 6xitin#All external interrupts $ause VM Exits

"uest OS $annot mas8 external interrupts

wen exe$utin# Interrupt 7as8in# instru$tionsV7&S Interrupt Window 6xitin#

VM Exito$$urs wene,er "uest OS read' toser,e external interrupts

Used b' V77 to $ontrol V7 interrupts

Virtualization

Intel VT-x MMU Virtualization


67/107

67Ivan Boule

Extended Pa#e Tables 6P1.Se$ond le,el o2 Pa#e 1ables in 77U

1ranslate "uest OS P'si$al Address into7a$ine P'si$al Address

&ontrolled b' V77

Virtual Pro$essor IDenti2ier VPI?.

Used to ta# 1L) entries

A,oid to 2lus 1L) upon V7 swit$

Virtualization

Virtual 7emor' Virtualization


68/107

68Ivan Boule

'

Process 1 Process 2 Process 1 Process 2VM 1 VM 2

Guest OS

Virtual Memory

Guest OS

Physical Memory

Machine

Physical Memory

Virtualization

Intel VT-x Extended Page Tables


69/107

69Ivan Boule

V77 $ontrols 6xtended Pa#e 1ables

6P1 used in V7( non-root operation

A$ti,ated on VM Enter

?esa$ti,ated on VM exit

6P1P re#ister points to 6xtended Pa#e 1ables

Instan$iated b' V77

Sa,ed in V7&S

Loaded 2rom V7&S onVM entry

Virtualization

Intel VT-x Extended Page Tables


70/107

70Ivan Boule

Translation Lookaside Buffer (TLB)

Guest PTEs (Guest VA Guest PA)

Extended PTEs (Guest PA Host PA)

Guest

Page Table

EPT

Page Table

Guest CR3

Guest VA Guest PA Machine PA

EPTR Base Pointer

Virtualization

1L) lus Issue


71/107

71Ivan Boule

1ranslationLoo8aside

)u22er

Virtual Address Spa$es

P'si$al7emor'

77U

case N

pte

4GB

case 0

Virtualization

Intel VT-x Virtual Processor Identifier


72/107

72Ivan Boule

>:-bit VPI? used to ta# 1L) entries

6nabled b' V77 in V7&S

UniBue VPI? is assi#ned b' V77 to ea$ V7

VPI? ; reser,ed 2or V77

&urrent VPI? is ;x;;;; wen

Outside V7( operation

In V7( root mode operation

In V7( non-root mode i2 VPI? disabled in V7&S

VPI? loaded 2rom V7&S on VM Enter

Virtualization

?7A Virtualization


73/107

73Ivan Boule

6nable "uest OS to mana#e I/O de,i$es

I/O de,i$es assi#ned b' V77 to "uest OSes

1ransparent mode

Use nati,e de,i$e dri,er o2 "uest OS

Unaware o2 p'si$al memor' Virtualization

6n2or$e isolation between "uest Oses

"uest OS onl' ,iew ardware ressour$esassi#ned b' V77 memor'! de,i$es.

Virtualization

?7A Prin$iples


74/107

74Ivan Boule

CPUBus

Controller

Memory

Device 1 Device 2 Device 3

DMA

Request

System

Bus

I/O Bus (PCI)

Virtualization

?7A Virtualization


75/107

75Ivan Boule

Device1

Applications

Guest OS

Applications

Guest OS

Applications

Guest OS

Device2

VM 1 VM 3VM 2

Isolation

Domain

VMM

Machine Physical

Memory

Virtualization

?7A Virtualization Issue


76/107

76Ivan Boule

"uest OS dri,er setup I/O re#isters o2

de,i$e wit "uest P'si$al Address o2I/O bu22ers

"uest P'si$al Address must be

translated into its $orrespondin#7a$ine P'si$al Address wen used 2or?7A operations b' de,i$e

"PA 1ranslation $annot be done b' V77V77 $annot $at$ de,i$e-spe$i2i$ dri,eroperations to setup I/O bu22ers addresses


77/107

Virtualization

Intel V1-d ?7A 1ranslation


78/107

78Ivan Boule

V1-d ardware treats address spe$i2ied in

?7A reBuest as D7AVirtual Address ?VA.

?VA C "PA o2 te V7 to wi$ te I/Ode,i$e is assi#ned

V1-d translates te ?VA into its$orrespondin# 7a$ine P'si$al Address

Support o2 multiple Prote$tion ?omains

?VA to 7PA translation table per Prote$tion?omain

7ust identi2' te de,i$e issuin# a ?7A reBuest

Virtualization

V1-d P&I 6xpress +ort )rid#e


79/107

79Ivan Boule

CPUMemory

Device 1 Device 2 Device 3

System

Bus

PCI Express Bus

North Bridge

VT-d

PCIe root ports

Virtualization

P&I ?7A 0eBuester Identi2i$ation


80/107

80Ivan Boule

7appin# between P&I ?e,i$e and Prote$tion

?omains>:-bit P&I ?7A 0eBuester Identi2ier

Assi#ned b' P&I $on2i#uration so2tware

)us D indexes )us &ontext 1able in 0oot

&ontext 1able?e,i$e D! un$tion D. indexes ?e,i$eProte$tion ?omain in )us &ontext 1able

15 8 7 3 2 0

PCI Bus # Device # Function #

Virtualization

?e,i$e / Prote$tion ?omain 7appin#

(Dev 0 Func 0)


81/107

81Ivan Boule

Bus 255

Bus N

Bus 0Context Table of Bus 0

(Dev 0, Func 0)

(Dev 0, Func 1)

(Dev 31, Func 7)

Root Context TableContext Table of Bus N

(Dev 0, Func 0)

(Dev 0, Func 1)

(Dev 31, Func 7)

Protection Domain 0

Protection Domain 1

VDA MPATranslation

Tables

VDA MPA

Translation

Tables

Virtualization

Virtual ?7A Address 1ranslation


82/107

82Ivan Boule

V?A E 7PA V1-d Pa#e 1ables similar toIA-=< pro$essor Pa#e 1ables

F@) or lar#er pa#e size #ranularit'

0ead/Write permissions

Prote$tion ?omains mana#ed b' V77

Initialized at V7 $reation time

Wit same translations o2 te V7 6xtendedPa#e 1able

Virtualization

?e,i$e Virtualization


83/107

83Ivan Boule

Sare I/O de,i$e amon# multiples V7sWit no per2orman$e lost

Wile en2or$in# V7 isolation and prote$tion

7o,e de,i$e ,irtualization 2rom te V77to te de,i$e itsel2

0eBuires support 2rom te de,i$e

6xample o2 6ternet $ontrollers

Virtualization

6ternet ?e,i$e Virtualization

VM 1 VM 2


84/107

84Ivan Boule

VM 1 VM 2

VMM

GuestOS 1 GuestOS 2

pNIC Driver

vNIC Driver vNIC Driver

Virtual

Function

Virtual

Function

LAN

Virtualization

Intel Sin#le Root I/OVirtualization


85/107

85Ivan Boule

S0-IOV $apable P&I ?e,i$e $an bepartitionned into multiple Virtual un$tions

S0-IOV ?e,i$e appears in P&I $on2i#urationspa$e as multiple P&I Virtual un$tions

6a$ ?e,i$e Virtual un$tion in$ludes

P&I $on2i#uration re#isters

?7A streams

Interrupts

0eBuires V1-d 2or ?7A ,irtualization

Virtualization

Intel S0-IOV

V77 i l P&I d i


86/107

86Ivan Boule

V77 mana#es p'si$al P&I de,i$e

&reate a P&I Virtual un$tion 2or ea$ V7

In$lude it into V7 P&I $on2i#uration spa$e tobe probed b' V7 "uestOS 8ernel

7ap it to Prote$tion ?omain o2 V7

Pro#rams te sarin# o2 p'si$al de,i$esressour$es between Vs

P&I ?e,i$e Virtual un$tions dire$tl'mana#ed b' spe$i2i$ V-Aware "uestOSdri,ers 8ind o2 Para-Virtualization.

Virtualization

Intel S0-IOVVM 1 VM 2 VM n


87/107

87Ivan Boule

MAC/PHY

Layer 2 Filtering (Mac Address)

OS vNIC

Driver

Packet

Queue 1

VF1

Packet

Queue 2

VF2

Packet

Queue n

VFn

OS vNIC

Driver

VMM

OS vNIC

Driver

Intel Niantic

10GbE Port

LAN

Virtualization

Intel S0-IOV - 6ternet example


88/107

88

Ivan Boule

Intel @awela >"). / +ianti$ >;"). 6ternet +I&s

7ultiple 0(/1( pa$8et Bueues per port

Virtual De,i$e Ma$ine Queues

> 0( paBuet Bueue per V

ilters multiple uni$ast 6ternet Addresses

La'er-< paBuet 2ilterin# based on 6ternet?estination Address

?upli$ate )road$ast / 7ulti$ast pa$8ets 2or all Vs

Load balan$in# between 1( paBuets sent b' Vs

Virtualization

PlanHistory


89/107

89

Ivan Boule




Transparent Hardware EmulationTransparent Hardware Virtualization

Paravirtualization




Virtualization

Old Embedded Systems


90/107

90

Ivan Boule

Relatively simple architectureSingle-purpose devices

Dominated by hardware constraints

Memory, battery charge

Dedicated functionalities, with moderated

software size and complexity

Real-time constraints

Virtualization

Old Embedded Systems (2)


91/107

91Ivan Boule

Closed environment ( black boxes )Fixed hardware configuration

Full software provided by device vendor

No dynamic loading of applications

Software updates rareful

Virtualization

Embedded Systems Now (1)

Take on features of general purpose OS's


92/107

92Ivan Boule

Take on features of general-purpose OS s

Growing functionalities

=> growing complexity and size

Run applications originally developed for PC's

Sophisticated Human Machine Interfaces (HMI)

Safari Web browser on iPhones

Dynamic loading of applications

Iphone

Google Android

Virtualization

Embedded Systems Now (2)

Dynamically load device's owner specific applications


93/107

93Ivan Boule

Dynamically load device s owner specific applications

Games

Applications developped by engineers with no expertise

in embedded systems

Java applicationsNeed for exchanges with external world

USB, Bluetooth, Wi-Fi

TCP/IPNeed for open API's, and openness in general

Need for high-level systems (Linux, Windows)

Virtualization

Embedded Systems Challenges

Still Real-Time systems (part of it)


94/107

94Ivan Boule

Still Real Time systems (part of it)

Baseband stack of mobile phones

Still hardware constraints

Battery

Memory (to minimize device's cost)

Also used in mission/life critical situations

Weapons

Cars

High requirements on reliability and security

Mobile HandsetsRun Android/Linux

applications on baseband


95/107

95

OS Virtualization La'er

7odem

01OS

Wireless Sta$8s3

"S7/"P0S

6d#e/U71S

1elepon'Ser,i$es

Appli$ations

Android/Linux

H7I

PI7

Internet

"ames

7ultimedia

Pone HW Sin#le &PU.

ppprocessor

Re-use existing legacymodem software stack withits RTOS (no changes)

Support of Linux at a

minimal development costOperating System

independence for futureevolutions

Security & Protection

through OS isolation

HMI: Human-Machine-InterfacePIM: Personal Information Mngt.

Virtualization

Virtualization in Embedded Systems

Support for heterogeneous OS's environments


96/107

96Ivan Boule

Real-time OS

Legacy software

Dedicated applications whose real-time constraints

cannot be achieved by General-Purpose systemsLicence issues ( GPL contamination )

General Purpose OS

Openness

HMI

Virtualization


Concurrent execution of RTOS and GP OS on


97/107

97Ivan Boule

Concurrent execution of RTOS and GP-OS on

the same CPU

Reduces cost (Bill Of Material)

Requires the underlying VMM to provideMemory isolation between OS's

CPU scheduling among OS's, with higher priority to

the RTOS

Device partitionning

Communication mechanism between OS's

Virtualization


Leverage multi-cores support with virtual


98/107

98Ivan Boule

ag s s pp a

machine abstraction

1 core per OS => no need for CPU scheduling

2 low-performance cores consume less power

than a single high performance CPU=> simplify power management

New model of software distribution, shipping

application with its own OSNo OS configuration/version incoherency

Virtualization

Security Through Virtualization


99/107

99Ivan Boule

Notion of Trusted Computing Base (TCB)Part of the system that provides security foundations

Should only include hardware and VMM

May also include RTOS, for performance/legacyreasons

Run GP OS in an isolated Virtual Machine

Avoid damaged GP OS to compromise the secure

parts (data, services) of the system

Virtualization

Embedded + Virtualization

Challenges (1)


100/107

100Ivan Boule

g ( )

ull isolation o2 V7s does not 2it$ooperation reBuirements between OSs

622i$ient $ommuni$ation me$anismsbetween V7s

"lobal s$edulin#! wit interlea,edpriorities

"lobal 6ner#' 7ana#ement

Virtualization

Embedded + VirtualizationChallenges (2)


101/107

101Ivan Boule

622i$ient $ommuni$ation me$anisms betweenV7s

Virtual Ethernet device not adapted

Need VMM-controlled shared memory transfers

Example: Video streaming on a Smartphone

Video data received via the baseband managed by RTOS

Video data displayed by a Media Player running on GPOS

Avoid copy of video data transfered between the 2 OS's !

Virtualization

Task Scheduling Issues

Standard server-oriented Virtualization model


102/107

102Ivan Boule

The VMM schedules VM's on the CPU

The OS on each VM runs its own scheduler

Interleaved priorities in Embedded Systems

Baseband task of RTOS with a high priority

But GPOS Media-Player must have a higher

priority than some low-priority tasks of RTOS

Enable a VM to yield the CPUUse a RT task as a proxy of GP OS application, and

make it yield the CPU

Virtualization

Multi-Users Devices

Mobile phone has 3 types of users, each with


103/107

103Ivan Boule

specific private data to protect from the othersThe person owning the device, with address book,

emails, documents, etc.

Different wireless providers, for example private

and professionnal: network access properlyauthenticated, ensure correct billing !

Third-party service providers, for instance

multimedia providers.

Owner and third-parties must be granted

secure financial transactions

Virtualization

Virtualization in Hardware

Only way to build a real TCB


104/107

104Ivan Boule

Only way to build a real TCB

Without penalizing performances

Should include support for

Memory Partitionning

Physical Memory / Machine Memory mapping

Coupled with multi-cores

Device Partitioning

Interrupt routing

I/O DMA coupled with memory partitioning &

Physical Memory / Machine Memory mapping

Virtualization

PlanHistory



105/107

105Ivan Boule




Transparent Hardware EmulationTransparent Hardware Virtualization

Paravirtualization

Hardware-Assisted VirtualizationVirtualization and Embedded Systems


Virtualization

Evolutions of Virtualization


106/107

106Ivan Boule

VMM shipped with hardwareBy CPU/Motherboard constructor

With extensions of Machine constructor

Plays BIOS role

VMM still a boot option ?

VMM available as free software ?

VMM available as Open Source ?

Virtualization

Evolutions of Virtualization

Public & Open VMM API


107/107

107Ivan Boule

Public & Open VMM API

Ease port of OS on top of VMM

Device Virtualization

Included in VMM APIAllow generic device drivers in OS's

Increase OS portability

os virtualization (1)

Documents