networks and operang systems chapter 21: virtual ......• “nested page tables” – relavely new...
TRANSCRIPT
![Page 1: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/1.jpg)
NetworksandOpera/ngSystemsChapter21:VirtualMachineMonitors
(252‐0062‐00)
DonaldKossmann&TorstenHoeflerFrühjahrssemester2013
©SystemsGroup|DepartmentofComputerScience|ETHZürich
![Page 2: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/2.jpg)
Last/me:I/O
• Networkstackimplementa/on• NetworkdevicesandnetworkI/O• MemorymanagementintheI/Osubsystem
• Performanceissues– Buffering– Mul/plequeuesandreceive‐sidescaling
![Page 3: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/3.jpg)
This/me:VirtualMachineMonitors
• Basicdefini/ons• Whywouldyouwantone?• Structure• Howdoesitwork?– CPU– MMU– Memory– Devices– Network
• Acknowledgement:ThankstoSteveHandforsomeoftheslides!
![Page 4: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/4.jpg)
WhatisaVirtualMachineMonitor?
• Virtualizesanen/re(hardware)machine– ContrastwithOSprocesses– Interfaceprovidedis“illusionofrealhardware”– Applica/onsarethereforecompleteOpera/ngSystemsthemselves
– Terminology:GuestOpera+ngSystems
• Oldidea:IBMVM/CMS(1960s)– Recentlyrevived:VMware,Xen,Hyper‐V,kvm,etc.
![Page 5: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/5.jpg)
VMMsandHypervisors
Realhardware
Hypervisor
Guestopera/ngsystem
App
App
Guestopera/ngsystem
App
App
VMM VMM
Somefolksdis/nguishtheVirtualMachineMonitorfromthe
Hypervisor(wewon’t)
Createsillusionofhardware
![Page 6: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/6.jpg)
Whywouldyouwantone?
• Diagrams:• Serverconsolida/on(programassumesownmachine)
• Performanceisola/on
• Backwardcompa/bility
• Cloudcompu/ng(unitofsellingcycles)
• SomethingundertheOS:replay,audi/ng,trustedcompu/ng,rootkits
![Page 7: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/7.jpg)
Runningmul/pleOSesononemachine
• Applica/oncompa/bility– IuseUbuntufor
almosteverything,butIeditslidesinPowerPoint
– SomepeoplecompileBarrelfishinaDebianVMoverWindows7withHyper‐V
• Backwardcompa/bility– Nothingbeatsa
Windows98virtualmachineforplayingoldcomputergames
Realhardware
Hypervisor
App
App
App
App
App
App
![Page 8: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/8.jpg)
Serverconsolida/on
• Manyapplica/onsassumetheyhavethemachinetothemselves
• Eachmachineismostlyidle
⇒ ConsolidateserversontoasinglephysicalmachineRealhardware
Hypervisor
App
lica/
on
App
lica/
on
App
lica/
on
![Page 9: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/9.jpg)
Resourceisola/on
• Surprisingly,modernOSesdonothaveanabstrac/onforasingleapplica/on
• Performanceisola/oncanbecri/calinsomeenterprises
• UsevirtualmachinesasresourcecontainersRealhardware
Hypervisor
App
lica/
on
App
lica/
on
App
lica/
on
![Page 10: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/10.jpg)
Cloudcompu/ng
• Sellingcompu/ngcapacityondemand– E.g.AmazonEC2,
GoGrid,etc.• Hypervisors
decouplealloca+onofresources(VMs)fromprovisioningofinfrastructure(physicalmachines)
Realhardware
Hypervisor
App
lica/
on
App
lica/
on
Realhardware
Hypervisor
App
lica/
on
App
lica/
on
Realhardware
Hypervisor
App
lica/
on
App
lica/
on
Realhardware
Hypervisor
App
lica/
on
App
lica/
on
Realhardware
Hypervisor
App
lica/
on
App
lica/
on
Realhardware
Hypervisor
App
lica/
on
App
lica/
on
![Page 11: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/11.jpg)
Opera/ngSystemdevelopment
• Buildingandtes/nganewOSwithoutneedingtorebootrealhardware
• VMMomengivesyoumoreinforma/onaboutfaultsthanrealhardwareanywayRealhardware
Hypervisor
Compiler
Edito
r
Visual
Stud
io
![Page 12: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/12.jpg)
Othercoolapplica/ons…
• Tracing• Debugging• Execu/onreplay
• Lock‐stepexecu/on
• Livemigra/on• Rollback• Specula/on• Etc….Realhardware
Hypervisor
Tracer
App
lica/
on
App
lica/
on
![Page 13: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/13.jpg)
Howdoesitallwork?
• Note:ahypervisorisbasicallyanOS– Withan“unusualAPI”
• Manyfunc/onsquitesimilar:– Mul/plexingresources– Scheduling,virtualmemory,devicedrivers
• Different:– Crea/ngtheillusionofhardwareto“applica/ons”– GuestOSesarelessflexibleinresourcerequirements
![Page 14: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/14.jpg)
HostedVMMs
Realhardware
Hostopera/ngsystem
App
lica/
on
Guestopera/ngsystem
App
App
VMM
App
lica/
on Examples:
• VMwareworksta/on• LinuxKVM• MicrosomHyper‐V
![Page 15: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/15.jpg)
Hypervisor‐basedVMMs
Realhardware
Hypervisor
Console(Mgmt)opera/ngsystem
Console
Mgm
t.
Guestopera/ngsystem
App
App
VMM VMM
Guestopera/ngsystem
App
App
VMM
Examples:• VMwareESX• IBMVM/CMS• Xen
![Page 16: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/16.jpg)
Howtovirtualize…
• TheCPU(s)?• TheMMU?
• Physicalmemory?
• Devices(disks,etc.)?• TheNetwork
and?
![Page 17: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/17.jpg)
VirtualizingtheCPU
• ACPUarchitectureisstrictlyvirtualizableifitcanbeperfectlyemulatedoveritself,withallnon‐privilegedinstruc/onsexecutedna/vely
• Privilegedinstruc/ons⇒trap– Kernel‐mode(i.e.theVMM)emulatesinstruc/on– Guest’skernelmodeisactuallyusermode
• Oranother,extraprivilegelevel(suchasring1)
• Examples:IBMS/390,Alpha,PowerPC
![Page 18: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/18.jpg)
VirtualizingtheCPU
• Astrictlyvirtualizableprocessorcanexecuteacompletena/veGuestOS– Guestapplica/onsruninusermodeasbefore– Guestkernelworksexactlyasbefore
• Problem:x86architectureisnotvirtualizable– About20instruc/onsaresensi/vebutnotprivileged– Mostlysegmentloadsandprocessorflagmanipula/on
![Page 19: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/19.jpg)
Non‐virtualizablex86:example
• PUSHF/POPFinstruc/ons– Push/popcondi/oncoderegister– Includesinterruptenableflag(IF)
• Unprivilegedinstruc/ons:fineinuserspace!– IFisignoredbyPOPFinusermode,notinkernelmode
⇒VMMcan’tdetermineifGuestOSwantsinterrruptsdisabled!– Can’tcauseatrapona(privileged)POPF – Preventscorrectfunc/oningoftheGuestOS
![Page 20: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/20.jpg)
Solu/ons1. Emula/on:emulateallkernel‐modecodeinsomware
– Veryslow–par/cularlyforI/Ointensiveworkloads– Usedby,e.g.,SomPC
2. Paravirtualiza8on:modifyGuestOSkernel– Replacewithexplicittrapinstruc/ontoVMM– Alsocalleda“HyperCall”(usedforallkindsofthings)– Usedby,e.g.,Xen
3. Binaryrewri/ng:– Protectkernelinstruc/onpages,traptoVMMonfirstIFetch– ScanpageforPOPFinstruc/onsandreplace– Restartinstruc/oninGuestOSandcon/nue– Usedby,e.g.VMware
4. Hardwaresupport:IntelVT‐x,AMD‐V– ExtraprocessormodecausesPOPFtotrap
![Page 21: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/21.jpg)
VirtualizingtheMMU
• HypervisorallocatesmemorytoVMs– Guestassumescontroloverallphysicalmemory
– VMMcan’tletGuestOStoinstallmappings
• Defini/onsneeded:– Virtualaddress:avirtualaddressintheguest– Physicaladdress:asseenbytheguest– Machineaddress:realphysicaladdress• AsseenbytheHypervisor
![Page 22: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/22.jpg)
Virtual/Physical/Machine
GuestVirtualAS
GuestPhysicalAS
MachineMemory
5
5
9
2
6
17Guest1:
Guest2:
![Page 23: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/23.jpg)
MMUVirtualiza/on
• Cri/calforperformance,challengingtomakefast,especiallySMP– Hot‐unplugunnecessaryvirtualCPUs– Usemul/castTLBflushparavirtualiza/onsetc
• Xensupports3MMUvirtualiza/onmodes1. Direct(“Writable”)pagetables2. Shadowpagetables3. HardwareAssistedPaging
• OSParavirtualiza/oncompulsoryfor#1,op/onal(andverybeneficial)for#2&3
![Page 24: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/24.jpg)
Paravirtualiza/onapproach
• GuestOScreatespagetablesthehardwareuses– VMMmustvalidateallupdatestopagetables– Requiresmodifica/onstoGuestOS– Notquiteenough…
• VMMmustcheckallwritestoPTEs– Write‐protectallPTEstotheGuestkernel– AddaHyperCalltoupdatePTEs– Batchupdatestoavoidtrapoverhead– OSisnowawareofmachineaddresses– Significantoverhead!
![Page 25: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/25.jpg)
Para‐VirtualizingtheMMU
• GuestOSesallocateandmanageownPTs– HypercalltochangePTbase
• VMMmustvalidatePTupdatesbeforeuse– Allowsincrementalupdates,avoidsrevalida/on
• Valida/onrulesappliedtoeachPTE:– 1.Guestmayonlymappagesitowns*
– 2.PagetablepagesmayonlybemappedRO
• VMMtrapsPTEupdatesandemulates,or‘unhooks’PTEpageforbulkupdates
![Page 26: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/26.jpg)
WriteablePageTables:1–Writefault
MMU
GuestOS
VMM
Hardware
pagefault
firstguestwrite
guestreads
Virtual→Machine
![Page 27: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/27.jpg)
WriteablePageTables:2–Emulate?
GuestOS
VMM
Hardware
firstguestwrite
guestreads
Virtual→Machine
emulate?
yes
MMU
![Page 28: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/28.jpg)
WriteablePageTables:3‐Unhook
GuestOS
VMM
Hardware
guestwrites
guestreads
Virtual→MachineX
MMU
![Page 29: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/29.jpg)
WriteablePageTables:4‐FirstUse
GuestOS
VMM
Hardware
pagefault
guestwrites
guestreads
Virtual→MachineX
MMU
![Page 30: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/30.jpg)
WriteablePageTables:5–Re‐hook
GuestOS
VMM
Hardware
validate
guestwrites
guestreads
Virtual→Machine
MMU
![Page 31: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/31.jpg)
Writeablepagetablesrequireparavirtualiza/on
GuestVirtualAS
MachineMemory
5
5
9
2
6
17Guest1:
Guest2:
GuestsdirectlyshareMachineMemory
![Page 32: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/32.jpg)
ShadowPageTables
• GuestOSsetsupitsownpagetables– Notusedbythehardware!
• VMMmaintainsshadowpagetables– MapdirectlyfromGuestVAstoMachineAddresses– HardwareswitchedwheneverGuestreloadsPTPR
• VMMmustkeepV→MtableconsistentwithGuestV→Ptableandit’sownP→Mtable– VMMwrite‐protectsallguestpagetables– Write⇒trap:applywritetoshadowtableaswell– Significantoverhead!
![Page 33: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/33.jpg)
ShadowPageTables
GuestVirtualAS
GuestPhysicalAS
MachineMemory
5
5
9
2
6
17Guest1:
Guest2:
Shadowpagetablemappings
![Page 34: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/34.jpg)
Shadowpagetables
MMU
GuestOS
VMM
Hardware
accessedanddirtybits
guestwrites
guestreads
Virtual→Guest‐Physical
Virtual→Machine
updates
• Guestchangesop/onal,buthelpwithbatching,knowingwhentounshadow
• Latestalgorithmsworkremarkablywell
![Page 35: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/35.jpg)
Hardwaresupport
• “Nestedpagetables”– Rela/velynewinAMD(NPT)andIntel(EPT)hardware
• Two‐leveltransla/onofaddressesintheMMU– Hardwareknowsabout:
• V→Ptables(intheGuest)• P→Mtables(intheHypervisor)
– TaggedTLBstoavoidexpensiveflushonaVMentry/exit
• Veryniceandeasytocodeto– Onereasonkvmissosmall
• Significantperformanceoverhead…
![Page 36: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/36.jpg)
Memoryalloca/on
• GuestOSisnotexpec/ngphysicalmemorytochangeinsize!
• Twoproblems:– HypervisorwantstoovercommitRAM– Howtoreallocate(machine)memorybetweenVMs
• Phenomenon:DoublePaging– Hypervisorpagesoutmemory– GuestOSdecidestopageoutphysicalframe– (Unwivngly)faultsitinviatheHypervisor,onlytowriteitoutagain
![Page 37: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/37.jpg)
Ballooning
• TechniquetoreclaimmemoryfromaGuest• Installa“balloondriver”inGuestkernel– Canallocateandfreekernelphysicalmemory• Justlikeanyotherpartofthekernel
– UsesHyperCallstoreturnframestotheHypervisor,andhavethemreturned• GuestOSisunware,simplyallocatesphysicalmemory
![Page 38: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/38.jpg)
Ballooning:takingRAMawayfromaVM
1. VMMasksballoondriverformemory
2. BalloondriverasksGuestOSkernelformoreframes– “inflatestheballoon”
3. BalloondriversendsphysicalframenumberstoVMM
4. VMMtranslatesintomachineaddressandclaimstheframes
Balloon
Guestphysicaladdressspace
Balloondriver
![Page 39: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/39.jpg)
Ballooning:takingRAMawayfromaVM
1. VMMasksballoondriverformemory
2. BalloondriverasksGuestOSkernelformoreframes– “inflatestheballoon”
3. BalloondriversendsphysicalframenumberstoVMM
4. VMMtranslatesintomachineaddressesandclaimstheframes
Balloon
Guestphysicaladdressspace
Physicalmemoryclaimedby
balloondriver
Balloondriver
![Page 40: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/40.jpg)
ReturningRAMtoaVM
1. VMMconvertsmachineaddressintoaphysicaladdresspreviouslyallocatedbytheballoondriver
2. VMMhandsPFNtoballoondriver
3. BalloondriverfreesphysicalframebacktoGuestOSkernel– “deflatestheballoon”
Balloon
Guestphysicaladdressspace
Balloondriver
![Page 41: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/41.jpg)
VirtualizingDevices
• Familiarbynow:trap‐and‐emulate– I/Ospacetraps– Protectmemoryandtrap– “Devicemodel”:somwaremodelofdeviceinVMM
• Interrupts→upcallstoGuestOS– Emulateinterruptcontroller(APIC)inGuest– EmulateDMAwithcopyintoGuestPAS
• Significantperformanceoverhead!
![Page 42: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/42.jpg)
Paravirtualizeddevices
• “Fake”devicedriverswhichcommunicateefficientlywithVMMviahypercalls– Usedforblockdeviceslikediskcontrollers– Networkinterfaces– “VMwaretools”ismostlyaboutthese
• Drama/callybeyerperformance!
![Page 43: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/43.jpg)
Networking
• VirtualnetworkdeviceintheGuestVM• Hypervisorimplementsa“somswitch”– En/revirtualIP/Ethernetnetworkonamachine
• Manydifferentaddressingop/ons– SeparateIPaddresses– SeparateMACaddresses
– NAT• Etc.
![Page 44: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/44.jpg)
Wherearetherealdrivers?
1. IntheHypervisor– E.g.VMwareESX– Problem:needtorewritedevicedrivers(newOS)
2. IntheconsoleOS– ExportvirtualdevicestootherVMs
3. In“driverdomains”– Maphardwaredirectlyintoa“trusted”VM
• DevicePassthrough– RunyourfavoriteOSjustforthedevicedriver– UseIOMMUhardwaretoprotectothermemoryfromdriverVM
4. Use“self‐virtualizingdevices”
![Page 45: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/45.jpg)
Xen3.xArchitecture
XenVirtualMachineMonitorEventChannel VirtualMMUVirtualCPUControlIF
Hardware(SMP,MMU,physicalmemory,Ethernet,SCSI/IDE)
GuestOS(XenLinux)
DeviceManager&Controls/w
Na/veDeviceDrivers
VM0
GuestOS(XenLinux)
UnmodifiedUser
Somware
VM1
SMPGuestOS(XenLinux)
UnmodifiedUser
Somware
Front‐EndDeviceDrivers
VM2
UnmodifiedGuestOS(WinXP)
UnmodifiedUser
Somware
Front‐EndDeviceDrivers
VM3
SafeHWIF
Virtualswitch
Front‐EndDeviceDrivers
ThankstoSteveHandforsomeofthesediagrams
![Page 46: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/46.jpg)
Rememberthiscard?
![Page 47: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/47.jpg)
SR‐IOV
• Single‐RootI/OVirtualiza/on• Keyidea:dynamicallycreatenew“PCIedevices”– PhysicalFunc/on(PF):originaldevice,fullfunc/onality
– VirtualFunc/on(VF):extra“device”,limitedfun/onality
– VFscreated/destroyedviaPFregisters• Fornetworking:– Par//onsanetworkcard’sresources– Withdirectassignmentcanimplementpassthrough
![Page 48: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/48.jpg)
SR‐IOVinac/on
SR‐IOVNICVirtualethernetbridge/switch,packetclassifier
LAN
Virtualfunc/on
Virtualfunc/on
Virtualfunc/on Physicalfunc/on
PCIe
IOMMU
VMM
VM
VFdriver
VM
VFdriver
VM
VFdriver
VM
VNICdrvr
VM
PFdriver
VSwitch
![Page 49: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/49.jpg)
Self‐virtualizingdevices
• Candynamicallycreateupto2048dis/nctPCIdevicesondemand!– HypervisorcancreateavirtualNICforeachVM– Somswitchdriverprograms“master”NICtodemuxpacketstoeachvirtualNIC
– PCIbusisvirtualizedineachVM– EachGuestOSappearstohave“real”NIC,talksdirecttotherealhardware
![Page 50: Networks and Operang Systems Chapter 21: Virtual ......• “Nested page tables” – Relavely new in AMD (NPT) and Intel (EPT) hardware • Two‐level translaon of addresses in](https://reader034.vdocument.in/reader034/viewer/2022042202/5ea2ec2830b9ba42ee163edb/html5/thumbnails/50.jpg)
NextWeek
Reliablestorage
OSResearch/Future™