cs5460: operating systems lecture: virtualization 2aburtsev/cs5460/lectures/...resume checkpoints...
TRANSCRIPT
![Page 1: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/1.jpg)
CS5460: Operating Systems
Lecture: Virtualization 2
Anton BurtsevApril, 2014
![Page 2: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/2.jpg)
Paravirtualization:Xen
![Page 3: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/3.jpg)
● Complete illusion of physical hardware● Trap _all_ sensitive
instructions● Example: page table
update
Full virtualization
Virtualized OS
Hypervisor
PTE update (mov)
![Page 4: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/4.jpg)
● Complete illusion of physical hardware● Trap _all_ sensitive
instructions● Example: page table
update
Full virtualization
Virtualized OS
Hypervisor
PTE update (mov)
Trap
![Page 5: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/5.jpg)
● Complete illusion of physical hardware● Trap _all_ sensitive
instructions● Example: page table
update
Full virtualization
Virtualized OS
Hypervisor
PTE update (mov)
if (safe) { update_pte(); emulate_mov(); }
Next instruction
Trap
● Traps are slow● Binary translation is
faster, for some events● Not for PTE updates,
why?
![Page 6: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/6.jpg)
Performance problems
Virtualized OS
Hypervisor
PTE update (mov)
if (safe) { update_pte(); emulate_mov(); }
Next instruction
Trap
● Traps are slow● Binary translation is faster
● For some events● Not for PTE updates, why?
![Page 7: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/7.jpg)
Paravirtualization
● No illusion of hardware● Instead: paravirtualized interface
● Explicit hypervisor calls to update sensitive state– Page tables, interrupt flag
● But Guest OS needs porting● Applications run natively in Ring 3
![Page 8: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/8.jpg)
ParavirtualizationParavirtualized OS
Hypervisor
PTE update
Batch updatesupdate 1update 2
Invoke hypervisor
if (safe) update
![Page 9: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/9.jpg)
Xen
![Page 10: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/10.jpg)
Segmentation and paging
![Page 11: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/11.jpg)
Hypervisor protection
![Page 12: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/12.jpg)
Hardware support for virtualization:KVM
![Page 13: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/13.jpg)
Basic idea
Host instruction stream
Guest instruction stream
VM Entry VM Exit
Host State
Guest State
VMCS
![Page 14: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/14.jpg)
New mode of operation:VMX root
● VMX root operation● 4 privilege levels
● VMX non-root operation● 4 privilege levels as well, but unable to invoke
VMX root instructions● Guest runs until it performs exception causing it
to exit● Rich set of exit events● Guest state and exit reason are stored in VMCS
![Page 15: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/15.jpg)
Virtual machine control structure (VMCS)
● Guest State● Loaded on entries● Saved on exits
● Host State● Saved on entries● Loaded on exits
● Control fields● Execution control, exits control, entries control
![Page 16: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/16.jpg)
Guest state
● Register state● Non-register state
● Activity state: – active– inactive (HLT, Shutdown, wait for Startup IPI
interprocessor interrupt))● Interruptibility state
![Page 17: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/17.jpg)
Host state
● Only register state● ALU registers,
● also:● Base page table address (CR3)● Segment selectors● Global descriptors table ● Interrupt descriptors table
![Page 18: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/18.jpg)
VM-execution controls(asynchronous events control)
Reserved
Bit 31 Bit 0
External interrupts (maskable or IRQs) cause exits(yes/no)If not, then they delivered through guestIDT
NMI cause exits (yes/no)If not, then they are delivered normally through guest IDT (descriptor 2)
![Page 19: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/19.jpg)
VM-execution controls(synchronous events control, not all reasons are shown)
Reserved
Bit 31 Bit 0PAUSE
MONITOR
Act
ivat
e I/
O b
itm
aps
Unc
ondi
tion
al I
/O
HLT
INVLPG
![Page 20: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/20.jpg)
Exception bitmap(one for each of 32 IA-32 exceptions)
Bit 31 Bit 0
● IA-32 defines 32 exception vectors (interrupts 0-31)
● Each of them is configured to cause or not VM-exit
14 – page fault
![Page 21: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/21.jpg)
I/O Bitmaps
● Two addresses on 4KB memory areas (A and B)
A B
Safe I/O addresses (not causing exits)
![Page 22: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/22.jpg)
Exit information
● Information describing conditions of VM-exit is saved in VMCS● It's different for different types of event
![Page 23: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/23.jpg)
KVM
![Page 24: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/24.jpg)
Memory virtualization: brute force.
Hypervisor
HardwareTLB
Guest
PD
CR3
PT
Helper structures describe actual guest VM layout Maintained for each guest. On VM-Exit hypervisor adjusts guest page accordingly.
Write / read protectedpage table area. Every access results in VM-Exit and passes control to hypervisor
CPU stores pointer onguest page table directory
![Page 25: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/25.jpg)
Memory virtualization: shadow page tables
HardwareTLB
Guest
PD
CR3
PT
Active page table hierarchy VMM maintains it for each VM that it supports
Guest page table hierarchy It's writable, but can be inconsistent with active page table hierarchy stored by the hypervisor
PD PT
CPU stores pointer on active page table hierarchy. On Intel CPUs TLB is always refilled from active page table directory
![Page 26: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/26.jpg)
Nested page tables
hPT gPT Host Physical
gPT
VMM Host Virtual
Guest Physical
Guest VirtualgCR3
hCR3
0
0
0
PT
CR3 used by VMM
Translation can be cached in TLB
paged by CR3
paged by hCR3
paged by gCR3
![Page 27: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/27.jpg)
Page table lookup ● 4-level page table
![Page 28: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/28.jpg)
Nested page table lookup
![Page 29: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/29.jpg)
Efficient I/O
![Page 30: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/30.jpg)
Where is the bottleneck● What is the bottleneck in case of
virtualization?● CPU?
– CPU bound workloads execute natively on the real CPU
– Sometimes JIT compilation (binary translation makes them even faster [Dynamo]
● Everything what is inside VM is fast!● What is the most frequent operation
disturbing execution of VM? ● Device I/O!
● Disk, Network, Graphics
![Page 31: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/31.jpg)
Virtual devices in Xen
31
![Page 32: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/32.jpg)
Virtual devices in Xen
32
![Page 33: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/33.jpg)
Virtual devices in Xen
33
![Page 34: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/34.jpg)
Virtual devices in Xen
34
![Page 35: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/35.jpg)
Virtual devices in Xen
35
![Page 36: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/36.jpg)
How to make the I/O fast?● Take into account specifics of the device-
driver communication● Bulk
– Large packets (512B – 4K)● Session oriented
– Connection is established once (during boot)– No short IPCs, like function calls– Costs of establishing an IPC channel are irrelevant
● Throughput oriented– Devices have high delays anyway
● Asynchronous– Again, no function calls, devices are already
asynchronous
![Page 37: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/37.jpg)
Shared rings and events
![Page 38: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/38.jpg)
Shared rings
![Page 39: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/39.jpg)
Shared rings
![Page 40: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/40.jpg)
Shared rings
![Page 41: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/41.jpg)
Shared rings
![Page 42: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/42.jpg)
Where is a performance bottleneck here?
![Page 43: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/43.jpg)
Eliminate cache thrashing
![Page 44: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/44.jpg)
GPUs● Sending frames from the framebuffer
● No hardware acceleration● Too slow
● OpenGL/DirectX level virtualization● Send high-level OpenGL commands over rings● OpenGL operations will be executed on the real
GPU
![Page 45: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/45.jpg)
Devices supporting virtualization
![Page 46: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/46.jpg)
Some VM tricks:suspend/resume, checkpoints
migration
![Page 47: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/47.jpg)
Suspend
![Page 48: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/48.jpg)
Resume
![Page 49: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/49.jpg)
Checkpoints● Checkpoints are almost suspend/resume● A copy of the entire VM’s state has to be
saved● Memory
– OK, it’s relatively small 128MB-4GB● Disk
– Problem: disks are huge 100GB-1TB
● How to save storage efficiently?
● How to make it efficient?
![Page 50: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/50.jpg)
Branching storage
![Page 51: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/51.jpg)
Branching storage: snapshot
![Page 52: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/52.jpg)
Branching storage: writes
![Page 53: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/53.jpg)
Branching storage: snapshot
![Page 54: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/54.jpg)
Migration● Migration is essentially a live checkpoint
between machines● The goal: minimal downtime
● How to make the checkpoint faster?
![Page 55: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/55.jpg)
Migration: memory
![Page 56: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/56.jpg)
Migration: memory
![Page 57: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/57.jpg)
Migration: memory
![Page 58: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/58.jpg)
Migration: memory
![Page 59: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/59.jpg)
Migration: storage
![Page 60: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/60.jpg)
Migration
![Page 61: CS5460: Operating Systems Lecture: Virtualization 2aburtsev/cs5460/lectures/...Resume Checkpoints Checkpoints are almost suspend/resume A copy of the entire VM’s state has to be](https://reader034.vdocument.in/reader034/viewer/2022042223/5eca135d242c3845870919d8/html5/thumbnails/61.jpg)
References
● Intel® 64 and IA-32 Architectures Software Developer's Manual. Volume 3C: System Programming Guide, Part 3
● Ravi Bhargava, Benjamin Serebrin, Francesco Spadini, and Srilatha Manne. Accelerating two-dimensional page walks for virtualized systems. In ASPLOS'08.