nohype : virtualized cloud infrastructure without the virtualization
DESCRIPTION
NoHype : Virtualized Cloud Infrastructure without the Virtualization. Eric Keller , Jakub Szefer , Jennifer Rexford, Ruby Lee. Princeton University Slides taken from: http://www.cs.princeton.edu/~jrex/talks/isca10.pptx. ISCA 2010. Virtualized Cloud Infrastructure. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/1.jpg)
NoHype: Virtualized Cloud Infrastructure
without the Virtualization
Eric Keller, Jakub Szefer, Jennifer Rexford, Ruby Lee
ISCA 2010
Princeton UniversitySlides taken from:
http://www.cs.princeton.edu/~jrex/talks/isca10.pptx
![Page 2: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/2.jpg)
Virtualized Cloud Infrastructure• Run virtual machines on a hosted infrastructure
• Benefits…– Economies of scale– Dynamically scale (pay for what you use)
![Page 3: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/3.jpg)
3
Without the Virtualization• Virtualization used to share servers
– Software layer running under each virtual machine
Physical Hardware
Hypervisor
OS OS
Apps Apps
Guest VM1 Guest VM2
servers
![Page 4: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/4.jpg)
4
Without the Virtualization• Virtualization used to share servers
– Software layer running under each virtual machine
• Malicious software can run on the same server– Attack hypervisor– Access/Obstruct other VMs
Physical Hardware
Hypervisor
OS OS
Apps Apps
Guest VM1 Guest VM2
servers
![Page 5: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/5.jpg)
5
Are these vulnerabilities imagined?• No headlines… doesn’t mean it’s not real
– Not enticing enough to hackers yet?(small market size, lack of confidential data)
• Virtualization layer huge and growing– 100 Thousand lines of code in hypervisor– 1 Million lines in privileged virtual machine
• Derived from existing operating systems – Which have security holes
![Page 6: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/6.jpg)
6
NoHype• NoHype removes the hypervisor
– There’s nothing to attack– Complete systems solution– Still retains the needs of a virtualized cloud infrastructure
Physical Hardware
OS OS
Apps Apps
Guest VM1 Guest VM2
No hypervisor
![Page 7: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/7.jpg)
7
Virtualization in the Cloud• Why does a cloud infrastructure use virtualization?
– To support dynamically starting/stopping VMs– To allow servers to be shared (multi-tenancy)
• Do not need full power of modern hypervisors– Emulating diverse (potentially older) hardware– Maximizing server consolidation
![Page 8: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/8.jpg)
8
Roles of the Hypervisor• Isolating/Emulating resources
– CPU: Scheduling virtual machines– Memory: Managing memory– I/O: Emulating I/O devices
• Networking• Managing virtual machines
![Page 9: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/9.jpg)
9
Roles of the Hypervisor• Isolating/Emulating resources
– CPU: Scheduling virtual machines– Memory: Managing memory– I/O: Emulating I/O devices
• Networking• Managing virtual machines
Push to HW /Pre-allocation
![Page 10: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/10.jpg)
10
Roles of the Hypervisor• Isolating/Emulating resources
– CPU: Scheduling virtual machines– Memory: Managing memory– I/O: Emulating I/O devices
• Networking• Managing virtual machines
Push to HW /Pre-allocation
Remove
![Page 11: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/11.jpg)
11
Roles of the Hypervisor• Isolating/Emulating resources
– CPU: Scheduling virtual machines– Memory: Managing memory– I/O: Emulating I/O devices
• Networking• Managing virtual machines
Push to HW /Pre-allocation
Remove
Push to side
![Page 12: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/12.jpg)
12
Roles of the Hypervisor• Isolating/Emulating resources
– CPU: Scheduling virtual machines– Memory: Managing memory– I/O: Emulating I/O devices
• Networking• Managing virtual machines
Push to HW /Pre-allocation
Remove
Push to side
NoHype has a double meaning… “no hype”
![Page 13: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/13.jpg)
13
Scheduling Virtual Machines• Scheduler called each time hypervisor runs
(periodically, I/O events, etc.)– Chooses what to run next on given core– Balances load across cores
hypervisor
timer
switc
h
I/O
switc
h
timer
switc
h
VMs
time
Today
![Page 14: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/14.jpg)
14
Dedicate a core to a single VM• Ride the multi-core trend
– 1 core on 128-core device is ~0.8% of the processor
• Cloud computing is pay-per-use– During high demand, spawn more VMs– During low demand, kill some VMs– Customer maximizing each VMs work,
which minimizes opportunity for over-subscription
NoHype
![Page 15: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/15.jpg)
15
Managing Memory• Goal: system-wide optimal usage
– i.e., maximize server consolidation
• Hypervisor controls allocation of physical memory0
100
200
300
400
500
600
VM/app 3 (max 400)VM/app 2 (max 300)VM/app 1 (max 400)
Today
![Page 16: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/16.jpg)
16
Pre-allocate Memory• In cloud computing: charged per unit
– e.g., VM with 2GB memory
• Pre-allocate a fixed amount of memory– Memory is fixed and guaranteed– Guest VM manages its own physical memory
(deciding what pages to swap to disk)
• Processor support for enforcing:– allocation and bus utilization
NoHype
![Page 17: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/17.jpg)
17
Emulate I/O Devices• Guest sees virtual devices
– Access to a device’s memory range traps to hypervisor– Hypervisor handles interrupts– Privileged VM emulates devices and performs I/O
Physical Hardware
Hypervisor
OS OS
Apps Apps
Guest VM1 Guest VM2
RealDrivers
Priv. VMDevice
Emulation
traptraphypercall
Today
![Page 18: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/18.jpg)
18
• Guest sees virtual devices– Access to a device’s memory range traps to hypervisor– Hypervisor handles interrupts– Privileged VM emulates devices and performs I/O
Emulate I/O Devices
Physical Hardware
Hypervisor
OS OS
Apps Apps
Guest VM1 Guest VM2
RealDrivers
Priv. VMDevice
Emulation
traptraphypercall
Today
![Page 19: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/19.jpg)
19
Dedicate Devices to a VM• In cloud computing, only networking and storage• Static memory partitioning for enforcing access
– Processor (for to device), IOMMU (for from device)
Physical Hardware
OS OS
Apps Apps
Guest VM1 Guest VM2
NoHype
![Page 20: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/20.jpg)
20
Virtualize the Devices• Per-VM physical device doesn’t scale• Multiple queues on device
– Multiple memory ranges mapping to different queues
Processor Chipset
MemoryC
lass
ifyM
UX M
AC
/PH
Y
Network Card
Peripheralbus
NoHype
PCI-SIG StandardSR-IOV (Single Root-IO Virt.)
![Page 21: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/21.jpg)
21
• Ethernet switches connect servers
Networking
server server
Today
![Page 22: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/22.jpg)
22
• Software Ethernet switches connect VMs
Networking (in virtualized server)
Virtual server Virtual server
Software Virtual switch
Today
![Page 23: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/23.jpg)
23
• Software Ethernet switches connect VMs
Networking (in virtualized server)
OS
Apps
Guest VM1
Hypervisor
OS
Apps
Guest VM2
hypervisor
Today
![Page 24: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/24.jpg)
24
• Software Ethernet switches connect VMs
Networking (in virtualized server)
OS
Apps
Guest VM1
Hypervisor
OS
Apps
Guest VM2
SoftwareSwitch
Priv. VM
Today
![Page 25: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/25.jpg)
25
Do Networking in the Network• Co-located VMs communicate through software
– Performance penalty for not co-located VMs– Special case in cloud computing– Artifact of going through hypervisor anyway
• Instead: utilize hardware switches in the network– Modification to support hairpin turnaround
NoHype
![Page 26: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/26.jpg)
26
Managing Virtual Machines• Allowing a customer to start and stop VMs
Wide Area Network
Request: Start VM
Cloud Customer
CloudProvider
Today
![Page 27: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/27.jpg)
27
Managing Virtual Machines• Allowing a customer to start and stop VMs
Wide Area Network
Servers
Request: Start VM
Cloud Customer
CloudProvider
.
.
.
VM images
Cloud Manager
Request: Start VM
Today
![Page 28: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/28.jpg)
28
Hypervisor’s Role in Management• Run as application in privileged VM
Physical Hardware
Hypervisor
Priv. VM
VM Mgmt.
Today
![Page 29: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/29.jpg)
29
Hypervisor’s Role in Management• Receive request from cloud manager
Physical Hardware
Hypervisor
Priv. VM
VM Mgmt.
Today
![Page 30: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/30.jpg)
30
Hypervisor’s Role in Management• Form request to hypervisor
Physical Hardware
Hypervisor
Priv. VM
VM Mgmt.
Today
![Page 31: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/31.jpg)
31
Hypervisor’s Role in Management• Launch VM
Physical Hardware
Hypervisor
Priv. VM
VM Mgmt.
OS
Apps
Guest VM1
Today
![Page 32: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/32.jpg)
32
Decouple Management And Operation• System manager runs on its own core
Core 0
SystemManager
Core 1
NoHype
![Page 33: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/33.jpg)
33
Decouple Management And Operation• System manager runs on its own core• Sends an IPI to start/stop a VM
Core 0
SystemManager
Core 1
IPI
NoHype
![Page 34: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/34.jpg)
34
Decouple Management And Operation• System manager runs on its own core• Sends an IPI to start/stop a VM• Core manager sets up core, launches VM
– Not run again until VM is killed
Core 0
SystemManager
Core 1
CoreManager OS
Apps
Guest VM2
IPI
NoHype
![Page 35: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/35.jpg)
35
Removing the Hypervisor Summary• Scheduling virtual machines
– One VM per core
• Managing memory– Pre-allocate memory with processor support
• Emulating I/O devices– Direct access to virtualized devices
• Networking– Utilize hardware Ethernet switches
• Managing virtual machines– Decouple the management from operation
![Page 36: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/36.jpg)
36
Security Benefits• Confidentiality/Integrity of data• Availability• Side channels
![Page 37: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/37.jpg)
37
Security Benefits• Confidentiality/Integrity of data• Availability• Side channels
![Page 38: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/38.jpg)
38
Confidentiality/Integrity of DataRequires access to the data
• System manager can alter memory access rules– But, guest VMs do not interact with the system manager
With hypervisor NoHypeRegisters upon VM exit No schedulingPackets sent through software switch
No software switch
Memory accessible by hypervisor
No hypervisor
![Page 39: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/39.jpg)
39
NoHype Double Meaning• Means no hypervisor, also means “no hype”
• Multi-core processors– Available now
• Extended (Nested) Page Tables– Available now
• SR-IOV and Directed I/O (VT-d)– Network cards now, Storage devices near future
• Virtual Ethernet Port Aggregator (VEPA)– Next-generation switches
![Page 40: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/40.jpg)
40
Conclusions and Future Work• Trend towards hosted and shared infrastructures• Significant security issue threatens adoption• NoHype solves this by removing the hypervisor• Performance improvement is a side benefit
• Future work:– Implement on current hardware– Assess needs for future processors
![Page 41: NoHype : Virtualized Cloud Infrastructure without the Virtualization](https://reader036.vdocument.in/reader036/viewer/2022062501/5681692f550346895de0758b/html5/thumbnails/41.jpg)
41
Questions?
Contact info:
http://www.princeton.edu/~ekeller
http://www.princeton.edu/~szefer