snowflock: rapid vm cloning for cloud computing h. andrés lagar-cavilla
DESCRIPTION
SnowFlock: Rapid VM Cloning for Cloud Computing http://sysweb.cs.toronto.edu/snowflock H. Andrés Lagar-Cavilla. Joe Whitney, Adin Scannell , Steve Rumble, Philip Patchin , Eyal de Lara, Michael Brudno , and M. Satyanarayanan * University of Toronto, *CMU. Cloud. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/1.jpg)
SnowFlock: Rapid VM Cloning for Cloud Computing
http://sysweb.cs.toronto.edu/snowflockH. Andrés Lagar-Cavilla
Joe Whitney, Adin Scannell, Steve Rumble, Philip Patchin, Eyal de Lara, Michael Brudno, and M. Satyanarayanan*
University of Toronto, *CMU
![Page 2: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/2.jpg)
Cloud
• Utility Computing• Many machines• Many users• Many applications• Virtualization is key
![Page 3: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/3.jpg)
Virtualization
• Decoupling• Consolidation
HW More HW
• Security Isolation• Configuration Isolation
BIG!• OS kernel• Processes• IPC, sockets, pipes• File system
![Page 4: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/4.jpg)
Dynamic Application Scaling
Server
Requests RequestsRequests
Parallel Computation
DNA search, render, quant finance
![Page 5: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/5.jpg)
Poor Cloud Application Scaling
1. The programming model is awkward
“Boot and push”Today: Application state transmitted explicitly
![Page 6: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/6.jpg)
Poor Cloud Application Scaling
2. The response time is slow
Big VM state swap-inToday: Predict load, pre-allocate, keep idle,
consolidate, migrate
![Page 7: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/7.jpg)
Problem: Poor Cloud App Scaling
• Not Stateful– New VMs need to have state pushed
• Not Swift– New VMs come up in minutes
Slow swap-in: full VM transmission– Boot from scratch– Live migrate– Suspend/resume
![Page 8: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/8.jpg)
Q: What’s stateful and swift?
A: Fork
![Page 9: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/9.jpg)
SnowFlock: VM Fork
Stateful swift cloning of VMs
• State inherited up to the point of cloning• Local modifications are not shared• Clones make up an impromptu cluster
VM 0
Host 0
VM 1
Host 1
VM 2
Host 2
VM 3
Host 3
VM 4
Host 4
VirtualNetwork
![Page 10: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/10.jpg)
Fork: Well Understood Semanticsif more load: fork extra workersif load is low: dealloc excess workers
trusted codeforkif child: untrusted code
partition datafork N workersif child: work on ith slice of data
if cycles available: fork worker if child: do fraction of long
computation
Parallel Computation Load-balancing Server
Opportunistic Computation
Sandboxing
![Page 11: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/11.jpg)
VM Fork Challenge
• Big VM State Tx– VMs are big:
OS, disk, processes, …– Big means slow– Big means not scalable
At odds with fundamental bottleneckShared IO Resources
Suspend/resume latency
VMs
Seco
nds
0 4 8 12 16 20 24 28 320
100
200
300
400
![Page 12: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/12.jpg)
SnowFlock Insights
• VMs are BIG: Don’t send all the state!• Clones need little state of the parent• Clones exhibit common locality patterns• Clones generate lots of private state
![Page 13: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/13.jpg)
The Secret Sauce
VirtualMachine
VM DescriptorVM DescriptorVM Descriptor Multicast
?
?
State:Disk, OS,
Processes
Metadata“Special” PagesPage tablesGDT, vcpu~1MB for 1GB VM
1. Start only with the basics2. Fetch state on-demand3. Multicast: exploit net hw parallelism4. Multicast: exploit locality to prefetchClone 1PrivateState
Clone 2 Private State
5. Heuristics: don’t fetch if I’ll overwrite
![Page 14: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/14.jpg)
Why SnowFlock is Fast
• Start only with the basics• Send only what you really need• Multicast– Network hardware parallelism– Prefetch: exploit locality patterns
• Heuristics– Don’t send if I’ll overwrite– Malloc: exploit apps generating new state
![Page 15: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/15.jpg)
Clone Time
2 4 8 16 320100200300400500600700800900
DevicesSpawnMulticastStart ClonesXendDescriptor
Clones
Mill
iseco
nds
Scalable Cloning: Roughly Constant
![Page 16: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/16.jpg)
SnowFlock
32 VMs800 Milliseconds
![Page 17: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/17.jpg)
Page Fetching, SHRiMP 32 Clones 1GB
Unicast Multicast Unicast Multicast0100000020000003000000400000050000006000000700000080000009000000 Requests
Served
Mill
ions
of P
ages
Heuristics OFF
HeuristicsON
10K
![Page 18: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/18.jpg)
SnowFlock
40 MB sent instead of 32GB
(no more BIG VM state)
![Page 19: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/19.jpg)
Application Evaluation
• Embarrassingly parallel– 32 hosts x 4 processors
• CPU-intensive• Internet server– Respond in seconds
• Bioinformatics• Quantitative Finance• Rendering
![Page 20: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/20.jpg)
Application Run Times
Aqsis BLAST ClustalW distcc QuantLib SHRiMP0
20
40
60
80
100
120
140Ideal SnowFlock
Seco
nds
1-4 second overhead
![Page 21: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/21.jpg)
SnowFlock
≤ 7% Runtime Overhead
~ 5 seconds
![Page 22: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/22.jpg)
Throwing Everything At It
• Four concurrent sets of VMs– BLAST , SHRiMP , QuantLib , Aqsis
• Cycling five times– Clone, do task, join
• Shorter tasks– Range of 25-40 seconds: interactive service
• Evil allocation
![Page 23: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/23.jpg)
Throwing Everything At It
• Four concurrent sets of VMs– BLAST , SHRiMP , QuantLib , Aqsis
• Cycling five times– Clone, do task, join
• Shorter tasks– Range of 25-40 seconds: interactive service
• Evil allocation
![Page 24: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/24.jpg)
Throwing Everything At It
Aqsis BLAST QuantLib SHRiMP0
5
10
15
20
25
30
35
40Ideal SnowFlock
Seco
nds
Fork. Process 128 x 100% CPU. Disappear.30 Seconds
![Page 25: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/25.jpg)
The Bottomline
• Stateful, Swift and Scalable– 800ms, 32VMs– ≤ 7% runtime overhead– Only 40MBs on IO bottleneck, instead of 32GB
• Intuitive, well-understood semantics
But how do use it?
![Page 26: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/26.jpg)
SnowFlock APItix = sf_request_ticket(howmany)prepare_computation(tix.granted)me = sf_clone(tix)do_work(me)if (me != 0)send_results_to_master()sf_sync()
elsereceive_results()sf_join(tix)
scp … more in the future
Just like UNIX fork()
Block…
Child VMs are gone
![Page 27: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/27.jpg)
Many Ways to Use SnowFlock
• Add VM Fork to your code• Implement MPI with SnowFlock– Patchin, HPC Virt 09
• Fun with Python scripting• SnowFlock-based job dispatcher• Clustered web server• Implement MapReduce with SnowFlock• Cloud/cluster manager
Present
Future
![Page 28: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/28.jpg)
SnowFlock: VMs on-demand
• Stateful, Swift and Scalable• VMs on-demand, when I need them, cheap• No more over-provisioning!– No load prediction– No pre-allocation– No idle VMs– No memory consolidation– No migration
![Page 29: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/29.jpg)
Conclusion: SnowFlock In One Slide
• VM fork: natural intuitive semantics• The cloud bottleneck is the IO– Clones need little parent state– Generate their own state– Exhibit common locality patterns
• Sub-second cloning time• Negligible runtime overhead• Scalable: experiments with 128 processors
![Page 30: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/30.jpg)
Thanks!
[email protected]://www.cs.toronto.edu/~andreslc
http://sysweb.cs.toronto.edu/snowflock
This slide is not endorsed by the conference chair
![Page 31: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/31.jpg)
Extra Slides
![Page 32: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/32.jpg)
Disk
• Just like memory– Multicast and heuristics
• Virtual disk only for the “boot volume”– SnowFlock is good at cloning common contents
• Big data in suitable FS– Future work
![Page 33: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/33.jpg)
Just-in-Time Cloud
• Opportunistically clone over the wide area• Use the cloud from your laptop• “Cloud bursting”, a form of cyber foraging– Memory-on-demand, heuristics– Prefetching, content-addressable storage
![Page 34: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/34.jpg)
The Future of SnowFlock
• SnowFlock + Big data– Make VMs agnostic to the file system– Allocate VMs based on data availability
• The Abstract Data Type– Data + operations/transformations– Big data objects + VM-encapsulated operation?
![Page 35: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/35.jpg)
SnowFlock APImpirun –np 128
• Hide behind parallel API (MPI)• Now you can use the cloud with unmodified apps• And well-known commands
• Patchin, HPC Virt 09
![Page 36: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/36.jpg)
Memory on Demand Latency
0
50
100
150
200
250
300
Mic
rose
cond
s Network (unicast)
Context switch to dom0
SnowFlock in hypervisor
Xen hypervisor (shadow PT)
Page Fault (HW)
Memtap logic(map page)
![Page 37: SnowFlock: Rapid VM Cloning for Cloud Computing H. Andrés Lagar-Cavilla](https://reader036.vdocument.in/reader036/viewer/2022070504/56816953550346895de0fa1b/html5/thumbnails/37.jpg)
Memtap: Memory-on-demandVM
Hypervisor
Dom0 - memtap
9g056c0ab6bg756776a503ba4
Page Table
00000c0ab6000000000003ba4
ShadowPage Table
Bitmap
0
Read-only00000
9g056
Kick
Maps
R/W
11
Kick back
00000
bg756
1
Page Fault
1
9g056
paused