![Page 1: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/1.jpg)
CRIU:Time and Space Travel Servicefor Linux Applications
Pavel EmelyanovLinuxCon NA, New Orleans, 2013
![Page 2: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/2.jpg)
2
Agenda
What is CRIU?
Project history and state
Usage scenarios Live migration
Reboot-less kernel upgrade
Slow services startup
Advanced debugging and testing
and more...
![Page 3: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/3.jpg)
3
What is CRIU?
Checkpoint Restore In Userspace
Checkpointor
Dump
Restoreor
Restart
Fullinfo
aboutstate
![Page 4: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/4.jpg)
4
Why in userspace?
Kernel
User-space
Dump:- Ptrace- /proc- netlink- syscalls
Restore:- syscalls
Process
kmod
C/R API
![Page 5: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/5.jpg)
5
CRIU background
Project started ~2 years ago
– an RFC on kernel memory API extension
– small command line tool
– minimal dump of process' internals
First release
– 23 Jul 2012
– x86 and basic stuff
Since then
– 150+ kernel patches merged
– new APIs for reading and setting process' state
![Page 6: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/6.jpg)
6
Current project state
The latest release
– v0.7
– supports x86 & ARM
– stuff typical applications use
Explicitly checked
– Apache, nginx, Oracle*, mysql, mongodb
– Ssh/sshd, openvpn*, cron, sendmail
– Java, gcc, make
– VNC + { gimp, mplayer, blender, supertux }
– Screen + { bash, top, tcpdump, tar/bz2 }
* some kernel tweaks required
![Page 7: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/7.jpg)
7
Usage scenarios
Live migration
– Useful in cluster
Kernel upgrade w/o reboot
Slow services startup
Periodic snapshots
– HPC case
Advanced debugging and testing
![Page 8: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/8.jpg)
8
Live migration
Host A Host B
![Page 9: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/9.jpg)
9
Live migration ++
Host A Host B
Shared FS
Pre-migrate memory
with memory tracker
![Page 10: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/10.jpg)
10
Load balancing on cluster
Host A
Host C
Host B
![Page 11: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/11.jpg)
11
Node maintenance
Host A Host B
![Page 12: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/12.jpg)
12
Kernel upgrade w/o reboot
Host
Kernel A
KexecKernel B
![Page 13: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/13.jpg)
13
Slow services startup
time# service foo start
Service readiness
Spawn process
Load config
Topup caches
Initialize resource pools
Ready
T
100%
![Page 14: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/14.jpg)
14
Slow services startup
time
Tt < T
Ready
Spawn process
100%
Service readiness
# service foo start
![Page 15: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/15.jpg)
15
Periodic snapshots
time
Memory tracker helpsto keep images smaller
![Page 16: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/16.jpg)
16
HPC
time
Powerfailure
0% 20% 40% 60% 60%
![Page 17: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/17.jpg)
17
Advanced debugging
Production Host
Applicationin trouble
Developer Host
Debugger
![Page 18: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/18.jpg)
18
Advanced testing
Start App
T ~ 30 sec
t ~ 0.1 sec
...
...
t ~ 0.1 sec
![Page 19: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/19.jpg)
19
Advanced testing
...
New testor
new hardware ?
![Page 20: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/20.jpg)
20
More (funny) usecases
Forgot to launch your program in screen
– Live-migrate it there
Playing a game without the save button
– Snapshot it
Suspend-to-RAM a VDI session
![Page 21: CRIU: Time and Space Travel Service for Linux Applications · 2017. 12. 14. · CRIU: Time and Space Travel Service for Linux Applications Pavel Emelyanov LinuxCon NA, New Orleans,](https://reader033.vdocument.in/reader033/viewer/2022051907/5ff9c5fd0a4fe81f807a306d/html5/thumbnails/21.jpg)
21
CRIU project resources
http://criu.org – project news and documentation
http://git.criu.org – git repo with tool sources
+CRIU page
[email protected] mailing list
[email protected] is me
Thank you!