Download - Python performance engineering in 2017
![Page 1: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/1.jpg)
Python performance engineering in 2017
Alex Chistyakov, an engineer @ some company
![Page 2: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/2.jpg)
Agenda- Is Python slow?
- How to profile a Python program?
- We are not going to talk about blockchain, sorry
2
![Page 3: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/3.jpg)
Is Python slow?- Definitely, because of GIL and lack of proper JIT
3
![Page 4: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/4.jpg)
Is Python slow?- Definitely, because of GIL and a lack of a proper JIT
compiler
4
![Page 5: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/5.jpg)
Is Python slow?- Seriously, how do you define “to be slow”?
5
![Page 6: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/6.jpg)
Okay, let’s do our homework finally- Vagrant + https://github.com/alexclear/piterpy2017
6
![Page 7: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/7.jpg)
Okay, let’s do our homework finally- Vagrant + https://github.com/alexclear/piterpy2017
- 2-cores VirtualBox VM
7
![Page 8: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/8.jpg)
Okay, let’s do our homework finally- Vagrant + https://github.com/alexclear/piterpy2017
- 2-cores VirtualBox VM
- Something called Sanic:
https://github.com/channelcat/sanic because it’s 2017
and 2.x era should have ended years ago
8
![Page 9: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/9.jpg)
Okay, let’s do our homework finally- Vagrant + https://github.com/alexclear/piterpy2017
- 2-cores VirtualBox VM
- Something called Sanic:
https://github.com/channelcat/sanic because it’s 2017
and 2.x era should have ended years ago
- A load testing tool called ab because we love classics
9
![Page 10: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/10.jpg)
What is a typical Python program?- https://goo.gl/Abz8LX (Please note that this link will
produce different results in 5 years)
- Something not too server-side (or a framework)
- Something scientifical
- Some utility
10
![Page 11: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/11.jpg)
What is a typical Python program?- https://goo.gl/Abz8LX (Please note that this link will
produce different results in 5 years)
- Something not too server-side
- Something scientifical
- Some utility
- An async web application!
11
![Page 12: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/12.jpg)
A crash course in modern profiling- Collect stack samples at all costs
12
![Page 13: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/13.jpg)
A crash course in modern profiling- Collect stack samples at all costs
- Convert collected samples to Brendan Gregg’s
flamegraph.pl format
13
![Page 14: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/14.jpg)
A crash course in modern profiling- Collect stack samples at all costs
- Convert collected samples to Brendan Gregg’s
flamegraph.pl format
- Create a flamegraph
14
![Page 15: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/15.jpg)
Collecting samples
15
![Page 16: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/16.jpg)
Collecting samples- Uber’s pyflame: https://github.com/uber/pyflame
16
![Page 17: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/17.jpg)
A bit of internal dark magic- pyflame uses ptrace(2) system call
17
![Page 18: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/18.jpg)
A bit of internal dark magic- pyflame uses ptrace(2) system call
- It attaches to a running process and traverses its memory
18
![Page 19: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/19.jpg)
Limitations- Linux only
- CPython only
- Should have enough privileges
- Can be slow (again, what is “to be slow”? I honestly have
no idea)
19
![Page 20: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/20.jpg)
Enough on that, let’s get real- Starting the server:
python3 ./sanic-app.py 1>/dev/null 2>/dev/null
- (It does basically nothing, just “hello world” stuff)
20
![Page 21: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/21.jpg)
Enough on that, let’s get real- Stressing the server:
ab -n 300000 -c 300 http://127.0.0.1:8000/
- Please note that using ab for real world simulations is
totally insane, but this is exactly what we want now
21
![Page 22: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/22.jpg)
How slow is pyflame?- We run tests with and without pyflame
22
![Page 23: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/23.jpg)
Interactive demo time!
23
![Page 24: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/24.jpg)
Conclusions- Pyflame is surprisingly fast!
- Python is not slow, it’s just drawn that way
24
![Page 25: Python performance engineering in 2017](https://reader031.vdocument.in/reader031/viewer/2022022415/5a659b4e7f8b9ac2368b475b/html5/thumbnails/25.jpg)
Questions, please?- [email protected]
- Skype, Telegram: demeliorator
- Shameless advertising: http://t.me/lhommequipleure
25