python performance engineering in 2017

Python performance engineering in 2017

Alex Chistyakov, an engineer @ some company

Agenda- Is Python slow?

- How to profile a Python program?

- We are not going to talk about blockchain, sorry

2

Is Python slow?- Definitely, because of GIL and lack of proper JIT

3

Is Python slow?- Definitely, because of GIL and a lack of a proper JIT

compiler

4

Is Python slow?- Seriously, how do you define “to be slow”?

5

Okay, let’s do our homework finally- Vagrant + https://github.com/alexclear/piterpy2017

6

Okay, let’s do our homework finally- Vagrant + https://github.com/alexclear/piterpy2017

- 2-cores VirtualBox VM

7

Okay, let’s do our homework finally- Vagrant + https://github.com/alexclear/piterpy2017

- 2-cores VirtualBox VM

- Something called Sanic:

https://github.com/channelcat/sanic because it’s 2017

and 2.x era should have ended years ago

8

Okay, let’s do our homework finally- Vagrant + https://github.com/alexclear/piterpy2017

- 2-cores VirtualBox VM

- Something called Sanic:

https://github.com/channelcat/sanic because it’s 2017

and 2.x era should have ended years ago

- A load testing tool called ab because we love classics

9

What is a typical Python program?- https://goo.gl/Abz8LX (Please note that this link will

produce different results in 5 years)

- Something not too server-side (or a framework)

- Something scientifical

- Some utility

10

What is a typical Python program?- https://goo.gl/Abz8LX (Please note that this link will

produce different results in 5 years)

- Something not too server-side

- Something scientifical

- Some utility

- An async web application!

11

A crash course in modern profiling- Collect stack samples at all costs

12

A crash course in modern profiling- Collect stack samples at all costs

- Convert collected samples to Brendan Gregg’s

flamegraph.pl format

13

A crash course in modern profiling- Collect stack samples at all costs

- Convert collected samples to Brendan Gregg’s

flamegraph.pl format

- Create a flamegraph

14

Collecting samples

15

Collecting samples- Uber’s pyflame: https://github.com/uber/pyflame

16

A bit of internal dark magic- pyflame uses ptrace(2) system call

17

A bit of internal dark magic- pyflame uses ptrace(2) system call

- It attaches to a running process and traverses its memory

18

Limitations- Linux only

- CPython only

- Should have enough privileges

- Can be slow (again, what is “to be slow”? I honestly have

no idea)

19

Enough on that, let’s get real- Starting the server:

python3 ./sanic-app.py 1>/dev/null 2>/dev/null

- (It does basically nothing, just “hello world” stuff)

20

Enough on that, let’s get real- Stressing the server:

ab -n 300000 -c 300 http://127.0.0.1:8000/

- Please note that using ab for real world simulations is

totally insane, but this is exactly what we want now

21

How slow is pyflame?- We run tests with and without pyflame

22

Interactive demo time!

23

Conclusions- Pyflame is surprisingly fast!

- Python is not slow, it’s just drawn that way

24

Questions, please?- alexclear@gmail.com

- Skype, Telegram: demeliorator

- Shameless advertising: http://t.me/lhommequipleure

25