fast python? don’t bother - russel · fast python? don’t bother russel winder...
TRANSCRIPT
Copyright © 2016 Russel Winder 1
Fast Python? Don’t Bother
Russel Winder
[email protected]://www.russel.org.uk
@russel_winder
Copyright © 2016 Russel Winder 2
There is a rumour I only come to PyCon UK to talk about languages other than Python…
Copyright © 2016 Russel Winder 3
…this is not true…
Copyright © 2016 Russel Winder 4
…well not entirely.
Copyright © 2016 Russel Winder 5
A gentle rant, but…
Copyright © 2016 Russel Winder 6
…a bit of a rant nonetheless.
Copyright © 2016 Russel Winder 7
Python is…● A dynamically typed (or not) language.
● Quite popular, in data science, and data visualisation.
● Quite popular for sys. admin.
● Quite popular for Web-y stuff.
● Having it’s development driven by IDEs.
Type signatures.
Copyright © 2016 Russel Winder 8
Slow
Copyright © 2016 Russel Winder 9
Python is not…● A high performance computational language.
● A language in which you can write parallel programs.
Copyright © 2016 Russel Winder 10
Copyright © 2016 Russel Winder 11
Maybe the GIL is a good thing,
it stops us using Python when we shouldn’t.
Copyright © 2016 Russel Winder 12
Why?
Copyright © 2016 Russel Winder 13
Parallelism
Copyright © 2016 Russel Winder 14
So how to do computation?● Cython
● Numba
● NumPy
Copyright © 2016 Russel Winder 15
Cython● Write Python and annotate the code to make it appear
like C so it is actually C by source-to-source translation.
Hats off to Pyrex.
Copyright © 2016 Russel Winder 16
Why not just write C if you want to write C?
Copyright © 2016 Russel Winder 17
Because C is a programming language for writing operating systems, not applications?
Copyright © 2016 Russel Winder 18
What we want is Fortran or even FORTRAN.
Copyright © 2016 Russel Winder 19
Or not.
Copyright © 2016 Russel Winder 20
Numba● Write Python code and decorate with decorator so as to
perform non-Python translation via LLVM to native code.
Copyright © 2016 Russel Winder 21
Does Python really havenative code semantics?
Copyright © 2016 Russel Winder 22
Does RPython really havenative code semantics?
Copyright © 2016 Russel Winder 23
No
Copyright © 2016 Russel Winder 24
NumPy● And the whole SciPy, Pandas, etc. kit and caboodle.
● A C subsystem, providing opaque data types and operations well integrated with Python as an API.
Copyright © 2016 Russel Winder 25
It’s just a C system with a nice API.
Lots of niceness to the API.
Copyright © 2016 Russel Winder 26
Is C good enough for computation
in the modern era?
Copyright © 2016 Russel Winder 27
No
Copyright © 2016 Russel Winder 28
Why?
Copyright © 2016 Russel Winder 29
Parallelism
Copyright © 2016 Russel Winder 30
Multi-multicore processors
GPGPU
Copyright © 2016 Russel Winder 31
OpenCL
Vulkan
CUDA
Copyright © 2016 Russel Winder 32
Actors
CSP
Dataflow
Data Parallelism
Copyright © 2016 Russel Winder 33
Parallel Processing● Threads and thread pools:
– C – but very low level.
– C++ – better than C.
– Java: Akka, Quasar, Hadoop, Apache Spark, GPars.
– D – actors and data parallelism.
● Partitioned Global Address Space (PGAS):– Chapel
– X10
There are man, many more:Pony, Nim, etc.
Copyright © 2016 Russel Winder 34
Where does Python fit in?
Copyright © 2016 Russel Winder 35
It doesn’t for the computation…
Copyright © 2016 Russel Winder 36
…but there is a way forward:
Copyright © 2016 Russel Winder 37
Microservices Architecture
Copyright © 2016 Russel Winder 38
Processes communicating over a network.
Processes on same computer communicating using inter-process communication.
Copyright © 2016 Russel Winder 39
Processes communicating is the only way pure Python code can create parallelism so it must be Pythonic.
Copyright © 2016 Russel Winder 40
Intermix Python processes with non-Python ones.
Copyright © 2016 Russel Winder 41
Intermix:● Python ● C
● C++
● Rust
● D
● Chapel
Copyright © 2016 Russel Winder 42
Peek at D and Chapel.
https://github.com/russel/Pi_Quadrature
Copyright © 2016 Russel Winder 43
Follow Up…
Short hands-on workshop:
Monday 2016-09-19T14:30+01:00Room A
Copyright © 2016 Russel Winder 44
Fast Python? Don’t Bother
Russel Winder
[email protected]://www.russel.org.uk
@russel_winder