web2day - speed up your python code going native
TRANSCRIPT
Cpu-bound intensive processing #Python #LLVM #Numba
iAdvize - @SamuelBerthe
SPEED UP YOUR PYTHON CODE GOING NATIVE
{ “about” : “me” }
UX explorer CoFound er ed
Happy DevOps \o/
Current status:
Hunger debugger in Python
2000+ CUSTOMERS 40+ ENERGY TOTAL (FR) Entega GmbH Gas Natural (ES) RWE (DE)
SERV ICES Pole Emploi La Poste DGFIP UWV
CARS Citroen Renault Toyota Fiat BMW
INSURANCE Amaguiz Direct Assurance Groupama Europ Assistance MAIF Asstel Smile.direct Malakoff Mederic AXA Maaf Mutuelle Générale
TELECOM Bouygues Telecom SRR (SFR) Canal+ Sosh - Orange
RETA IL Fnac Cdiscount Florajet Monoprix House of Fraser Newlook Ukais Darty Redoute GrosBill Electrodepot Worten Charlotte Olympia Devanlay 3 Suisses Matches Fashion
LUXURY Louis Vuitton Christian Dior Christofle Chanel Boucheron Bottega Veneta Christian Louboutin Lacoste
TRAVEL TUI DE Air France VSC Marmara Pierre et Vacances Air Austral Corsair Thomas Cook Thalys
F INANCE Banque Populaire Crédit Mutuel Boursorama Banque Monabanq Fortuneo Beobank Cofidis Cofinoga Sofinco Cetelem VIVUS AXA Banque Groupama Banque Natixis Banque Degroof
6 YEARS . 190+ TALENTS . 10+ NATIONALITIES . 9 CORE VALUES . 100+% GROWTH/Y . 5TH GPTW . FAST 50
“Python ecosystem is huge, but performances are really BAD”
Anonymous, StackOverflow
CPU-BOUND INTENSIVE PROCESSING PYTHON - LLVM - NUMBA 101
SAMUEL BERTHE
DATA SCIENCE , MACHINE LEARNING
CPU-BOUND INTENSIVE PROCESSING PYTHON - LLVM - NUMBA 101
SAMUEL BERTHE
VIDEO ENCODING
“PYTHON CAN BE FAST”
Me
CPU-BOUND INTENSIVE PROCESSING PYTHON - LLVM - NUMBA 101
SAMUEL BERTHE
RUN AT LOW LEVEL
C Extension ?
Cython ?
Pypy ?
NUMBA LOW-LEVEL RUNTIME BASED ON LLVM
Numba 0.26 - conda install numba
WHY IS LLVM USEFUL FOR YOUR PYTHON RUNTIME ?
LL… what ? C++
JAVA
HASKELL
GOLANG
ARM
X86
C++
INTERMEDIATE REPRESENTATION
Going NATIVE at RUNTIME
Just-in-time Execution(JIT)
MAN NUMBA
PURE-PYTHON Numpy addition
NUMBA Numpy addition
PERF ??? + GIF = BIMMM
From 300ms to 900μs
Nested calls - Inlining
Generators
MORE COMPLEX JOBS ?
@cuda.jit
THAT’S ALL ! NO MORE OBSCUR STUFF WITH CYTHON
CALL. REPEAT. JIT =
TRUTH IS IN BENCHMARK !
OBSERVED PERFORMANCES
CNUMBACYTHON
C EXTENSIONPYPY
PYTHON
X 1 X 2 X 3 X 4 X 5Speed-up indice
NOT READY FOR RECURSION
LIMITATIONS
LIMITATIONS
Exception handling Lists (partial)
`with` statement Dictionaries
About Numba dependencies
$ docker pull jupyter/datascience-notebook
Q/A @SamuelBerthe
github.com/samber
