web2day - speed up your python code going native

Cpu-bound intensive processing #Python #LLVM #Numba

iAdvize - @SamuelBerthe

SPEED UP YOUR PYTHON CODE GOING NATIVE

{ “about” : “me” }

UX explorer CoFound er ed

Happy DevOps \o/

Current status:

Hunger debugger in Python

2000+ CUSTOMERS 40+ ENERGY TOTAL (FR) Entega GmbH Gas Natural (ES) RWE (DE)

SERV ICES Pole Emploi La Poste DGFIP UWV

CARS Citroen Renault Toyota Fiat BMW

INSURANCE Amaguiz Direct Assurance Groupama Europ Assistance MAIF Asstel Smile.direct Malakoff Mederic AXA Maaf Mutuelle Générale

TELECOM Bouygues Telecom SRR (SFR) Canal+ Sosh - Orange

RETA IL Fnac Cdiscount Florajet Monoprix House of Fraser Newlook Ukais Darty Redoute GrosBill Electrodepot Worten Charlotte Olympia Devanlay 3 Suisses Matches Fashion

LUXURY Louis Vuitton Christian Dior Christofle Chanel Boucheron Bottega Veneta Christian Louboutin Lacoste

TRAVEL TUI DE Air France VSC Marmara Pierre et Vacances Air Austral Corsair Thomas Cook Thalys

F INANCE Banque Populaire Crédit Mutuel Boursorama Banque Monabanq Fortuneo Beobank Cofidis Cofinoga Sofinco Cetelem VIVUS AXA Banque Groupama Banque Natixis Banque Degroof

6 YEARS . 190+ TALENTS . 10+ NATIONALITIES . 9 CORE VALUES . 100+% GROWTH/Y . 5TH GPTW . FAST 50

“Python ecosystem is huge, but performances are really BAD”

Anonymous, StackOverflow

CPU-BOUND INTENSIVE PROCESSING PYTHON - LLVM - NUMBA 101

SAMUEL BERTHE

DATA SCIENCE , MACHINE LEARNING

CPU-BOUND INTENSIVE PROCESSING PYTHON - LLVM - NUMBA 101

SAMUEL BERTHE

VIDEO ENCODING

“PYTHON CAN BE FAST”

Me

CPU-BOUND INTENSIVE PROCESSING PYTHON - LLVM - NUMBA 101

SAMUEL BERTHE

RUN AT LOW LEVEL

C Extension ?

Cython ?

Pypy ?

NUMBA LOW-LEVEL RUNTIME BASED ON LLVM

Numba 0.26 - conda install numba

WHY IS LLVM USEFUL FOR YOUR PYTHON RUNTIME ?

LL… what ? C++

JAVA

HASKELL

GOLANG

ARM

X86

C++

INTERMEDIATE REPRESENTATION

Going NATIVE at RUNTIME

Just-in-time Execution(JIT)

MAN NUMBA

PURE-PYTHON Numpy addition

NUMBA Numpy addition

PERF ??? + GIF = BIMMM

From 300ms to 900μs

Nested calls - Inlining

Generators

MORE COMPLEX JOBS ?

@cuda.jit

THAT’S ALL ! NO MORE OBSCUR STUFF WITH CYTHON

CALL. REPEAT. JIT =

TRUTH IS IN BENCHMARK !

OBSERVED PERFORMANCES

CNUMBACYTHON

C EXTENSIONPYPY

PYTHON

X 1 X 2 X 3 X 4 X 5Speed-up indice

NOT READY FOR RECURSION

LIMITATIONS

LIMITATIONS

Exception handling Lists (partial)

`with` statement Dictionaries

About Numba dependencies

$ docker pull jupyter/datascience-notebook

Q/A @SamuelBerthe

github.com/samber