profiling em python
DESCRIPTION
Slides da palestra sobre profiling em python dada na PythonBrasil[9] em 2013TRANSCRIPT
![Page 1: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/1.jpg)
globo.com Profiling em Python
Friday, October 4, 13
![Page 2: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/2.jpg)
porque profiling é importante?
Friday, October 4, 13
![Page 3: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/3.jpg)
Friday, October 4, 13
![Page 4: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/4.jpg)
Friday, October 4, 13
![Page 5: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/5.jpg)
então vamos otimizar tudo!
Friday, October 4, 13
![Page 6: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/6.jpg)
Friday, October 4, 13
![Page 7: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/7.jpg)
“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil” - Donald Knuth
Friday, October 4, 13
![Page 8: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/8.jpg)
from timeit import timeit
if __name__ == "__main__": setup = "from htmlmin.minify import html_minify;" setup += "from data import raw_html" t = timeit( stmt="html_minify(raw_html)", setup=setup, number=100) print(t)
benchmark.py
Friday, October 4, 13
![Page 9: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/9.jpg)
$ python benchmark.py
25.8121981621
Friday, October 4, 13
![Page 10: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/10.jpg)
conheça seu código
Friday, October 4, 13
![Page 11: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/11.jpg)
‣ cProfile
‣ Profile
‣ hotshot (deprecated)
‣ trace
‣ line profiler
‣ memory profiler
Friday, October 4, 13
![Page 12: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/12.jpg)
from data import raw_htmlfrom htmlmin.minify import html_minify
if __name__ == "__main__": html_minify(raw_html)
profile.py
Friday, October 4, 13
![Page 13: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/13.jpg)
$ python -m cProfile profile.py
Friday, October 4, 13
![Page 14: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/14.jpg)
Friday, October 4, 13
![Page 15: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/15.jpg)
Friday, October 4, 13
![Page 16: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/16.jpg)
Friday, October 4, 13
![Page 17: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/17.jpg)
Friday, October 4, 13
![Page 18: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/18.jpg)
Friday, October 4, 13
![Page 19: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/19.jpg)
Friday, October 4, 13
![Page 20: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/20.jpg)
Friday, October 4, 13
![Page 21: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/21.jpg)
Friday, October 4, 13
![Page 22: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/22.jpg)
Friday, October 4, 13
![Page 23: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/23.jpg)
Friday, October 4, 13
![Page 24: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/24.jpg)
$ python -m cProfile -o out profile.py
Friday, October 4, 13
![Page 25: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/25.jpg)
$ python -m cProfile -o out profile.py
Friday, October 4, 13
![Page 26: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/26.jpg)
>>> import pstats>>> p = pstats.Stats("out")>>> p.sort_stats("cumulative").print_stats(10)
315165 function calls (311828 primitive calls) in 1.334 seconds
Ordered by: cumulative time List reduced from 675 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function) 1 0.092 0.092 1.358 1.358 profile.py:2(<module>) 1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py:7(<module>) 1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py:17(<module>) 1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py:1(<module>) 1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py:2(<module>) 1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py:12(<module>) 1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py:26(html_minify) 2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py:80(__init__) 2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py:193(_feed) 2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py:33(feed)
Friday, October 4, 13
![Page 27: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/27.jpg)
>>> import pstats>>> p = pstats.Stats("out")>>> p.sort_stats("cumulative").print_stats(10)
315165 function calls (311828 primitive calls) in 1.334 seconds
Ordered by: cumulative time List reduced from 675 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function) 1 0.092 0.092 1.358 1.358 profile.py:2(<module>) 1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py:7(<module>) 1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py:17(<module>) 1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py:1(<module>) 1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py:2(<module>) 1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py:12(<module>) 1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py:26(html_minify) 2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py:80(__init__) 2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py:193(_feed) 2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py:33(feed)
Friday, October 4, 13
![Page 28: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/28.jpg)
>>> import pstats>>> p = pstats.Stats("out")>>> p.sort_stats("cumulative").print_stats(10)
315165 function calls (311828 primitive calls) in 1.334 seconds
Ordered by: cumulative time List reduced from 675 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function) 1 0.092 0.092 1.358 1.358 profile.py:2(<module>) 1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py:7(<module>) 1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py:17(<module>) 1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py:1(<module>) 1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py:2(<module>) 1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py:12(<module>) 1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py:26(html_minify) 2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py:80(__init__) 2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py:193(_feed) 2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py:33(feed)
Friday, October 4, 13
![Page 29: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/29.jpg)
>>> import pstats>>> p = pstats.Stats("out")>>> p.sort_stats("cumulative").print_stats(10)
315165 function calls (311828 primitive calls) in 1.334 seconds
Ordered by: cumulative time List reduced from 675 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function) 1 0.092 0.092 1.358 1.358 profile.py:2(<module>) 1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py:7(<module>) 1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py:17(<module>) 1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py:1(<module>) 1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py:2(<module>) 1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py:12(<module>) 1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py:26(html_minify) 2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py:80(__init__) 2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py:193(_feed) 2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py:33(feed)
Friday, October 4, 13
![Page 30: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/30.jpg)
>>> import pstats>>> p = pstats.Stats("out")>>> p.sort_stats("cumulative").print_stats(10)
315165 function calls (311828 primitive calls) in 1.334 seconds
Ordered by: cumulative time List reduced from 675 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function) 1 0.092 0.092 1.358 1.358 profile.py:2(<module>) 1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py:7(<module>) 1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py:17(<module>) 1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py:1(<module>) 1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py:2(<module>) 1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py:12(<module>) 1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py:26(html_minify) 2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py:80(__init__) 2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py:193(_feed) 2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py:33(feed)
Friday, October 4, 13
![Page 31: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/31.jpg)
>>> import pstats>>> p = pstats.Stats("out")>>> p.sort_stats("cumulative").print_stats(10)
315165 function calls (311828 primitive calls) in 1.334 seconds
Ordered by: cumulative time List reduced from 675 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function) 1 0.092 0.092 1.358 1.358 profile.py:2(<module>) 1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py:7(<module>) 1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py:17(<module>) 1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py:1(<module>) 1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py:2(<module>) 1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py:12(<module>) 1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py:26(html_minify) 2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py:80(__init__) 2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py:193(_feed) 2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py:33(feed)
Friday, October 4, 13
![Page 32: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/32.jpg)
>>> import pstats>>> p = pstats.Stats("out")>>> p.sort_stats("cumulative").print_stats(10)
315165 function calls (311828 primitive calls) in 1.334 seconds
Ordered by: cumulative time List reduced from 675 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function) 1 0.092 0.092 1.358 1.358 profile.py:2(<module>) 1 0.038 0.038 0.879 0.879 /(...)/htmlmin/minify.py:7(<module>) 1 0.001 0.001 0.840 0.840 /(...)/bs4/__init__.py:17(<module>) 1 0.073 0.073 0.839 0.839 /(...)/bs4/builder/__init__.py:1(<module>) 1 0.027 0.027 0.511 0.511 /(...)/bs4/builder/_html5lib.py:2(<module>) 1 0.028 0.028 0.483 0.483 /(...)/html5lib/__init__.py:12(<module>) 1 0.001 0.001 0.387 0.387 /(...)/htmlmin/minify.py:26(html_minify) 2 0.000 0.000 0.297 0.148 /(...)/bs4/__init__.py:80(__init__) 2 0.000 0.000 0.296 0.148 /(...)/bs4/__init__.py:193(_feed) 2 0.000 0.000 0.296 0.148 /(...)/bs4/builder/_html5lib.py:33(feed)
Friday, October 4, 13
![Page 33: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/33.jpg)
$ kernprof.py -l -v minify.py
Friday, October 4, 13
![Page 34: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/34.jpg)
Friday, October 4, 13
![Page 35: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/35.jpg)
Friday, October 4, 13
![Page 36: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/36.jpg)
Friday, October 4, 13
![Page 37: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/37.jpg)
Friday, October 4, 13
![Page 38: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/38.jpg)
Friday, October 4, 13
![Page 39: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/39.jpg)
Friday, October 4, 13
![Page 40: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/40.jpg)
Friday, October 4, 13
![Page 41: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/41.jpg)
$ python -m memory_profiler minify.py
Friday, October 4, 13
![Page 42: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/42.jpg)
Friday, October 4, 13
![Page 43: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/43.jpg)
Friday, October 4, 13
![Page 44: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/44.jpg)
Friday, October 4, 13
![Page 45: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/45.jpg)
Friday, October 4, 13
![Page 46: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/46.jpg)
Friday, October 4, 13
![Page 47: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/47.jpg)
gui?
Friday, October 4, 13
![Page 48: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/48.jpg)
Friday, October 4, 13
![Page 49: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/49.jpg)
Friday, October 4, 13
![Page 50: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/50.jpg)
outras ferramentas‣ meliae
‣ heapy (guppy)
‣ benchy
‣ valgrind
‣ python object graphs (objgraph)
‣ plop
‣ pycounters
Friday, October 4, 13
![Page 51: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/51.jpg)
bônus
Friday, October 4, 13
![Page 52: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/52.jpg)
django profiling
Friday, October 4, 13
![Page 53: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/53.jpg)
algumas ferramentas‣ django-debug-toolbar
‣ django-profiler
‣ new relic
Friday, October 4, 13
![Page 54: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/54.jpg)
Friday, October 4, 13
![Page 55: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/55.jpg)
$ newrelic-admin run-program gunicorn -w 3 wsgi:application
Friday, October 4, 13
![Page 56: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/56.jpg)
Friday, October 4, 13
![Page 57: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/57.jpg)
globo.com Estamos contratando!
Friday, October 4, 13
![Page 58: Profiling em Python](https://reader034.vdocument.in/reader034/viewer/2022051412/54b7a2f04a79591c048b4615/html5/thumbnails/58.jpg)
globo.com obrigada!@flaviamissi
Friday, October 4, 13