introduction - blogs | seapro ling python -m cprofile [-o output file] [-s sort order] script.py...
TRANSCRIPT
![Page 2: Introduction - Blogs | SEAPro ling python -m cProfile [-o output file] [-s sort order] script.py examples/1 intro/test prof1.py 1 importmath 2 deffunction(arg): 3 res = [] 4 foriinrange(-10000000,](https://reader035.vdocument.in/reader035/viewer/2022070110/604a71a9ed35b3220000dc53/html5/thumbnails/2.jpg)
Why Python
• Easy!
• Fast development
• Nice, readable code
• Great for prototyping
• Many third party libraries
• Data analysis
• Many users!
A. Gomez 2 Introduction
![Page 3: Introduction - Blogs | SEAPro ling python -m cProfile [-o output file] [-s sort order] script.py examples/1 intro/test prof1.py 1 importmath 2 deffunction(arg): 3 res = [] 4 foriinrange(-10000000,](https://reader035.vdocument.in/reader035/viewer/2022070110/604a71a9ed35b3220000dc53/html5/thumbnails/3.jpg)
Data Types
Dynamic language, but also a strongly typed language
• Objects have a type, which is determined at runtime
• A variable is a value bound to a name: the value has a type, but thevariable doesn’t
• The interpreter keeps track of all variable types
• You can’t do anything that’s incompatible with the type of datayou’re working with:
– You can do ’string+string’ and it will concatenate the strings– You can do ’integer+integer’– You can’t do ’string+integer’
A. Gomez 3 Introduction
![Page 4: Introduction - Blogs | SEAPro ling python -m cProfile [-o output file] [-s sort order] script.py examples/1 intro/test prof1.py 1 importmath 2 deffunction(arg): 3 res = [] 4 foriinrange(-10000000,](https://reader035.vdocument.in/reader035/viewer/2022070110/604a71a9ed35b3220000dc53/html5/thumbnails/4.jpg)
Data Structures
Python List
• Dynamic arrays
• Indexed structure
• Items: Python objects
• Items of different types
• Insertion and deletion at random positions
A. Gomez 4 Introduction
![Page 5: Introduction - Blogs | SEAPro ling python -m cProfile [-o output file] [-s sort order] script.py examples/1 intro/test prof1.py 1 importmath 2 deffunction(arg): 3 res = [] 4 foriinrange(-10000000,](https://reader035.vdocument.in/reader035/viewer/2022070110/604a71a9ed35b3220000dc53/html5/thumbnails/5.jpg)
Data Structures
Dictionary
• Associative arrays (key - value pairs)
• Indexed by key (string or number)
• Key: unique
• Value: any Python object
• Main operation: store a value with some key and extract the valuegiven the key
A. Gomez 5 Introduction
![Page 6: Introduction - Blogs | SEAPro ling python -m cProfile [-o output file] [-s sort order] script.py examples/1 intro/test prof1.py 1 importmath 2 deffunction(arg): 3 res = [] 4 foriinrange(-10000000,](https://reader035.vdocument.in/reader035/viewer/2022070110/604a71a9ed35b3220000dc53/html5/thumbnails/6.jpg)
Python in HPC
• You’ll hear that Python is slow
– Is it Python or your code?– Remember: Python is easy → bad programming
• If it’s slow, why should you use it?
• If you already have a Python code, what should you do?
A. Gomez 6 Introduction
![Page 7: Introduction - Blogs | SEAPro ling python -m cProfile [-o output file] [-s sort order] script.py examples/1 intro/test prof1.py 1 importmath 2 deffunction(arg): 3 res = [] 4 foriinrange(-10000000,](https://reader035.vdocument.in/reader035/viewer/2022070110/604a71a9ed35b3220000dc53/html5/thumbnails/7.jpg)
![Page 8: Introduction - Blogs | SEAPro ling python -m cProfile [-o output file] [-s sort order] script.py examples/1 intro/test prof1.py 1 importmath 2 deffunction(arg): 3 res = [] 4 foriinrange(-10000000,](https://reader035.vdocument.in/reader035/viewer/2022070110/604a71a9ed35b3220000dc53/html5/thumbnails/8.jpg)
Python in Stampede
• python/2.7.3-epd-7.3.2 (deprecated)
• python/2.7.6
• python/2.7.9
You can install your own modules
1. python setup.py install --user
2. python setup.py install --home=<dir>
3. pip install --user module name
• You can use virtualenv
A. Gomez 8 Introduction
![Page 9: Introduction - Blogs | SEAPro ling python -m cProfile [-o output file] [-s sort order] script.py examples/1 intro/test prof1.py 1 importmath 2 deffunction(arg): 3 res = [] 4 foriinrange(-10000000,](https://reader035.vdocument.in/reader035/viewer/2022070110/604a71a9ed35b3220000dc53/html5/thumbnails/9.jpg)
Before We Begin
Connect to Stampedessh -Y [email protected]
Python Exercises
cp ∼train00/python-hpc.tar.gz .
tar -xzf python-hpc.tar.gz
module load intel/14.0.1.106
module load python/2.7.6
idev -t 2:00:00
A. Gomez 9 Introduction
![Page 10: Introduction - Blogs | SEAPro ling python -m cProfile [-o output file] [-s sort order] script.py examples/1 intro/test prof1.py 1 importmath 2 deffunction(arg): 3 res = [] 4 foriinrange(-10000000,](https://reader035.vdocument.in/reader035/viewer/2022070110/604a71a9ed35b3220000dc53/html5/thumbnails/10.jpg)
Profilingpython -m cProfile [-o output file] [-s sort order] script.py
examples/1 intro/test prof1.py
1 import math
2 def function(arg):
3 res = []
4 for i in range ( -10000000 , 10000000):
5 res.append(math.sqrt(abs(i+1)*arg **5))
6 return res
78 print sum(function (10.0))
1.33333333332e+13
60000004 function calls in 17.351 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall
1 0.141 0.141 17.351 17.351 test_prof1.py:1
1 12.835 12.835 17.039 17.039 test_prof1.py:2
20000000 1.312 0.000 1.312 0.000 {abs}
20000000 1.429 0.000 1.429 0.000 {math.sqrt}
20000000 1.463 0.000 1.463 0.000 {’append ’}
1 0.000 0.000 0.000 0.000 {’disable ’}
1 0.171 0.171 0.171 0.171 {sum}
A. Gomez 10 Introduction
![Page 11: Introduction - Blogs | SEAPro ling python -m cProfile [-o output file] [-s sort order] script.py examples/1 intro/test prof1.py 1 importmath 2 deffunction(arg): 3 res = [] 4 foriinrange(-10000000,](https://reader035.vdocument.in/reader035/viewer/2022070110/604a71a9ed35b3220000dc53/html5/thumbnails/11.jpg)
Profiling
line profiler
• Installation
pip install line profiler --user
export PATH=$PATH:$HOME/.local/bin
• Add @profile decorators to the functions you want to profile
examples/1 intro/test prof1b.py
1 from builtins import profile
2 import math
3 @profile
4 def function(arg):
5 res = []
6 for i in range ( -10000000 , 10000000):
7 res.append(math.sqrt(abs(i+1)*arg **5))
8 return res
910 print sum(function (10.0))
Run itkernprof -l -v test prof1b.py
A. Gomez 11 Introduction
![Page 12: Introduction - Blogs | SEAPro ling python -m cProfile [-o output file] [-s sort order] script.py examples/1 intro/test prof1.py 1 importmath 2 deffunction(arg): 3 res = [] 4 foriinrange(-10000000,](https://reader035.vdocument.in/reader035/viewer/2022070110/604a71a9ed35b3220000dc53/html5/thumbnails/12.jpg)
Profiling
examples/1 intro/test prof2.py
1 from builtins import profile
2 import math
3 @profile
4 def function(arg):
5 res = []
6 factor = arg**5
7 for i in xrange ( -10000000 , 10000000):
8 if i<0:
9 res.append(math.sqrt((-i+1)*factor))
10 else:
11 res.append(math.sqrt((i+1)*factor))
12 return res
1314 print sum(function (10.0))
Run itkernprof -l -v test prof2.py
A. Gomez 12 Introduction
![Page 13: Introduction - Blogs | SEAPro ling python -m cProfile [-o output file] [-s sort order] script.py examples/1 intro/test prof1.py 1 importmath 2 deffunction(arg): 3 res = [] 4 foriinrange(-10000000,](https://reader035.vdocument.in/reader035/viewer/2022070110/604a71a9ed35b3220000dc53/html5/thumbnails/13.jpg)
Profiling
examples/1 intro/test prof3.py
1 from builtins import profile
2 import math
3 @profile
4 def function(arg):
5 factor = arg **5
6 res = range ( -10000000 , 10000000)
7 res = map(abs , res)
8 res = [(x+1)*factor for x in res]
9 res = map(math.sqrt , res)
10 return res
1112 print sum(function (10.0))
Run itkernprof -l -v test prof3.py
A. Gomez 13 Introduction
![Page 14: Introduction - Blogs | SEAPro ling python -m cProfile [-o output file] [-s sort order] script.py examples/1 intro/test prof1.py 1 importmath 2 deffunction(arg): 3 res = [] 4 foriinrange(-10000000,](https://reader035.vdocument.in/reader035/viewer/2022070110/604a71a9ed35b3220000dc53/html5/thumbnails/14.jpg)
License
c©The University of Texas at Austin, 2015
This work is licensed under the Creative Commons Attribution Non-Commercial 3.0Unported License. To view a copy of this license, visithttp://creativecommons.org/licenses/by-nc/3.0/
When attributing this work, please use the following text: ”HPC Python”, TexasAdvanced Computing Center, 2015. Available under a Creative Commons AttributionNon-Commercial 3.0 Unported License.