Download - October 4, 2011 Joe Cross
![Page 1: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/1.jpg)
Useful Python Techniques:A brief introduction to List Comprehensions,
Functional Programming, and Generators
October 4, 2011Joe Cross
![Page 2: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/2.jpg)
Contents1. Looping and List Comprehensions2. Filter, Map, Reduce (and lambda!)3. Functional Programming Basics + Decorators4. Generators
![Page 3: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/3.jpg)
1. Looping and List Comprehensions
for(int i = 0; i < 5; i++){
cout << witty_banter(i) << endl;
}
![Page 4: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/4.jpg)
Loopingcompared to Java/C++
Looping
for i in range(0, 10, 1): a /= 2.0 for i in xrange(10): a /= 2.0
for(int i = 0; i < 10; i++){ a /= 2.0;}
Python
Java/C++
![Page 5: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/5.jpg)
LoopingRange + Iter
Looping
aList = [0, 1, 2, 'hello', 2**-4] for i in range(len(aList)): print aList[i] for item in aList: print item
![Page 6: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/6.jpg)
Loopingstill not as fun as…
Looping
To double the values in a list and assign to a new variable:
winning_lottery_numbers = [0, 4, 3, 2, 3, 1] fake_lottery_numbers = []for i in range(len(winning_lottery_numbers)): fake_lottery_numbers.append(2 * winning_lottery_numbers[i]) fake_lottery_numbers = []for number in winning_lottery_numbers: fake_lottery_numbers.append(2 * number)
Even though it’s an improvement over the tedium of c++ et. al, we can still do better.
![Page 7: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/7.jpg)
List ComprehensionsWoohooo!
List Comprehensions
winning_lottery_numbers = [0, 4, 3, 2, 3, 1] fake_lottery_numbers = [2*n for n in winning_lottery_numbers]
List Comprehensions allow us to do all sorts of things:•Single-function single-line code•Apply a function to each item of an iterable•Filter using conditionals•Cleanly nest loops
Syntax:
[<expression> for <value> in <collection> if <condition>]
![Page 8: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/8.jpg)
List ComprehensionsDon’t nest too many!
List Comprehensions
Multi-variable functions in a single line using zip:
vec1 = [3, 10, 2]vec2 = [-20, 5, 1] dot_mul = [u*v for u, v in zip(vec1, vec2)]dot_prod = sum(dot_mul)
Filtering:
readings = [-1.2, 0.5, 12, 1.8, -9.0, 5.3] good_readings = [r for r in readings if r > 0]
Bad:
orig = [15, 30, 78, 91, 25] finals = [min(s, 100) for s in [f+5 for f in orig]]
![Page 9: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/9.jpg)
2. Filter, Map, Reduce
Life = map(evolution, irreducible complexity)assert(sum(Life) == 42)
![Page 10: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/10.jpg)
Filter
Filter, Map, Reduce
def isPos(number, lim = 1E-16): return number > lim >>> a = [-1,2,-3,4,-5,6,-7,8,-9,10]>>> filter(isPos, a)[2, 4, 6, 8, 10]
>>> filter(not isPos, a) Traceback (most recent call last): File "<pyshell#7>", line 1 filter(not isZero, a)TypeError: 'bool' object is not callable
Syntax:
result = filter(aFunction, aSequence)
![Page 11: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/11.jpg)
Filter + Lambda
Filter, Map, Reduce
def isPos(number, lim = 1E-16): return number > lim
>>> filter(lambda n: not isPos(n), a)[-1, -3, -5, -7, -9]
[fnName] = lambda [args]: expression
Syntax:
![Page 12: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/12.jpg)
Lambda vs. def
Filter, Map, Reduce
def add(x, y): return x + y Ladd = lambda x, y: x+y def printWords(): print "Words" LprintWords = lambda: print "Words"
![Page 13: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/13.jpg)
So… why use lambda?
Filter, Map, Reduce
When using verbose function declaration it is often the case that the function’sverbose declaration can be verbose, even for functions that don’t require such verbosity.
def ispos(n): return n > 0b = filter(ispos, aList)
Also, there are some valid concerns about namespace clutter and the like.Verbose verbose verbose.
b = filter(lambda n: n > 0, aList)
Vs.
b = []for a in aList: if a > 0: b.append(a)
Verbose
Not Verbose
![Page 14: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/14.jpg)
Map
Filter, Map, Reduce
Compare to list comprehension:
winning_lottery_numbers = [0, 4, 3, 2, 3, 1]
fake_lottery_numbers = [2*n for n in winning_lottery_numbers]
fake_lottery_numbers = map(lambda n: 2*n, winning_lottery_numbers)
1.2.
Syntax:
result = map(aFunction, aSequence)
![Page 15: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/15.jpg)
Reduce
Filter, Map, Reduce
Syntax:
result = reduce(aFunction, aSequence, [initial])
lambda factorial n: reduce(operator.mul, xrange(1, n))
NOTE: results get accumulated on the left, and new values applied to the right.so reduce(add, [1,2,3,4]) is processed as (((1+2)+3)4)
![Page 16: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/16.jpg)
3. Functional Programming + Decorators
This isn’t your dad’sProcedural (imperative) programming
![Page 17: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/17.jpg)
A Simple Example
def add(x, y): return x + y def sub(x, y): return x - y def mul(x, y): return x * y def div(x, y): return x / y def mod(x, y): return x % y
def op(fn, x, y): return fn(x, y)
Functional Programming
![Page 18: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/18.jpg)
Nested FunctionsSpeed vs. Obfuscation
Functional Programming
def randNoGap(min_, max_): #random.random() -> [0,1) v = random.random() return (max_ - min_) * v - min_ def randWithGap(min_, max_): s = random.random() v = randNoGap(min_, max_) if s < 0.5: return v else: return -v #Same conditional using Python’s #Ternary operator #return v if s < 0.5 else -v
def rand(min_, max_, hasGap = False): if hasGap: return randWithGap(min_, max_) else: return randNoGap(min_, max_)
![Page 19: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/19.jpg)
Nested FunctionsSpeed vs. Obfuscation (Continued)
Functional Programming
def randomExplosion(minv, maxv, n): particles = [] for _ in xrange(n): vx = rand(minv, maxv, True) vy = rand(minv, maxv, True) vz = rand(minv, maxv, True) vx2 = rand(minv, maxv, True) vy2 = rand(minv, maxv, True) vz2 = rand(minv, maxv, True) r = rand(0,255,False) g = rand(0,255,False) b = rand(0,255,False) mainParticle = [vx,vy,vz,r,g,b] secondParticle = [vx2,vy2,vz2,r,g,b] particleGroup = (mainParticle, secondParticle) particles.append(particleGroup) return particles
NO
![Page 20: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/20.jpg)
Nested FunctionsSpeed vs. Obfuscation (Continued)
Functional Programming
What we’d like to do:
velocities = [rndV() for _ in xrange(6)]
What it actually looks like:
velocities = [rand(minv,maxv,True) for i in xrange(6)]
With a functional wrapper, we re-map:
rndV -> make_rand_fnc(minv, maxv, True)
rndV() -> make_rand_fnc(minv, maxv, True)()
![Page 21: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/21.jpg)
Nested FunctionsSpeed vs. Obfuscation (Continued)
Functional Programming
def mkRand(min_, max_, hasGap = False): def wrapper(): return rand(min_, max_, hasGap) return wrapper
rand(minv, maxv, True)rand(minv, maxv, True)rand(minv, maxv, True) def rand(min_, max_, hasGap = False):
def randomExplosion(minv, maxv, n): rVel = mkRand(minv, maxv, True) rCol = mkRnd(0,255,False) for _ in xrange(n): vx = rVel() vy = rVel() vz = rVel() vx2 = rVel() vy2 = rVel() vz2 = rVel() r = rCol() g = rCol() b = rCol()
![Page 22: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/22.jpg)
Nested FunctionsSpeed vs. Obfuscation (Continued)
Functional Programming
def randomExplosion(minv, maxv, n): particles = [] rndV = mkRand(minv, maxv, True) rndC = mkRnd(0,255,False) for _ in xrange(n): velocities = [rndV() for i in xrange(6)] r,g,b = [rndC() for i in xrange(3)] mainParticle = velocities[:3] + [r,g,b] secondParticle = velocities[3:] + [r,g,b] particleGroup = (mainParticle, secondParticle) particles.append(particleGroup) return particles
![Page 23: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/23.jpg)
DecoratorsQuickly apply common tasks to methods
Decorators
Common pre + post function call tasks, such as:•Caching•Timing•Counting function calls•Access rights
@decoratordef myFunc(arg1): print “arg1: “, arg1 myFunc = decorator(myFunc)
@f1(arg)@f2def func(): pass def func(): passfunc = f1(arg)(f2(func))
![Page 24: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/24.jpg)
DecoratorsQuickly apply common tasks to methods
Decorators
def decorator(f): print "This line is run once during func = decorator(func)" def wrapper(*args, **kwargs): print "This line is executed just before the function is called" #Call the function ret = f(*args, **kwargs) print "This line is executed just after the function is called" #Return the function's return return ret return wrapper @decoratordef foo(bar): print bar
On running, we get this output:>>> ================================ RESTART ================================>>> This line is run once during func = decorator(func)>>> foo(1)This line is executed just before the function is called1This line is executed just after the function is called
![Page 25: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/25.jpg)
DecoratorsQuickly apply common tasks to methods
Decorators
Decorators using classesclass decorator(object): def __init__(self, f): print "This line is run once during func = decorator(func)" self.f = f def __call__(self, *args, **kwargs): print "This line is executed just before the function is called"
#Call the function ret = self.f(*args)
print "This line is executed just after the function is called"
#Return the function's return return ret
![Page 26: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/26.jpg)
DecoratorsQuickly apply common tasks to methods
Decorators
(Rough) Timing
import time class TIMED(object): def __init__(self, f): self.f = f
def __call__(self, *args): start = time.clock() ret = self.f(*args) stop = time.clock() print "{0}: {1} ms.".format(self.f.func_name, 1000*(stop-start)) return ret
![Page 27: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/27.jpg)
DecoratorsQuickly apply common tasks to methods
Decorators
@TIMEDdef euler(f, t0, y0, h): """ Euler's Method """ yn = y0 + h*f(t0,y0) return yn @TIMEDdef RK2(f, t0, y0, h): """ Heun's Method """ y_hat = y0 + h*f(t0,y0) yn = y0 + h/2.0*(f(t0,y0)+f(t0+h, y_hat)) return yn @TIMEDdef RK4(f, t0, y0, h): """ Standard RK4 """ k1 = f(t0, y0) k2 = f(t0+h/2.0,y0 + h*k1/2.0) k3 = f(t0+h/2.0,y0 + h*k2/2.0) k4 = f(t0+h/2.0,y0 + h*k3) yn = y0 + 1.0/6.0*h*(k1 + 2.0*k2 + 2.0*k3 + k4) return yn
![Page 28: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/28.jpg)
fns = [euler, RK2, RK3, RK4, jRK4]t0 = scipy.linspace(-1,1)y0 = scipy.ones(50)h = 0.025args = (f, t0, y0, h) for fn in fns: print fn(*args) print
DecoratorsQuickly apply common tasks to methods
Decorators
>>> euler: 0.0181114469778 ms.[ ... ] RK2: 0.041656328049 ms.[ ... ] RK3: 0.0606733473757 ms.[ ... ] RK4: 0.0745587900587 ms.[ ... ] jRKN: 0.00150928724815 ms.jRK4: 1.57358288492 ms.[ ... ]
![Page 29: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/29.jpg)
4. Generators
Memory-conscious patternsare kind of a big deal in scientific computing
![Page 30: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/30.jpg)
Binary Tree from Array(simplified interface)
class T(object): def __init__(self, values = None, index = 0): self.left = None self.right = None self.v = None if values is not None: self.loadValues(values, index) def loadValues(self, values, index): self.v = values[index] n = len(values) if index * 2 + 1 < n: self.left = T(values, index * 2 + 1) if index * 2 + 2 < n: self.right = T(values, index * 2 + 2)
Generators
![Page 31: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/31.jpg)
Guessing Game
def makeT(val, delta, levels, level = 0): if level < levels: t = T() t.v = val t.left = makeT(val-delta, delta/2, levels, level+1) t.right = makeT(val+delta, delta/2, levels, level+1) return t
Generators
![Page 32: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/32.jpg)
Clean Code
def inorder(t): if t: for v in inorder(t.left): yield v yield t.v for v in inorder(t.right): yield v
Generators
![Page 33: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/33.jpg)
Using our Generator
for v in inorder(a_tree): print v a = []for v in inorder(a_tree): a.append(v) b = [v for v in inorder(a_tree)]
Generators
![Page 34: October 4, 2011 Joe Cross](https://reader035.vdocument.in/reader035/viewer/2022062801/56814420550346895db0bd85/html5/thumbnails/34.jpg)
Questions?
“Yes, the slide with the code. What did that one do?”