61A Lecture 13

Announcements

Measuring Efficiency

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(3)

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)fib(3)

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Recursive Computation of the Fibonacci Sequence

Our first example of tree recursion:

4

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

(Demo)

http://en.wikipedia.org/wiki/File:Fibonacci.jpg

def fib(n): if n <= 1: return 0 elif n == 1: return 1 else: return fib(n-2) + fib(n-1)

Memoization

Memoization

Idea: Remember the results that have been computed before

6

Memoization

Idea: Remember the results that have been computed before

def memo(f):

6

Memoization

Idea: Remember the results that have been computed before

def memo(f):

cache = {}

6

Memoization

Idea: Remember the results that have been computed before

def memo(f):

cache = {} def memoized(n):

6

Memoization

Idea: Remember the results that have been computed before

def memo(f):

cache = {} def memoized(n):

if n not in cache:

6

Memoization

Idea: Remember the results that have been computed before

def memo(f):

cache = {} def memoized(n):

if n not in cache: cache[n] = f(n)

6

Memoization

Idea: Remember the results that have been computed before

def memo(f):

cache = {} def memoized(n):

if n not in cache: cache[n] = f(n)

return cache[n]

6

Memoization

Idea: Remember the results that have been computed before

def memo(f):

cache = {} def memoized(n):

if n not in cache: cache[n] = f(n)

return cache[n] return memoized

6

Memoization

Idea: Remember the results that have been computed before

def memo(f):

cache = {} def memoized(n):

if n not in cache: cache[n] = f(n)

return cache[n] return memoized

Keys are arguments that map to return values

6

Memoization

Idea: Remember the results that have been computed before

def memo(f):

cache = {} def memoized(n):

if n not in cache: cache[n] = f(n)

return cache[n] return memoized

Keys are arguments that map to return values

Same behavior as f, if f is a pure function

6

Memoization

Idea: Remember the results that have been computed before

def memo(f):

cache = {} def memoized(n):

if n not in cache: cache[n] = f(n)

return cache[n] return memoized

Keys are arguments that map to return values

Same behavior as f, if f is a pure function

6

(Demo)

Memoized Tree Recursion

7

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Memoized Tree Recursion

7

Call to fibfib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Memoized Tree Recursion

7

Call to fib

Found in cachefib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Skipped

Space

The Consumption of Space

9

The Consumption of Space

Which environment frames do we need to keep during evaluation?

9

The Consumption of Space

Which environment frames do we need to keep during evaluation?

At any moment there is a set of active environments

9

The Consumption of Space

Which environment frames do we need to keep during evaluation?

At any moment there is a set of active environments

Values and frames in active environments consume memory

9

The Consumption of Space

Which environment frames do we need to keep during evaluation?

At any moment there is a set of active environments

Values and frames in active environments consume memory

Memory that is used for other values and frames can be recycled

9

The Consumption of Space

Which environment frames do we need to keep during evaluation?

At any moment there is a set of active environments

Values and frames in active environments consume memory

Memory that is used for other values and frames can be recycled

9

Active environments:

The Consumption of Space

Which environment frames do we need to keep during evaluation?

At any moment there is a set of active environments

Values and frames in active environments consume memory

Memory that is used for other values and frames can be recycled

9

Active environments:

• Environments for any function calls currently being evaluated

The Consumption of Space

Which environment frames do we need to keep during evaluation?

At any moment there is a set of active environments

Values and frames in active environments consume memory

Memory that is used for other values and frames can be recycled

9

Active environments:

• Environments for any function calls currently being evaluated

• Parent environments of functions named in active environments

The Consumption of Space

Which environment frames do we need to keep during evaluation?

At any moment there is a set of active environments

Values and frames in active environments consume memory

Memory that is used for other values and frames can be recycled

9

Active environments:

• Environments for any function calls currently being evaluated

• Parent environments of functions named in active environments

(Demo)

Interactive Diagram

Fibonacci Space Consumption

10

Fibonacci Space Consumption

10

fib(5)

Fibonacci Space Consumption

10

fib(5)

fib(3)

Fibonacci Space Consumption

10

fib(5)

fib(4)fib(3)

Fibonacci Space Consumption

10

fib(5)

fib(4)fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Fibonacci Space Consumption

10

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Fibonacci Space Consumption

10

Assume we have reached this step

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Fibonacci Space Consumption

11

Assume we have reached this step

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Fibonacci Space Consumption

11

Assume we have reached this step

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Has an active environment

Fibonacci Space Consumption

11

Assume we have reached this step

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Has an active environmentCan be reclaimed

Fibonacci Space Consumption

11

Assume we have reached this step

fib(5)

fib(4)

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

fib(2)

fib(0) fib(1)

0 1

fib(3)

fib(1)

1

fib(2)

fib(0) fib(1)

0 1

Has an active environmentCan be reclaimedHasn't yet been created

Time

Comparing Implementations

13

Comparing Implementations

Implementations of the same functional abstraction can require different resources

13

Comparing Implementations

Implementations of the same functional abstraction can require different resources

13

Problem: How many factors does a positive integer n have?

Comparing Implementations

Implementations of the same functional abstraction can require different resources

13

Problem: How many factors does a positive integer n have?

A factor k of n is a positive integer that evenly divides n

Comparing Implementations

Implementations of the same functional abstraction can require different resources

13

Problem: How many factors does a positive integer n have?

A factor k of n is a positive integer that evenly divides n

def factors(n):

Comparing Implementations

Implementations of the same functional abstraction can require different resources

13

Problem: How many factors does a positive integer n have?

A factor k of n is a positive integer that evenly divides n

Slow: Test each k from 1 through n

def factors(n):

Comparing Implementations

Implementations of the same functional abstraction can require different resources

13

Problem: How many factors does a positive integer n have?

A factor k of n is a positive integer that evenly divides n

Slow: Test each k from 1 through n

Fast: Test each k from 1 to square root n For every k, n/k is also a factor!

def factors(n):

Comparing Implementations

Implementations of the same functional abstraction can require different resources

13

Problem: How many factors does a positive integer n have?

A factor k of n is a positive integer that evenly divides n

Slow: Test each k from 1 through n

Fast: Test each k from 1 to square root n For every k, n/k is also a factor!

def factors(n):

Question: How many time does each implementation use division? (Demo)

Comparing Implementations

Implementations of the same functional abstraction can require different resources

13

Time (number of divisions)

Problem: How many factors does a positive integer n have?

A factor k of n is a positive integer that evenly divides n

Slow: Test each k from 1 through n

Fast: Test each k from 1 to square root n For every k, n/k is also a factor!

def factors(n):

Question: How many time does each implementation use division? (Demo)

n

Comparing Implementations

Implementations of the same functional abstraction can require different resources

13

Time (number of divisions)

Problem: How many factors does a positive integer n have?

A factor k of n is a positive integer that evenly divides n

Slow: Test each k from 1 through n

Fast: Test each k from 1 to square root n For every k, n/k is also a factor!

def factors(n):

Question: How many time does each implementation use division? (Demo)

n

Comparing Implementations

Implementations of the same functional abstraction can require different resources

13

Time (number of divisions)

Problem: How many factors does a positive integer n have?

A factor k of n is a positive integer that evenly divides n

Slow: Test each k from 1 through n

Fast: Test each k from 1 to square root n For every k, n/k is also a factor!

def factors(n):

Greatest integer less thanpn

Question: How many time does each implementation use division? (Demo)

Orders of Growth

Order of Growth

15

Order of Growth

A method for bounding the resources used by a function by the "size" of a problem

15

Order of Growth

A method for bounding the resources used by a function by the "size" of a problem

15

n: size of the problem

Order of Growth

A method for bounding the resources used by a function by the "size" of a problem

15

n: size of the problem

R(n): measurement of some resource used (time or space)

R(n) = �(f(n))

Order of Growth

A method for bounding the resources used by a function by the "size" of a problem

15

n: size of the problem

R(n): measurement of some resource used (time or space)

R(n) = �(f(n))

Order of Growth

A method for bounding the resources used by a function by the "size" of a problem

15

n: size of the problem

R(n): measurement of some resource used (time or space)

means that there are positive constants k1 and k2 such that

R(n) = �(f(n))

k1 · f(n) � R(n) � k2 · f(n)

Order of Growth

A method for bounding the resources used by a function by the "size" of a problem

15

n: size of the problem

R(n): measurement of some resource used (time or space)

means that there are positive constants k1 and k2 such that

R(n) = �(f(n))

k1 · f(n) � R(n) � k2 · f(n)

Order of Growth

A method for bounding the resources used by a function by the "size" of a problem

15

n: size of the problem

R(n): measurement of some resource used (time or space)

means that there are positive constants k1 and k2 such that

for all n larger than some minimum m

R(n) = �(f(n))

k1 · f(n) � R(n) � k2 · f(n)

Order of Growth

A method for bounding the resources used by a function by the "size" of a problem

15

n: size of the problem

R(n): measurement of some resource used (time or space)

means that there are positive constants k1 and k2 such that

for all n larger than some minimum m

R(n) = �(f(n))

k1 · f(n) � R(n) � k2 · f(n)

Order of Growth

A method for bounding the resources used by a function by the "size" of a problem

15

n: size of the problem

R(n): measurement of some resource used (time or space)

means that there are positive constants k1 and k2 such that

for all n larger than some minimum m

Order of Growth of Counting Factors

Implementations of the same functional abstraction can require different amounts of time

16

Problem: How many factors does a positive integer n have?

A factor k of n is a positive integer that evenly divides n

Slow: Test each k from 1 through n

Fast: Test each k from 1 to square root n For every k, n/k is also a factor!

def factors(n):

Order of Growth of Counting Factors

Implementations of the same functional abstraction can require different amounts of time

16

Problem: How many factors does a positive integer n have?

A factor k of n is a positive integer that evenly divides n

Slow: Test each k from 1 through n

Fast: Test each k from 1 to square root n For every k, n/k is also a factor!

def factors(n): Time Space

Order of Growth of Counting Factors

Implementations of the same functional abstraction can require different amounts of time

16

Problem: How many factors does a positive integer n have?

A factor k of n is a positive integer that evenly divides n

Slow: Test each k from 1 through n

Fast: Test each k from 1 to square root n For every k, n/k is also a factor!

def factors(n): Time Space

�(n) �(1)

Order of Growth of Counting Factors

Implementations of the same functional abstraction can require different amounts of time

16

Problem: How many factors does a positive integer n have?

A factor k of n is a positive integer that evenly divides n

Slow: Test each k from 1 through n

Fast: Test each k from 1 to square root n For every k, n/k is also a factor!

def factors(n): Time Space

�(n) �(1)

⇥(pn) �(1)

Order of Growth of Counting Factors

Implementations of the same functional abstraction can require different amounts of time

16

Problem: How many factors does a positive integer n have?

A factor k of n is a positive integer that evenly divides n

Slow: Test each k from 1 through n

Fast: Test each k from 1 to square root n For every k, n/k is also a factor!

def factors(n): Time Space

�(n) �(1)

⇥(pn) �(1)

Assumption: integers occupy a fixed amount of

space

Order of Growth of Counting Factors

Implementations of the same functional abstraction can require different amounts of time

16

Problem: How many factors does a positive integer n have?

A factor k of n is a positive integer that evenly divides n

Slow: Test each k from 1 through n

Fast: Test each k from 1 to square root n For every k, n/k is also a factor!

def factors(n): Time Space

�(n) �(1)

⇥(pn) �(1)

Assumption: integers occupy a fixed amount of

space

(Demo)

Exponentiation

Exponentiation

18

Exponentiation

Goal: one more multiplication lets us double the problem size

18

Exponentiation

Goal: one more multiplication lets us double the problem size

18

def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)

bn =

�1 if n = 0

b · bn�1 otherwise

Exponentiation

Goal: one more multiplication lets us double the problem size

18

def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)

bn =

�1 if n = 0

b · bn�1 otherwise

bn =

���

��

1 if n = 0

(b12 n)2 if n is even

b · bn�1 if n is odd

Exponentiation

Goal: one more multiplication lets us double the problem size

18

def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)

bn =

�1 if n = 0

b · bn�1 otherwise

bn =

���

��

1 if n = 0

(b12 n)2 if n is even

b · bn�1 if n is odd

Exponentiation

Goal: one more multiplication lets us double the problem size

18

def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)

def square(x): return x*x

def exp_fast(b, n): if n == 0: return 1 elif n % 2 == 0: return square(exp_fast(b, n//2)) else: return b * exp_fast(b, n-1)

bn =

�1 if n = 0

b · bn�1 otherwise

bn =

���

��

1 if n = 0

(b12 n)2 if n is even

b · bn�1 if n is odd

Exponentiation

Goal: one more multiplication lets us double the problem size

18

def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)

def square(x): return x*x

def exp_fast(b, n): if n == 0: return 1 elif n % 2 == 0: return square(exp_fast(b, n//2)) else: return b * exp_fast(b, n-1)

(Demo)

Exponentiation

19

Time Space

Goal: one more multiplication lets us double the problem size

def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)

def square(x): return x*x

def exp_fast(b, n): if n == 0: return 1 elif n % 2 == 0: return square(exp_fast(b, n//2)) else: return b * exp_fast(b, n-1)

Exponentiation

19

Time Space

�(n) �(n)

Goal: one more multiplication lets us double the problem size

def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)

def square(x): return x*x

def exp_fast(b, n): if n == 0: return 1 elif n % 2 == 0: return square(exp_fast(b, n//2)) else: return b * exp_fast(b, n-1)

Exponentiation

19

Time Space

�(n) �(n)

�(log n) �(log n)

Goal: one more multiplication lets us double the problem size

def exp(b, n): if n == 0: return 1 else: return b * exp(b, n-1)

def square(x): return x*x

def exp_fast(b, n): if n == 0: return 1 elif n % 2 == 0: return square(exp_fast(b, n//2)) else: return b * exp_fast(b, n-1)

Comparing Orders of Growth

Properties of Orders of Growth

21

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

⇥(n)

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

⇥(n) ⇥(500 · n)

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

⇥(n) ⇥(500 · n) ⇥(1

500· n)

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

Logarithms: The base of a logarithm does not affect the order of growth of a process

⇥(n) ⇥(500 · n) ⇥(1

500· n)

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

Logarithms: The base of a logarithm does not affect the order of growth of a process

⇥(n) ⇥(500 · n) ⇥(1

500· n)

⇥(log2 n)

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

Logarithms: The base of a logarithm does not affect the order of growth of a process

⇥(n) ⇥(500 · n) ⇥(1

500· n)

⇥(log2 n) ⇥(log10 n)

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

Logarithms: The base of a logarithm does not affect the order of growth of a process

⇥(n) ⇥(500 · n) ⇥(1

500· n)

⇥(log2 n) ⇥(log10 n) ⇥(lnn)

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

Logarithms: The base of a logarithm does not affect the order of growth of a process

Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps

⇥(n) ⇥(500 · n) ⇥(1

500· n)

⇥(log2 n) ⇥(log10 n) ⇥(lnn)

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

Logarithms: The base of a logarithm does not affect the order of growth of a process

Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps

⇥(n) ⇥(500 · n) ⇥(1

500· n)

⇥(log2 n) ⇥(log10 n) ⇥(lnn)

def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

Logarithms: The base of a logarithm does not affect the order of growth of a process

Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps

⇥(n) ⇥(500 · n) ⇥(1

500· n)

⇥(log2 n) ⇥(log10 n) ⇥(lnn)

def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count

Outer: length of a

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

Logarithms: The base of a logarithm does not affect the order of growth of a process

Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps

⇥(n) ⇥(500 · n) ⇥(1

500· n)

⇥(log2 n) ⇥(log10 n) ⇥(lnn)

def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count

Outer: length of a

Inner: length of b

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

Logarithms: The base of a logarithm does not affect the order of growth of a process

Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps

⇥(n) ⇥(500 · n) ⇥(1

500· n)

⇥(log2 n) ⇥(log10 n) ⇥(lnn)

def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count

Outer: length of a

Inner: length of b

If a and b are both length n, then overlap takes steps⇥(n2)

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

Logarithms: The base of a logarithm does not affect the order of growth of a process

Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps

⇥(n) ⇥(500 · n) ⇥(1

500· n)

⇥(log2 n) ⇥(log10 n) ⇥(lnn)

def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count

Outer: length of a

Inner: length of b

If a and b are both length n, then overlap takes steps⇥(n2)

Lower-order terms: The fastest-growing part of the computation dominates the total

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

Logarithms: The base of a logarithm does not affect the order of growth of a process

Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps

⇥(n) ⇥(500 · n) ⇥(1

500· n)

⇥(log2 n) ⇥(log10 n) ⇥(lnn)

def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count

Outer: length of a

Inner: length of b

If a and b are both length n, then overlap takes steps⇥(n2)

Lower-order terms: The fastest-growing part of the computation dominates the total

⇥(n2)

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

Logarithms: The base of a logarithm does not affect the order of growth of a process

Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps

⇥(n) ⇥(500 · n) ⇥(1

500· n)

⇥(log2 n) ⇥(log10 n) ⇥(lnn)

def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count

Outer: length of a

Inner: length of b

If a and b are both length n, then overlap takes steps⇥(n2)

Lower-order terms: The fastest-growing part of the computation dominates the total

⇥(n2 + n)⇥(n2)

Properties of Orders of Growth

Constants: Constant terms do not affect the order of growth of a process

21

Logarithms: The base of a logarithm does not affect the order of growth of a process

Nesting: When an inner process is repeated for each step in an outer process, multiply the steps in the outer and inner processes to find the total number of steps

⇥(n) ⇥(500 · n) ⇥(1

500· n)

⇥(log2 n) ⇥(log10 n) ⇥(lnn)

def overlap(a, b): count = 0 for item in a: if item in b: count += 1 return count

Outer: length of a

Inner: length of b

If a and b are both length n, then overlap takes steps⇥(n2)

Lower-order terms: The fastest-growing part of the computation dominates the total

⇥(n2 + n)⇥(n2) ⇥(n2+ 500 · n+ log2 n+ 1000)

Comparing orders of growth (n is the problem size)

22

Comparing orders of growth (n is the problem size)

22

�(bn)

Comparing orders of growth (n is the problem size)

22

�(bn) Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Comparing orders of growth (n is the problem size)

22

�(bn) Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Comparing orders of growth (n is the problem size)

22

�(bn)

⇥(n2)

Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Comparing orders of growth (n is the problem size)

22

�(bn)

⇥(n2)

Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Quadratic growth. E.g., overlap

Comparing orders of growth (n is the problem size)

22

�(bn)

⇥(n2)

Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n

Comparing orders of growth (n is the problem size)

22

�(bn)

�(n)

⇥(n2)

Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n

Comparing orders of growth (n is the problem size)

22

�(bn)

�(n)

⇥(n2)

Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Linear growth. E.g., slow factors or exp

Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n

Comparing orders of growth (n is the problem size)

22

�(bn)

�(n)

⇥(n2)

Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Linear growth. E.g., slow factors or exp

Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n

⇥(pn)

Comparing orders of growth (n is the problem size)

22

�(bn)

�(n)

⇥(n2)

Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Linear growth. E.g., slow factors or exp

Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n

⇥(pn) Square root growth. E.g., factors_fast

Comparing orders of growth (n is the problem size)

22

�(bn)

�(n)

�(log n)

⇥(n2)

Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Linear growth. E.g., slow factors or exp

Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n

⇥(pn) Square root growth. E.g., factors_fast

Comparing orders of growth (n is the problem size)

22

�(bn)

�(n)

�(log n)

⇥(n2)

Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Linear growth. E.g., slow factors or exp

Logarithmic growth. E.g., exp_fast

Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n

⇥(pn) Square root growth. E.g., factors_fast

Comparing orders of growth (n is the problem size)

22

�(bn)

�(n)

�(log n)

⇥(n2)

Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Linear growth. E.g., slow factors or exp

Logarithmic growth. E.g., exp_fastDoubling the problem only increments R(n).

Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n

⇥(pn) Square root growth. E.g., factors_fast

Comparing orders of growth (n is the problem size)

22

�(bn)

�(n)

�(log n)

�(1)

⇥(n2)

Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Linear growth. E.g., slow factors or exp

Logarithmic growth. E.g., exp_fastDoubling the problem only increments R(n).

Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n

⇥(pn) Square root growth. E.g., factors_fast

Comparing orders of growth (n is the problem size)

22

�(bn)

�(n)

�(log n)

�(1)

⇥(n2)

Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Linear growth. E.g., slow factors or exp

Logarithmic growth. E.g., exp_fastDoubling the problem only increments R(n).

Constant. The problem size doesn't matter

Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n

⇥(pn) Square root growth. E.g., factors_fast

Comparing orders of growth (n is the problem size)

22

�(bn)

�(n)

�(log n)

�(1)

⇥(n2)

Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Linear growth. E.g., slow factors or exp

Logarithmic growth. E.g., exp_fastDoubling the problem only increments R(n).

Constant. The problem size doesn't matter

Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n

⇥(pn) Square root growth. E.g., factors_fast

Comparing orders of growth (n is the problem size)

22

�(bn)

�(n)

�(log n)

�(1)

⇥(n2)

Exponential growth. Recursive fib takes

�(�n) � =1 +

�5

2� 1.61828steps, where

Incrementing the problem scales R(n) by a factor

Linear growth. E.g., slow factors or exp

Logarithmic growth. E.g., exp_fastDoubling the problem only increments R(n).

Constant. The problem size doesn't matter

Quadratic growth. E.g., overlapIncrementing n increases R(n) by the problem size n

⇥(pn) Square root growth. E.g., factors_fast