Complexity

• A decidable problem is computationally solvable.• But what resources are needed to solve the problem?

– How much time will it require?– How much memory will it require?

• In this chapter, we study time complexity.• Chapter 8 covers the space complexity of a problem.

– Space corresponds to memory.– We do not cover space complexity; this topic is rarely covered

• in introductory theory courses.

• Goal: transfer all n disks from peg A to peg C• Rules:

– move one disk at a time– never place larger disk above smaller one

• Recursive solution:– transfer n 1 disks from A to B– move largest disk from A to C– transfer n 1 disks from B to C

• Total number of moves:– T(n) 2T(n 1) 1

Towers of Hanoi

Towers of Hanoi (2)

• Recurrence relation:T(n) 2 T(n 1) 1T(1) 1

• Solution by unfolding:T(n) =2 (2 T(n - 2) + 1) + 1 = = 4 T(n - 2) + 2 + 1 = = 4 (2 T(n - 3) + 1) + 2 + 1 = = 8 T(n - 3) + 4 + 2 + 1 = ... = 2i T(n - i) + 2i-1 +2i-2 +...+21 +20

• the expansion stops when i n 1T(n) = 2n – 1 + 2n – 2 + 2n – 3 + ... + 21 + 20

Towers of Hanoi (3)

• This is a geometric sum, so that we haveT(n) 2n 1

• Good or bad news?– the Tibetans were confronted with a tower problem of 64

rings...– Assuming the priests move one ring per second, it would take

~585 billion years to complete the process!

Reasonable vs. UnreasonableProcessing 1 elements takes 0.01 sec

function/n

10 20 50 100 300

n2 1/10,000second

1/2,500second

1/400second

1/100second

9/100second

n5 1/10second

3.2seconds

5.2minutes

2.8hours

28.1 days

2n 1/1000second

1second

35.7years

400 trillioncenturies

a 75 digit-number of centuries

nn 2.8 hours

3.3 trillionyears

a 70 digit-number of centuries

a 185 digit-number of centuries

a 728 digit-number of centuries

Exponential

Polynomial

Goals

• Basics of time complexity theory• Introduce method for measuring the time needed to

solve a problem.• Show how to classify problems according to the amount

of time required.• Show that certain classes of problems require enormous

amounts of time.• We will also see how to determine whether we have this

type of problem.

Language vs. Problem

• We will use “Language” and “Problem” interchangeably

• Any Problem can be converted to a Language, and vice-versa– Problem: Determine if a number is prime– Language: L = {p : p is prime }

– Problem: Determine if a Turing Machine M halts on an input string w

– Language: L = {e(M), e(w) : M halts on w}

Calculating Running Time of a Program

• To determine the total time add up the time required for each statement.

• The running time of a simple statement is O(1).

• The running time of an if statement is the larger of the running times of the two branches, plus the time required to evaluate the condition

• The running time of a sequence of statements is the running time of the longest step.

• The running time of a loop is the time to execute one iteration times the number of iterations.

Why Algorithm Analysis

• Solve a problem on a computer seek an algorithm that makes efficient use of the computer resources – time – space

• Algorithm analysis - mathematical approach to describe (in terms of the problem size):– The amount of resources used by the algorithm

• Space• Computational (running) time

– Running time:• The number of primitive operations (steps) executed before

termination– Order of growth

• The leading term of a formula• Expresses the behavior of a function toward infinity lim

n

Slides

Asymptotic upper bound• Suppose f and g are two nonnegative functions• Definition [Big “oh”]:

– f(n)=O(g(n)) iff. there exist positive constants c and n0

such that f(n)≤ c • g(n) for all n, n≥ n0 .

• Examples– 3n+2=O(n) as 3n+2 ≤ 4n for n ≥ 2– 10n2+4n+2 = O(n2) as 10n2+4n+2 ≤ 11n2 for n ≥ 5

• g(n) should be as small as possible– f(n) = O(n2)– f(n) = O(n)

Logarithms

• The notation "ln(x)" means loge(x);

• The notation "lg(x)" means log10(x);

• The notation "lb(x)" means log2(x).

Logarithms

Towers of Hanoi (3)

• This is a geometric sum, so that we haveT(n) 2n 1 O(2n)

• The running time of this algorithm is exponential (kn) rather than polynomial (nk), where n is input size.

• Good or bad news?– the Tibetans were confronted with a tower problem

of 64 rings...– Assuming the priests move one ring per second, it

would take ~585 billion years to complete the process!