1

Computer AlgorithmsLecture 7

Master Theorem

Some of these slides are courtesy of D. Plaisted, UNC and M. Nicolescu, UNR

Recurrence Relations• Equation or an inequality that characterizes a function by its values on

smaller inputs.

• Solution Methods (Chapter 4)– Substitution Method.– Recursion-tree Method.– Master Method.

• Recurrence relations arise when we analyze the running time of iterative or recursive algorithms.– Ex: Divide and Conquer.

T(n) = (1) if n cT(n) = a T(n/b) + D(n) + C(n) otherwise

2

Master Method

• Based on the Master theorem.

• “Cookbook” approach for solving recurrences of the form T(n) = aT(n/b) + f(n)

• a 1, b > 1 are constants.– a subproblems of size n/b are solved recursively, each in time T(n/b)

• f(n) is asymptotically positive.– An asymptotically positive function is one that is positive for all– sufciently large n.– f(n) is the cost of dividing the problem and combining the results

• n/b may not be an integer, but we ignore floors and ceilings. Why?

• Requires memorization of three cases.• Idea: compare f(n) with

– f(n) is asymptotically smaller or larger than by a polynomial factor n

– f(n) is asymptotically equal with abnlog

abnlog

abnlog

3

Recursion tree view

Θ(1)

f(n)

f(n/b) f(n/b) f(n/b)

f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2)

a

a a a…

… … …

a a a a a a

… … … … … …

Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1)

f(n)

af(n/b)

a2f(n/b2)

(nlogba)

Total:

1log

0

log )/()()(n

j

jjab

b bnfannTThere are leaveslog logb bn aa n

Split problem into a parts at logbn levels.

4

The master method

The master method applies to recurrences of the form

T(n) = a T(n/b) + f (n) ,

where a ³ 1, b > 1, and f is asymptotically positive.

5

Why ?

)()( nfb

naTnT

b

naTnT )(

• Assume n = bk k = logbn

• At the end of iterations, i = k:• to solve the recurrence, we need only to characterize the dominant term

anni

in bbbb naTa

b

bTanT loglogloglog )1()(

22

b

nTa

33

b

nTa

ib

nTanT

ii

)(

abnlog

Master Theorem

Theorem 4.1

Let a 1 and b > 1 be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b. T(n) can be bounded asymptotically in three cases:

1. If f(n) = O(nlogba–) for some constant > 0, then T(n) = (nlogba).

2. If f(n) = (nlogba), then T(n) = (nlogbalg n).3. If f(n) = (nlogba+) for some constant > 0,

and if, for some constant c < 1 and all sufficiently large n,

we have a·f(n/b) c f(n), then T(n) = (f(n)).

Theorem 4.1

Let a 1 and b > 1 be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b. T(n) can be bounded asymptotically in three cases:

1. If f(n) = O(nlogba–) for some constant > 0, then T(n) = (nlogba).

2. If f(n) = (nlogba), then T(n) = (nlogbalg n).3. If f(n) = (nlogba+) for some constant > 0,

and if, for some constant c < 1 and all sufficiently large n,

we have a·f(n/b) c f(n), then T(n) = (f(n)).

Whether the problem cost is dominated by the root or by the leaves

7

Master Theorem – What it means?

• Case 1: If f(n) = O(nlogba–) for some constant > 0,

then T(n) = (nlogba).– nlogba = alogbn : Number of leaves in the recursion tree.– f(n) = O(nlogba–) Sum of the cost of the nodes at each internal

level asymptotically smaller than the cost of leaves by a polynomial factor.

– Cost of the problem dominated by leaves, hence cost is (nlogba).

8

Master Theorem – What it means?

• Case 2: If f(n) = (nlogba), then T(n) = (nlogbalg n).– nlogba = alogbn : Number of leaves in the recursion tree.– f(n) = (nlogba) Sum of the cost of the nodes at each level

asymptotically the same as the cost of leaves.– There are (lg n) levels.

– Hence, total cost is (nlogba lg n).

9

Master Theorem – What it means?

• Case 3: If f(n) = (nlogba+) for some constant > 0, and if, for some constant c < 1 and all sufficiently large n, we have a·f(n/b) c f(n), then T(n) = (f(n)).

– nlogba = alogbn : Number of leaves in the recursion tree.– f(n) = (nlogba+) Cost is dominated by the root. Cost of the root is

asymptotically larger than the sum of the cost of the leaves by a polynomial factor.

– Hence, cost is (f(n)).

10

Summary)()( nf

b

naTnT

b

naTnT )(

• Assume n = bk k = logbn

• At the end of iterations, i = k:• to solve the recurrence, we need only to

characterize the dominant term

anni

in bbbb naTa

b

bTanT loglogloglog )1()(

22

b

nTa

33

b

nTa

ib

nTanT

ii

)(

• Case 1: Running time dominated by

cost at leaves

– If f(n) is dominated by :

• T(n) = ( )

• Case 3: Running evenly distributed

throughout the tree

– If f(n) dominates :

• T(n) = (f(n))

• Case 2: Running time dominated by

cost at root

– If f(n) = ( ):• T(n)

abnlog

abnlog

abnlog

)log( log nn ab

abnlog

11

Strategy

• Extract a, b, and f(n) from a given recurrence• Determine • Compare f(n) and asymptotically • Determine appropriate MT case, and apply

logb anlogb an

12

Master Method – Examples

• T(n) = 16T(n/4)+n– a = 16, b = 4, nlogba = nlog416 = n2.– f(n) = n = O(nlogba-) = O(n2- ), where = 1 Case 1.

– Hence, T(n) = (nlogba ) = (n2).

• T(n) = T(3n/7) + 1– a = 1, b=7/3, and nlogba = nlog 7/3 1 = n0 = 1– f(n) = 1 = (nlogba) Case 2.

– Therefore, T(n) = (nlogba lg n) = (lg n)

13

Master Method – Examples

• T(n) = 3T(n/4) + n lg n– a = 3, b=4, thus nlogba = nlog43 = O(n0.793)– f(n) = n lg n = (nlog43 + ) where 0.2 Case 3.– Therefore, T(n) = (f(n)) = (n lg n).

• T(n) = 2T(n/2) + n lg n– a = 2, b=2, f(n) = n lg n, and nlogba = nlog22 = n– f(n) is asymptotically larger than nlogba, but not polynomially larger.

The ratio lg n is asymptotically less than n for any positive . Thus, the Master Theorem doesn’t apply here.

14

Master Theorem Summarized

• Given a recurrence of the form

• The master method cannot solve every recurrence of this form; there is a gap between cases 1 and 2, as well as cases 2 and 3

( ) ( / ) ( )T n aT n b f n

log

log

log

log

log0

1. ( )

( )

2. ( )

( ) lg

3. ( ) and ( / ) ( ), for some 1,

( ) ( )

b

b

b

b

b

a

a

a

a

a

f n O n

T n n

f n n

T n n n

f n n af n b cf n c n n

T n f n

15

f (n/b)

Idea of master theorem

f (n/b) f (n/b)

T (1)

…

Recursion tree:

…

f (n)a

f (n/b2)f (n/b2) f (n/b2)…

ah = logbn

f (n)

a f (n/b)

a2 f (n/b2)

…

nlogbaT (1)CASE 1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight.

CASE 1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight.

Q(nlogba)

16

f (n/b)

Idea of master theorem

f (n/b) f (n/b)

T (1)

…

Recursion tree:

…

f (n)a

f (n/b2)f (n/b2) f (n/b2)…

ah = logbn

f (n)

a f (n/b)

a2 f (n/b2)

…

nlogbaT (1)CASE 2: (k = 0) The weight is approximately the same on each of the logbn levels.

CASE 2: (k = 0) The weight is approximately the same on each of the logbn levels.

Q(nlogbalg n)

17

f (n/b)

Idea of master theorem

f (n/b) f (n/b)

T (1)

…

Recursion tree:

…

f (n)a

f (n/b2)f (n/b2) f (n/b2)…

ah = logbn

f (n)

a f (n/b)

a2 f (n/b2)

…

nlogbaT (1)CASE 3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight.

CASE 3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight.

Q( f (n))

18

More Examples

19

Master Method Case 1

T(n) = a T(n/b) + f(n)

f(n) = O(nlogb a - ) for some >0 T(n) = (nlogb a)

T(n) = (nlg 7)

cn2 =? O(nlogb a - ) = O(nlog2 7 - ) O(n2.8 - )

Yes, for any 0.8.

T(n) = 7T(n/2) + cn2 a=7, b=2

20

Master Method Case 2

T(n) = a T(n/b) + f(n)

f(n) = (nlogb a) T(n) = (nlogb a lg n)

T(n) = 2T(n/2) + cn a=2, b=2E.g., mergesort.

cn =? (nlogb a) = (nlog2 2) = (n)

Yes.

T(n) = (n lg n)21

Master Method Case 3T(n) = a T(n/b) + f(n)

f(n) = (nlogb a + ) for some >0 and

af(n/b) cf(n) for some c<1 and all large enough n

T(n) = (f(n))

T(n) = 4T(n/2) + n3 a=4, b=2

n3 =? (nlogb a + ) = (nlog2 4 + ) = (n2 + )

Yes, for any 1.

4(n/2)3 = ½n3 ? cn3

Yes, for any c ½.

I.e., is the constant factor

shrinking?

T(n) = (n3)22

Master Method Case 4

T(n) = a T(n/b) + f(n)

None of previous apply. Master method doesn’t help.

T(n) = 4T(n/2) + n2/lg n a=4, b=2

Case 1?

n2/lg n =? O(nlogb a - ) = O(nlog2 4 - ) = O(n2 - ) = O(n2/n)

No, since lg n is asymptotically less than n.

Thus, n2/lg n is asymptotically greater than n2/n.23

Examples

Ex. T(n) = 4T(n/2) + na = 4, b = 2 nlogba = n2; f (n) = n.CASE 1: f (n) = O(n2 – e) for e = 1. T(n) = Q(n2).

Ex. T(n) = 4T(n/2) + n2

a = 4, b = 2 nlogba = n2; f (n) = n2. CASE 2: f (n) = Q(n2lg0n), that is, k = 0. T(n) = Q(n2lg n).

24

Examples

Ex. T(n) = 4T(n/2) + n3

a = 4, b = 2 nlogba = n2; f (n) = n3. CASE 3: f (n) = W(n2 + e) for e = 1and 4(n/2)3 £ cn3 (reg. cond.) for c = 1/2. T(n) = Q(n3).

25

Master Method Case 4

T(n) = a T(n/b) + f(n)

None of previous apply. Master method doesn’t help.

T(n) = 4T(n/2) + n2/lg n a=4, b=2

Case 2?

n2/lg n =? (nlogb a) = (nlog2 4) = (n2)

No.

26

Master Method Case 4

T(n) = a T(n/b) + f(n)

None of previous apply. Master method doesn’t help.

T(n) = 4T(n/2) + n2/lg n a=4, b=2

Case 3?

n2/lg n =? (nlogb a + ) = (nlog2 4 + ) = (n2 + )

No, since 1/lg n is asymptotically less than n.

27

Examples

2

3

log 1

log 9

2

( ) ( / 2) 1

1, 2; 1

also ( ) 1, ( ) (1)

( ) (lg )

(

Case 2:

Cas

) 9 ( / 3)

9, 3;

( ) , ( ) ( ) with 1

( )e 1:

T n T n

a b n

f n f n

T n n

T n T n n

a b

f n n f n O n

T n n

Binary-search(A, p, r, s): q¬(p+r)/2 if A[q]=s then return q else if A[q]>s then Binary-search(A, p, q-1, s) else Binary-search(A, q+1, r, s)

Binary-search(A, p, r, s): q¬(p+r)/2 if A[q]=s then return q else if A[q]>s then Binary-search(A, p, q-1, s) else Binary-search(A, q+1, r, s)

28

Examples (2)

4

4

2

log 3 0.793

log 3

log 2 1

( ) 3 ( / 4) lg

3, 4;

( ) lg , ( ) ( ) with 0.2

Regularity condition

( / ) 3( / 4) l

Case

g( / 4) (3 / 4) lg ( ) for 3 / 4

( ) ( lg )

( ) 2 ( / 2) lg

2, 2;

3:

T n T n n n

a b n n

f n n n f n n

af n b n n n n cf n c

T n n n

T n T n n n

a b n n

f

1

1

( ) lg , ( ) ( ) with ?

also l

neither Case 3 nor Case 2!

g / lg

n n n f n n

n n n n

29

Examples (3)

2

3

log 4 2

3 2

3

3 3

3 3

( ) 4 ( / 2)

4, 2;

( ) ; ( ) ( )

( )

Checking the regularity condition

4 ( /

Cas

2) ( )

4 / 8

/ 2

3/ 4

e 3:

1

T n T n n

a b n n

f n n f n n

T n n

f n cf n

n cn

n cn

c

30