lecture 5 today, how to solve recurrences we learned “guess and proved by induction” we also...
Post on 06-Jan-2018
214 Views
Preview:
DESCRIPTION
TRANSCRIPT
Lecture 5
Today, how to solve recurrences We learned “guess and proved by induction” We also learned “substitution” method Today, we learn the “master theorem”
More divide and conquer: closest pair problem matrix multiplication
Master Theorem
Theorem 4.1 (CLRS, Theorem 4.1) Let a ≥ 1 and b > 1 be constants. Let f(n) be a function and let T(n) be defined on the nonnegative integers by
T(n) = aT(n/b) + f(n). Then
10
largefor )()/( AND )(
)(
)(
)(
log)(
log
log
log
log
log
c
nncfbnafnnf
nnf
nOnf
nf
nn
n
nT
a
a
a
a
a
b
b
b
b
b
Note
Only apply to a particular family of recurrences.
f(n) is positive for large n. Key is to compare f(n) with nlog_b a Case 2, more general is f(n) = Θ( nlog_b a lgkn).
Then the result is T(n) = Θ( nlog_b a lgk+1n). Sometimes it does not apply. Ex. T(n) =
4T(n/2) + n2 /logn.
Proof ideas of Master Theorem Consider a tree with T(n) at the root, and apply the recursion to
each node, until we get down to T(1) at the leaves. The first recursion is T(n) = aT(n/b) + f(n), so assign a cost of f(n) to the root. At the next level we have “a” nodes, each with a cost of T(n/b). When we apply the recursion again, we get a cost of af(n/b) for all of these. At the next level we have a2 nodes, each with a cost of T(n/b2). We get a cost of a2f(n/b2). We continue down to T(1) at the leaves. There are alog_b n leaves and each costs Θ(1), which gives Θ(alog_b n). The total cost associated with f is Σ 0 ≤ i ≤ log_b n - 1 ai f(n/bi).
Thus T(n) = Θ(n log_b a) + Σ 0 ≤ i ≤ (log_b n) - 1 ai f(n/bi). The three cases now come from deciding which term is
dominant. In case (1), the Θ term is dominant. In case (2), the terms are roughly equal (but the second term has an extra lg n factor). In case (3), the f(n) term is dominant. The details are somewhat painful, but can be found in CLRS, pp. 76-84.
f (n/b)
Idea of master theorem
f (n/b) f (n/b)
(1)
…Recursion tree:
…
f (n) a
f (n/b2)f (n/b2) f (n/b2)…ah = logbn
f (n)
a f (n/b)
a2 f (n/b2)
…#leaves = ah
= alogbn
= nlogba
nlogba(1)
Three common cases
Compare f (n) with nlogba:1. f (n) = O(nlogba – ) for some constant > 0.
• f (n) grows polynomially slower than nlogba (by an n factor).
Solution: T(n) = (nlogba) .
f (n/b)
Idea of master theorem
f (n/b) f (n/b)
(1)
…Recursion tree:
…
f (n) a
f (n/b2)f (n/b2) f (n/b2)…ah = logbn
f (n)
a f (n/b)
a2 f (n/b2)
…
nlogba(1)CASE 1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight. (nlogba)
These functions increasefrom top to bottomgeometrically, hence weonly need to have thelast bottom term
Case 2
Compare f (n) with nlogba:
2. f (n) = (nlogba lgkn) for some constant k 0.• f (n) and nlogba grow at similar rates.• This is clear for k=0. For k>0, the intuition is
that lgk n factor remain for constant fraction of levels, hence sum to the following
Solution: T(n) = (nlogba lgk+1n) .
f (n/b)
Idea of master theorem
f (n/b) f (n/b)
(1)
…Recursion tree:
…
f (n) a
f (n/b2)f (n/b2) f (n/b2)…ah = logbn
f (n)
a f (n/b)
a2 f (n/b2)
…
nlogba(1)CASE 2: (k = 0) The weight is approximately the same on each of the logbn levels.
(nlogbalg n)
All levels same
Case 3, c<1, akf(n/bk) geometrically decreases hence = Θ(f(n))
Compare f (n) with nlogba:3. f (n) = (nlogba + ) for some constant > 0.
• f (n) grows polynomially faster than nlogba (by an n factor),
and f (n) satisfies the regularity condition that a f (n/b) c f (n) for some constant c < 1.Solution: T(n) = ( f (n) ) .
f (n/b)
Idea of master theorem
f (n/b) f (n/b)
(1)
…Recursion tree:
…
f (n) a
f (n/b2)f (n/b2) f (n/b2)…ah = logbn
f (n)
a f (n/b)
a2 f (n/b2)
…
nlogba(1)CASE 3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight. ( f (n))
af(n/b)<(1-ε)f(n)
Examples for the Master Theorem The Karatsuba recurrence has a = 3, b = 2, f(n) = cn. Then case
1 applies, and so T(n) = Θ(n l og_2 3 ), as we found. The mergesort recurrence has a = 2, b = 2, f(n) = n. Then case
2 applies, and so T(n) = Θ(n lg n). Finally, a recurrence like T(n) = 3T(n/2) + n2 gives rise to case 3.
In this case f(n) = n2, so 3f(n/2) = 3 (n/2)2 = (3/4) n2 ≤ c n2 for c = 3/4, and so T(n) = Θ(n2).
Note that the master theorem does not cover all cases. In particular, it does not cover the case
T(n) = 2 T(n/2) + n / lg n since then the only applicable case is case 3, but then the
inequality involving f does not hold.
Closest pair problem
Input: A set of points P = {p1,…, pn} in two
dimensions Output:
The pair of points pi, pj that minimize the Euclidean distance between them.
Distances Euclidean distance
1x 2x
2y
1y 11, yx
22 , yx
221
2212211 ,, yyxxyxyx
Closest Pair Problem
Closest Pair Problem
Divide and Conquer
O(n2) time algorithm is easy Assumptions:
No two points have the same x-coordinates No two points have the same y-coordinates
How do we solve this problem in 1 dimension? Sort the number and walk from left to right to find
minimum gap.
Divide and Conquer
Divide and conquer has a chance to do better than O(n2).
We can first sort the points by their x-coordinates and sort also by y-coordinates
Closest Pair Problem
Divide and Conquer for the Closest Pair Problem
Divide by x-median
Divide
Divide by x-median
L R
Conquer
Conquer: Recursively solve L and R
L R
1
2
Combination I
Take the smaller one of 1 , 2 : = min(1 , 2 )
L R
2
Combination IIIs there a point in L and a point in R whose distance is smaller than ?
= min(1 , 2 )
L R
Combination II
If the answer is “no” then we are done!!! If the answer is “yes” then the closest such
pair forms the closest pair for the entire set
How do we determine this?
Combination IIIs there a point in L and a point in R whose distance is smaller than ?
L R
Combination IIIs there a point in L and a point in R whose distance is smaller than ?
Need only to consider the narrow bandO(n) time
L R
Combination IIIs there a point in L and a point in R whose distance is smaller than ?
Denote this set by S, assume Sy is the sorted list of S by the y-coordinates.
L R
Combination II
There exists a point in L and a point in R whose distance is less than if and only if there exist two points in S whose distance is less than .
If S is the whole thing, did we gain anything?
CLAIM: If s and t in S have the property that ||s-t|| < , then s and t are within 15 positions of each other in the sorted list Sy.
Combination IIIs there a point in L and a point in R whose distance is smaller than ?
L R
There are at most one point in each box of size δ/2 by δ/2. Thus s and t cannot be too far apart.
Closest-Pair Preprocessing:
Construct Px and Py as sorted-list by x- and y-coordinates Closest-pair(P, Px,Py)
Divide Construct L, Lx , Ly and R, Rx , Ry
Conquer Let 1= Closest-Pair(L, Lx , Ly ) Let 2= Closest-Pair(R, Rx , Ry )
Combination Let = min(1 , 2 ) Construct S and Sy For each point in Sy, check each of its next 15 points down
the list If the distance is less than , update the as this smaller
distance
Complexity Analysis
Preprocessing takes O(n lg n) time Divide takes O(n) time Conquer takes 2 T(n/2) time Combination takes O(n) time T(n) = 2T(n/2) + cn
So totally takes O(n lg n) time.
Matrix Multiplication
Suppose we multiply two NxN matrices together. Regular method is NxNxN = N3 multiplications O(N3)
Can we Divide and Conquer?
C11 = A11*B11 + A12*B21 C12 = A11*B12 + A12*B22 C21 = A21*B11 + A22*B21 C22 = A21*B12 + A22*B22
Complexity : T(N) = 8T(N/2) + O(N2) = O(Nlog
28) = O(N3)
22211211
AAAA
A =
22211211
BBBB
B =
22211211
CCCC
C= A*B =
No improvement
Strassen’s Matrix Multiplication
P1 = (A11+ A22)(B11+B22) P2 = (A21 + A22) * B11 P3 = A11 * (B12 - B22) P4 = A22 * (B21 - B11) P5 = (A11 + A12) * B22 P6 = (A21 - A11) * (B11 + B12) P7 = (A12 - A22) * (B21 + B22)
C11 = P1 + P4 - P5 + P7
C12 = P3 + P5 C21 = P2 + P4 C22 = P1 + P3 - P2 + P6
And do this recursively as usual.
VolkerStrassen
Time analysis
T(n) = 7T(n/2) + O(n2 ) = 7logn by the Master Theorem =nlog7
=n2.81
Best bound: O(n2.376) by Coppersmith-Winograd.
Best known (trivial) lower bound: Ω(n2). Open: what is the true complexity of matrix
multiplication?
top related