project 2 due … project 2 due … project 2 project 2
Post on 31-Dec-2015
258 Views
Preview:
TRANSCRIPT
Project 2 due …
Chapter 2
Divide and Conquer
Recurrence Relations
Equation or an inequality that describes a function by its values on smaller inputs.
Recurrence relations arise when we analyze the running time of iterative or recursive algorithms. Ex: Divide and Conquer.
T(n) = O(1) if n cT(n) = a T(n/b) + D(nd) otherwise
Solution Methods Substitution Method. Recursion-tree Method. Master Method.
Divide and Conquer Strategy
How does Divide and Conquer Strategy work?
Examples?
Another Example: Merge Sort
Sorting Problem: Sort a sequence or n elements into non-decreasing order.
Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each
Conquer: Sort the two subsequences recursively using merge sort.
Combine: Merge the two sorted subsequences to produce the sorted answer.
Merge Sort – Example
43 15 9 1 22 26 19 55 37 43 99 2
18 26 32 6 43 15 9 1 22 26 19 55 37 43 99 2
18 26 32 6 43 15 9 1 22 26 19 55 37 43 99 2
18 26 32 6 43 15 9 1 22 26 19 55 37 43 99 2
18 26 32 6 43 15 9 1 22 26 19 55 37 43 99 2
18 26 32 6
Merge Sort – Example
18 26 32 6 43 15 9 1
18 26 32 6 43 15 9 1
18 26 32 6 43 15 9 1
2618 6 32 1543 1 9
18 26 32 6 43 15 9 1
18 26 32 6 43 15 9 1
18 26 32 6 15 43 1 9
6 18 26 32 1 9 15 43
1 6 9 15 18 26 32 43
18 26
18 26
18 26
32
32
6
6
32 6
18 26 32 6
43
43
15
15
43 15
9
9
1
1
9 1
43 15 9 1
18 26 32 6 43 15 9 1
18 26 6 32
6 26 3218
1543 1 9
1 9 15 43
1 6 9 1518 26 32 43
Original Sequence Sorted Sequence
Analysis of Merge Sort
Running time T(n) of Merge Sort: Divide: computing the middle takes O(1) Conquer: solving 2 sub-problems takes
2T(n/2) Combine: merging n elements takes O(n) Total:
T(n) = O(1) if n = 1T(n) = 2T(n/2) + O(n) if n > 1
T(n) = ?
Binary searchAlgorithm:
if (no element in the array), return –1; else
check K against the middle value of the array
if (same), return position;else if (K is smaller)
binary search the 1st half of the array;
elsebinary search the 2st half of
the array;
T(n) = T(n/2) + 1;T(1) = 1;
? T(n) = ?
Multiplication
x=10011010, y = 10101011 x*y = ? – divide and conquer?
Running Time
Recurrence: T(n) = O(1) if n = 1 T(n) = 3T(n/2) + O(n)
if n > 1
T(n) = ?
Quicksort
http://en.wikipedia.org/wiki/Quicksort
Recurrence Relations
Equation or an inequality that describes a function by its values on smaller inputs.
Recurrence relations arise when we analyze the running time of iterative or recursive algorithms. Ex: Divide and Conquer.
T(n) = O(1) if n cT(n) = a T(n/b) + D(nd) otherwise
Solution Methods Substitution Method. Recursion-tree Method. Master Method.
Substitution Method
Guess the form of the solution, then use mathematical induction to show it correct.
Substitute guessed answer for the function when the inductive hypothesis is applied to smaller values.
Works well when the solution is easy to guess.
No general way to guess the correct solution.
Example –AsymptoticsTo Solve: T(n) = 2T(n/2) + n Guess: T(n) = O(n lg n) Need to prove: T(n) cn lg n, for some c > 0. Hypothesis: T(k) ck lg k, for all k < n. Calculate:
T(n) 2c n/2 lg n/2 + n
c n lg (n/2) + n
= c n lg n – c n lg2 + n
= c n lg n – n (c lg 2 – 1)
c n lg n
(The last step is true for c 1 / lg2.)
Exercises
Solution of T(n) = 3T(n/3) + n is
Solution of T(n) = T(n/3) + 1 is
Solve T(n) = 2T(n/2) + 1
O(nlogn)
O(lgn)
O(n)
Running Time
Recurrence: T(n) = O(1) if n = 1 T(n) = 3T(n/2) + O(n)
if n > 1
T(n) = ?
Recursion-tree Method Making a good guess is sometimes difficult
with the substitution method. Use recursion trees to devise good
guesses. Recursion Trees
Show successive expansions of recurrences using trees.
Keep track of the time spent on the subproblems of a divide and conquer algorithm.
Help organize the algebraic bookkeeping necessary to solve a recurrence.
Recursion Tree – Example
Running time of Merge Sort:T(n) = (1) if n = 1T(n) = 2T(n/2) + (n) if n > 1
Rewrite the recurrence asT(n) = c if n = 1 T(n) 2T(n/2) + cn if n > 1
c > 0: Running time for the base case and
time per array element for the divide and
combine steps.
Recursion Tree for Merge Sort
For the original problem, we have a cost of cn, plus two subproblems each of size (n/2) and running time T(n/2).
cn
T(n/2) T(n/2)
Each of the size n/2 problems has a cost of cn/2 plus two subproblems, each costing T(n/4).
cn
cn/2 cn/2
T(n/4) T(n/4) T(n/4) T(n/4)
Cost of divide and merge.
Cost of sorting subproblems.
Recursion Tree for Merge Sort
Continue expanding until the problem size reduces to 1.cn
cn/2 cn/2
cn/4 cn/4 cn/4 cn/4
c c c cc c
lg n
cn
cn
cn
cn
Total : cnlgn+cn
Other Examples
Use the recursion-tree method to determine a guess for the recurrences
T(n) = 4T(n/4)+4 T(n) = T(n/2) + 2T(n/3) + n
The Master Method
Based on the Master theorem. “Cookbook” approach for solving
recurrences of the form T(n) = aT(n/b) + f(nd)
a 1, b > 1 are constants. f(n) is asymptotically positive. n/b does not have to be an integer, but we
ignore floors and ceilings. Why?
Requires memorization of three cases.
Master Theorem
Master Theorem
Let a 1, b > 1 be constants, f(n) be a function. Let T(n) be defined on nonnegative integers by T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b.
Then T(n) can be bounded asymptotically in three cases:
1. If f(n) = O(nlogba–) for some > 0, then T(n) = (nlogba).
2. If f(n) = (nlogba), then T(n) = (nlogbalg n).
3. If f(n) = (nlogba+) for some constant > 0, and if, for some constant c < 1 and all sufficiently
large n, we have af(n/b) c f(n), then T(n) = (f(n)).
Master Method – Examples
T(n) = 16T(n/4)+n a = 16, b = 4, nlogba = nlog416 = n2. f(n) = n = O(nlogba-) = O(n2- ), where =
1 Case 1. Hence, T(n) = (nlogba ) = (n2).
T(n) = T(3n/7) + 1 a = 1, b=7/3, and nlogba = nlog 7/3 1 = n0 =
1 f(n) = 1 = (nlogba) Case 2. Therefore, T(n) = (nlogba lg n) = (lg n)
Master Method – Examples
T(n) = 3T(n/2) + n ?
Exercise
Which method? T(n) = 4T(n/3)+4 T(n) = 2T(n/2 + 17) + n T(n) = T(n/2) + 2T(n/4) + n T(n) = T(n-1)+n
Matrix Multiplications
Matrix multiplications
Example: AB=C
Let A,B,C be nn matrices. What’s the cost to obtain C ?
(assuming n is a power of 2)
ut
sr
hg
fe
dc
ba
r = ae + bgs = af + bht = ce + dgu = cf + dh
Cost of the direct method
It takes T(n/2) to obtain each of ae, bf, …dh.
It takes n/2n/2 additions to obtain r, s, t, or u.
Therefore:T(n) = 8T(n/2)+(n2)
= 8T(n/2)+cn2
= 8( 8T(n/22)+c(n/2)2 )+cn2
= ?
Cost of the direct method
It takes T(n/2) to obtain each of ae, bf, …dh.
It takes n/2n/2 additions to obtain r, s, t, or u.
Therefore:T(n) = 8T(n/2)+(n2)
= (n3)
Strassen’s Algorithm Strassen’s Algorithm: an undetermined
coefficient method. It’s based on that fact that A+B is much cheaper to calculate than AB. Outline of the proof: Let Pi = AiBi, i=1,…,7
where Ai = (i1a+i2b+i3c+i4d), Bi = (i1e+i2f+i3g+i4h), ij, ij {-1,0,1}.
Try to determine ij, ij such that r, s, t, u = ijPi, ij {-1,0,1}
Note: The hidden coefficients in front of nlog2
7
is larger than the one in front of n3.
Strassen’s Algorithm
Determine the coefficients ij, ij: A1 = a, A2 = a+b, A3 = c+d, A4 = d,
A5 = a+d, A6 = b d, A7 = a c B1 = f h, B2 = h, B3 = e, B4 = g e,
B5 = e+h, B6 = g+h, B7 = e+f
r = P5+P4-P2+P6
s = P1+P2
t = P3+P4
u = P5+P1P3 P7
Verify u=…
ut
sr
hg
fe
dc
ba
Cost for n=8 ?
Strassen’s Algorithm
Strassen’s Algorithm:T(n) = 7T(n/2)+(n2) → T(n)= (nlog
27) =
O(n2.8)
Strassen’s Algorithm
n2.8
n3
Insertion sort and selection sort
Recurrence: T(n) = O(1) if n = 1T(n) = T(n-1) + O(n) if n > 1
T(n) = ?
Complexity?No divide and conquer algorithms
top related