divide-and-conquer dr. b c dhara department of information technology jadavpur university
TRANSCRIPT
Divide-and-Divide-and-ConquerConquer
Dr. B C DharaDr. B C Dhara
Department of Information Department of Information TechnologyTechnology
Jadavpur UniversityJadavpur University
Divide-and-ConquerDivide-and-Conquer Divide-and conquerDivide-and conquer is a general is a general
algorithm design paradigm:algorithm design paradigm: BreakingBreaking the problem into several the problem into several
sub-problems that are similar to sub-problems that are similar to the original problem but smaller in the original problem but smaller in size,size,
Conquer the subproblems by solving them recursively (successively and independently), (successively and independently), and thenand then
CombineCombine these solutions to these solutions to subproblems to create a solution subproblems to create a solution to the original problem.to the original problem.
The base case for the recursion The base case for the recursion are subproblems of constant are subproblems of constant sizesize
Analysis can be done using Analysis can be done using recurrence equationsrecurrence equations
It is a top-down technique for
designing algorithms
Binary searchBinary search
Binary Search is an extremely well-known Binary Search is an extremely well-known instance of divide-and-conquer paradigm. instance of divide-and-conquer paradigm.
Given an ordered (increasing) array of n Given an ordered (increasing) array of n elements (a[1,…,n]), the basic idea of elements (a[1,…,n]), the basic idea of binary search is that for a given element binary search is that for a given element we "probe" the middle element of the array. we "probe" the middle element of the array.
We continue in either the lower or upper We continue in either the lower or upper segment of the array, depending on the segment of the array, depending on the outcome of the probe until we reached the outcome of the probe until we reached the required (given) element. required (given) element.
Binary searchBinary search Divide: middle = (low+high)/2Divide: middle = (low+high)/2
Conquer:Conquer:if(low>high) return 0;if(low>high) return 0;if (a[middle]==key) return key;if (a[middle]==key) return key;else else
if(a[middle]>key) search(a, low, if(a[middle]>key) search(a, low, middle-1);middle-1);
else search(a,middle+1,high);else search(a,middle+1,high); Combine: noneCombine: none
Binary searchBinary search
T(n) = T(n/2) + T(n) = T(n/2) + (1)(1)
Merge sortMerge sort The merge sort algorithm closely follows
the divide-and-conquer paradigm. Intuitively, it operates as follows.
Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each.
Conquer: Sort the two subsequences recursively using merge sort.
Combine: Merge the two sorted subsequences to produce the sorted answer.
Merge sortMerge sort
We note that the recursion “bottoms out” when the sequence to be sorted has length 1, in which case there is no work to be done, since every sequence of length 1 is already in sorted order.
Merge sortMerge sort
Input: An array A and indices p and rOutput: An sorted array AMERGE-SORT(A, p, r)1. if p < r2. then q = (p + r) / 23. MERGE-SORT(A, p, q)4. MERGE-SORT(A, q+1, r)5. MERGE(A, p, q, r)
Merge sortMerge sort MERGE(A, p, q, r)
1. i = p, j = q + 1, n = r - p + 1 2. for k = 1 to n // for 13. if [ (A[i] < A[j]) or (j > r)] and (i ≤ q) 4. B[k] = A[i]5. i = i +1 6. else7. B[k] = A[j]8. j = j +1
9. for k = 0 to n – 110. A[p + k] = B[k]
Complexity of merge sort is T(n)= T(n/2) + T(n/2) + (n)(n)
Quick sortQuick sort
Divide: partition the data set into two Divide: partition the data set into two subsets respect to the pivot elementsubsets respect to the pivot element
Conquer: recursively use the quick sort Conquer: recursively use the quick sort to solve the subsetsto solve the subsets
Combine: none, as it is inplace sorting.Combine: none, as it is inplace sorting.
Quick sortQuick sort
Quicksort(A,p,r)Quicksort(A,p,r) if p if p r then return; r then return; q = partition(A,p,r);q = partition(A,p,r); Quicksort(A,p,q-1);Quicksort(A,p,q-1); Quicksort(A,q+1,r);Quicksort(A,q+1,r);
Partition(a,first,last)Partition(a,first,last)pivot =a[first]pivot =a[first]up=first; down=last;up=first; down=last;while (up <down)while (up <down){{while(pivot >=a[up] && while(pivot >=a[up] &&
up <=last) up++;up <=last) up++;while( pivot < a[down]) while( pivot < a[down])
down--;down--;if(up<down) swap(a, up, if(up<down) swap(a, up,
down);down);}}swap(a, first, down);swap(a, first, down);return down;return down;
Complexity of quick sort Complexity of quick sort isis
T(n) = T(i) + T(n-i-1) + T(n) = T(i) + T(n-i-1) + O(n)O(n)
Counting inversions Counting inversions problemproblem
Let L = {xLet L = {x11, x, x22, x, x33,…, x,…, xnn} a collection of } a collection of distinct integers between 1 and n.distinct integers between 1 and n.
The number of pairs (i,j), 1 ≤ i <j ≤ n The number of pairs (i,j), 1 ≤ i <j ≤ n such that xsuch that xii > x > xjj
If counter is (n-1) + (n-2) + … + 1 = If counter is (n-1) + (n-2) + … + 1 = n(n-1)/2, then the given data is in n(n-1)/2, then the given data is in ascending order.ascending order.
If counter value is 0, the given set is in If counter value is 0, the given set is in descending order.descending order.
Counting inversions Counting inversions problemproblem
3 3 5 5 2 2 1 1 7 7 9 9 8 8 4 4 66
1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 99
Number of crossing point is the Number of crossing point is the answeranswer
We can apply the merge sort like We can apply the merge sort like method to solve the problemmethod to solve the problem
Counting inversions Counting inversions problemproblem
Divide: Partition the list L into two lists A Divide: Partition the list L into two lists A and B with elements n/2and B with elements n/2
Conquer:Conquer: Recursively count the number of inversions in Recursively count the number of inversions in
AA Recursively count the number of inversions in Recursively count the number of inversions in
BB Combine: Count the number of inversions Combine: Count the number of inversions
involving one element in A and one involving one element in A and one element in B (use merge procedure and element in B (use merge procedure and increase the counter appropriately)increase the counter appropriately)
Counting inversions Counting inversions problemproblem
sort-and-count(L)sort-and-count(L)
If |L}==1, return 0;If |L}==1, return 0;
Else Else
AA first first n/2n/2 elements elements
BB remaining remaining n/2n/2 elements elements
(ca, A)= sort-and-count(A)(ca, A)= sort-and-count(A)
(cb, B)= sort-and-count(B)(cb, B)= sort-and-count(B)
(cm, L) = merge-and-(cm, L) = merge-and-count(A,B)count(A,B)
return ca + cb + cm with return ca + cb + cm with sorted list Lsorted list L
Running time of the method is T(n) ≤2T(n/2) + O(n)
Counting inversions Counting inversions problemproblem
merge-and-count (A,B)merge-and-count (A,B)1.1. Maintain a current pointer for each list and Maintain a current pointer for each list and
initially point to the first elementinitially point to the first element2.2. Maintain a variable count initialized to 0Maintain a variable count initialized to 03.3. While both lists are nonemptyWhile both lists are nonempty
1.1. Let ai and bj are the current elementsLet ai and bj are the current elements2.2. Append the smaller of the two to the output listAppend the smaller of the two to the output list3.3. If bj is smaller, increment the count by number If bj is smaller, increment the count by number
of elements in A from current position to lastof elements in A from current position to last4.4. Advance the current pointer of smaller element Advance the current pointer of smaller element
listlist
4.4. Append the rest of the non-empty list to the Append the rest of the non-empty list to the outputoutput
5.5. Return count and the merged list.Return count and the merged list.
Notes Notes In ‘sort-and-count’ processIn ‘sort-and-count’ process
The lists A and B inputed to the ‘merge-The lists A and B inputed to the ‘merge-and-count’ function are sorted.and-count’ function are sorted.
3 4 2 5 8 6 7 1 9
3 4 2 5 8 6 7 1 9
3 4 2 5 8 6 7 1 9
3 4 2 5 8 6 7 1 9
3 4 2 5 8 6 7 1 9
3 4 2 5 8 6 7 1 9
Counter =0
Example Example
3 4 2 5 8 6 7 1 9 Counter =0
Counter =2
2 3 4 5 8 6 7 1 9
2 3 4 5 8 6 7 1 9
2 3 4 5 8 6 7 1 9
Counter =2
2 3 4 5 8 6 7 1 9 Counter =2
Example Example 2 3 4 5 8 6 7 1 9 Counter =2
2 3 4 5 8 6 7 1 9
2 3 4 5 8 6 7 1 9
2 3 4 5 8 6 7 1 9 Counter =2
2 3 4 5 8 6 7 1 9
Example Example 2 3 4 5 8 6 7 1 9 Counter =2
2 3 4 5 8 6 7 1 9 Counter =2
2 3 4 5 8 1 6 7 9 Counter =4
1 Counter =9
1 2 3 4 5 Counter =9
1 2 3 4 5 6 Counter =10
1 2 3 4 5 6 7 Counter =11
1 2 3 4 5 6 7 8 Counter =11
1 2 3 4 5 6 7 8 9 Counter =11
Closest pair of pointsClosest pair of points
A set P of n points in the plane, find A set P of n points in the plane, find the pair of points in P that are closest the pair of points in P that are closest to each other.to each other. The simplest solution of the problem, The simplest solution of the problem,
consider all pair and find the distance of consider all pair and find the distance of each pair and then find the closets one.each pair and then find the closets one.
Complexity is O(nComplexity is O(n22)) Another solution, using divide-and-Another solution, using divide-and-
conquer approach with complexity O(n conquer approach with complexity O(n log n).log n).
Closest pair of pointsClosest pair of points
In 1DIn 1D Without any loss assume points are on Without any loss assume points are on
the x-axisthe x-axis Shortest distance is the one between Shortest distance is the one between
adjacent pair of points.adjacent pair of points. Sort the given points and compute the Sort the given points and compute the
distance for n-1 adjacent pairsdistance for n-1 adjacent pairs Complexity: O(n log n) + O(n) = O(n Complexity: O(n log n) + O(n) = O(n
log n)log n)
Closest pair of pointsClosest pair of points
1D1D
Divide and conquer approach after the sorting:Divide and conquer approach after the sorting:
Divide equally in left half and right halfDivide equally in left half and right half
Closest pair could be:Closest pair could be:
Closest pair in left half: distance Closest pair in left half: distance ll
Closest pair in right half: distanceClosest pair in right half: distance rr
Closest pair that span the left and right Closest pair that span the left and right halves and are at most min(halves and are at most min(ll , , r r ) apart (only ) apart (only one such pair?)one such pair?)
Closest pair of pointsClosest pair of points
Extend the concept of 1D for 2DExtend the concept of 1D for 2D Let the given point set P = {pLet the given point set P = {p11, p, p22,,
…,p…,pnn}with p}with pii=(x=(xii, y, yii)) If |P|≤3, use brute-force methodIf |P|≤3, use brute-force method X X sort(P) respect to x value sort(P) respect to x value
(presort)(presort) Y Y sort(P) respect to y value sort(P) respect to y value
(presort)(presort)
Closest pair of pointsClosest pair of points
Divide: Divide: partition, respect to a vertical line L, P partition, respect to a vertical line L, P
into Pinto PLL and P and PRR with with n/2n/2 and and n/2n/2 points points XXLL X(P X(PLL); Y); YLL Y(P Y(PLL);); XXRR X(P X(PRR); Y); YRR Y(P Y(PRR););
Conquer:Conquer: LL = Closesrpair(P = Closesrpair(PLL, X, XLL, Y, YLL)) RR = Closesrpair(P = Closesrpair(PRR, X, XRR, Y, YRR)) = min(= min(LL, , RR))
Closest pair of pointsClosest pair of points
Combine:Combine:
closest pair is in Left set, or closest pair is in Left set, or closest pair is in Right set, or closest pair is in Right set, or closest pair formed by one point from closest pair formed by one point from
each of the setseach of the sets
if minimum distance is if minimum distance is , then we reach to , then we reach to the solutionthe solution
else (i.e., less than else (i.e., less than ), then one point (p), then one point (pLL) ) in Pin PLL and other point (p and other point (pRR) is in P) is in PRR and both and both must reside within must reside within unit from L. unit from L.
Closest pair of pointsClosest pair of points Combine (contd…)Combine (contd…)Consider vertical strip (S) of width 2Consider vertical strip (S) of width 2
centered at L // centered at L // Ignore the points Ignore the points outside the stripoutside the strip
Now, Y’ Now, Y’ strip(Y), sorted on y value strip(Y), sorted on y valueFor each point p in SFor each point p in S compute the distance with compute the distance with next 5 next 5
points (in Y’)points (in Y’) if distance is less update the minimum if distance is less update the minimum
distance and track the closest pair.distance and track the closest pair.
Closest pair of pointsClosest pair of points
All points in PAll points in PLL are at least are at least unit apart, there can be at unit apart, there can be at most 4 point in most 4 point in x x with with distance distance
Similarly, in other half, so Similarly, in other half, so total 4+4-2 = 6 pointstotal 4+4-2 = 6 points
Closest pair within x x 22
Closest pair of pointsClosest pair of points
Running timeRunning timePresort (respect to x , y value): O(n log Presort (respect to x , y value): O(n log
n)n)Each recursive call required the sorted Each recursive call required the sorted
array in x and y for the sub points. array in x and y for the sub points. This can be done by O(n). This can be done by O(n). How?How?
size(Xsize(XLL)=n/2 size(X)=n/2 size(XRR)=n/2, t=0; r=0;)=n/2, t=0; r=0; for i = 1 to nfor i = 1 to n
if X[i] if X[i] P PLL X XLL[t][t]X[i]; t++X[i]; t++
if X[i] if X[i] P PRR X XRR[t][t]X[i]; r++X[i]; r++
Closest pair of pointsClosest pair of points
Running timeRunning timeIn combine step,In combine step,
for each point within the stripfor each point within the stripcompare with next 5 pointscompare with next 5 points
O(n)O(n)Complexity is:Complexity is: T(n) + O(n log n), whereT(n) + O(n log n), where T(n) = 2T(n/2) + O(n)T(n) = 2T(n/2) + O(n)
Integer multiplicationInteger multiplication
Instance: two n-digits number X and Y Instance: two n-digits number X and Y (decimal number or binary number)(decimal number or binary number)
Output: XY, a number with at most 2n Output: XY, a number with at most 2n digits, for both the cases.digits, for both the cases.
In school method, In school method, number of number of operations is O(noperations is O(n22), ), n is the number of n is the number of digits.digits.
School method
Integer multiplicationInteger multiplication
We consider 2 decimal, We consider 2 decimal, non-negative integersnon-negative integers
Input:Input: X = xX = xn-1 n-1 xxn-2 n-2 xxn-3n-3 … x … x2 2 xx1 1 xx0 0
Y = yY = yn-1 n-1 yyn-2 n-2 yyn-3n-3 … y … y2 2 yy1 1 yy00
Output:Output: Z = zZ = z2n-2 2n-2 zz2n-3 2n-3 zz2n-42n-4 … z … z2 2 zz1 1 zz00
1
0
10n
i
iixX
1
0
10n
i
iiyY
1
0
1
0
22
0
10*1010n
j
jj
n
i
ii
n
i
ii yxzZ
Integer multiplicationInteger multiplication
0122/12/
2/12/21
0122/12/
2/12/21
...
...
...
...
yyyyd
yyyyc
xxxxb
xxxxa
nn
nnnn
nn
nnnn
dcY
baXn
n
2/
2/
10*
10*
)*(10*)**(10*)*(
)10*(*)10*(*2/
2/2/
dbcbdacaZ
dcbaYXZnn
nn
Integer multiplicationInteger multiplication
LetLet U=a*cU=a*c V=b*dV=b*d W=(a+b)*(c+d) = a*c + a*d + b*c + W=(a+b)*(c+d) = a*c + a*d + b*c +
b*db*d Z = U*10Z = U*10nn + (W-U-V)*10 + (W-U-V)*10n/2n/2 + V + V We need three multiplications and We need three multiplications and
number of digits is n/2number of digits is n/2
Integer multiplicationInteger multiplication
integerMultiplication(X, Y)integerMultiplication(X, Y)
U= integerMultiplication(a, c)U= integerMultiplication(a, c)
V= integerMultiplication(b, d)V= integerMultiplication(b, d)
W= integerMultiplication((a+b),W= integerMultiplication((a+b),(c+d))(c+d))
return (U.10return (U.10nn+ (W-U-V)*10+ (W-U-V)*10n/2n/2 + V ) + V )
Integer multiplicationInteger multiplication
Complexity:Complexity:
T(n) = 3T(n/2) + O(n)T(n) = 3T(n/2) + O(n) The same algorithm can be used for The same algorithm can be used for
two binary numbers multiplicationtwo binary numbers multiplication We have to replace 10 by 2We have to replace 10 by 2
Matrix multiplicationMatrix multiplication
We consider n x n We consider n x n matricesmatrices
Using brut-force Using brut-force method, method, (n(n33))
a, b, c, d, e, f, g, h, a, b, c, d, e, f, g, h, r, s, t and u are n/2 r, s, t and u are n/2 x n/2 matricesx n/2 matrices
r = ae + bfr = ae + bf s= ag + bhs= ag + bh t = ce + dft = ce + df u = cg + dhu = cg + dh
nndc
baA
nnhf
geB
nnut
srC
Eight n/2 x n/2 multiplications and four additions Eight n/2 x n/2 multiplications and four additions of size n/2 x n/2of size n/2 x n/2
T(n) = 8T(n/2) + T(n) = 8T(n/2) + (n(n22))
Matrix multiplicationMatrix multiplication
Compute 14 n/2 x Compute 14 n/2 x n/2 matrices n/2 matrices A1,B1,A2,…,A7,B7A1,B1,A2,…,A7,B7 Pi = AiBiPi = AiBi
Then compute r, s, t, Then compute r, s, t, u by simple matrix u by simple matrix addition/subtraction addition/subtraction with different with different combination of Picombination of Pi
Seven n/2 x n/2 multiplications and some Seven n/2 x n/2 multiplications and some additions of size n/2 x n/2additions of size n/2 x n/2
T(n) = 7T(n/2) + T(n) = 7T(n/2) + (n(n22))
Strassen’s methodStrassen’s method r = ae + bfr = ae + bf s= ag + bhs= ag + bh t = ce + dft = ce + df u = cg + dhu = cg + dh
P5=(a+d).(e+h) = P5=(a+d).(e+h) = ae + ah + de + dhae + ah + de + dh
P6 = (b-d).(f+h) = bf P6 = (b-d).(f+h) = bf + bh –df –dh+ bh –df –dh
P7 = (a-c).(e+g) = P7 = (a-c).(e+g) = ae + ag –ce –cgae + ag –ce –cg
r = P5 + P4 –P2 +P6r = P5 + P4 –P2 +P6 = ae + bf= ae + bf
u = P5 + P1 – P3 – u = P5 + P1 – P3 – P7P7 = cg + dh = cg + dh
nndc
baA
nnhf
geB
P1 = ag –ah = a.(g-h)P1 = ag –ah = a.(g-h) P2 = bh+ ah = (a+b).hP2 = bh+ ah = (a+b).h s = P1 + P2s = P1 + P2
P3 = ce + de = (c+d).eP3 = ce + de = (c+d).e P4 = df -de = d.(f-e)P4 = df -de = d.(f-e) t = P3 + P4t = P3 + P4
Analysis of divide-and-Analysis of divide-and-conquer algorithmsconquer algorithms
Let the input problem of size n divides Let the input problem of size n divides into subproblems of size ninto subproblems of size n11, n, n22, n, n33,…, n,…, nrr..
The running time of the problem is T(n)The running time of the problem is T(n) Let tLet tdd(n) and t(n) and tcc(n) represents the time (n) represents the time
complexity of the divide step and complexity of the divide step and combine step respectively. Then,combine step respectively. Then,
T(n) = tT(n) = tdd(n) + (n) + T(nT(nii) + t) + tcc(n) when n> (n) when n> certain valuecertain value
Analysis of divide-and-Analysis of divide-and-conquer algorithmsconquer algorithms
Usually (not always) the subproblems Usually (not always) the subproblems are of same size, say, n/b and thenare of same size, say, n/b and then
T(n) = r.T(n/b)+ f(n) when n> certain T(n) = r.T(n/b)+ f(n) when n> certain value and f(n)= tvalue and f(n)= tdd(n) + t(n) + tcc(n) (n)
If not of same size, sometimes we If not of same size, sometimes we write the expression as (depending on write the expression as (depending on the requirement)the requirement)
T(n) ≤ r.T(n/b)+ f(n) n>cT(n) ≤ r.T(n/b)+ f(n) n>c
Analysis of divide-and-Analysis of divide-and-conquer algorithmsconquer algorithms
Binary search: Binary search: T(n) = T(n/2) + T(n) = T(n/2) + (1)(1) Merge sort: Merge sort: T(n)= T(n/2) + T(n/2) + (n)(n) Quick sort: Quick sort: T(n) = T(i) + T(n-i-1) + O(n)T(n) = T(i) + T(n-i-1) + O(n) Counting inver prob: Counting inver prob: T(n) ≤2T(n/2) + O(n)T(n) ≤2T(n/2) + O(n) Closest pair: T(n) + O(n log n), where T(n) Closest pair: T(n) + O(n log n), where T(n)
= 2T(n/2) + O(n)= 2T(n/2) + O(n) Integer multiplication: T(n) = 3T(n/2) + O(n)Integer multiplication: T(n) = 3T(n/2) + O(n) Matrix multiplication:Matrix multiplication:
T(n) = 8T(n/2) + T(n) = 8T(n/2) + (n(n22)) T(n) = 7T(n/2) + T(n) = 7T(n/2) + (n(n22))
Solution of recurrence Solution of recurrence relationrelation
Three approaches:Three approaches: Substitution methodSubstitution method Iteration methodIteration method Master methodMaster method
Substitution methodSubstitution method
We ‘guess’ that a certain function is We ‘guess’ that a certain function is a solution (or an upper or lower a solution (or an upper or lower bound on the solution) to the bound on the solution) to the recurrence relation, and we verify recurrence relation, and we verify that this correct by inductionthat this correct by induction
Consider T(n) = 2T(Consider T(n) = 2T(n/2n/2) + O(n), let ) + O(n), let T(n)=O(n log n)T(n)=O(n log n)
Substitution methodSubstitution method
Making a good guessMaking a good guess
Avoiding pitfallsAvoiding pitfalls T(n) = 2T(n/2) + n, let T(n) = cn for c>0T(n) = 2T(n/2) + n, let T(n) = cn for c>0 T(n) = 2.c.n/2 + n = cn +n = O(n), T(n) = 2.c.n/2 + n = cn +n = O(n), a a
wrong solwrong sol.. The initial assumption was wrongThe initial assumption was wrong
Substitution methodSubstitution method
Changing variable:Changing variable:
Consider m = lg Consider m = lg n ,i.e, n = 2n ,i.e, n = 2mm
T(2T(2mm) = 2T(2) = 2T(2m/2m/2) + m) + m S(m) = 2S(m/2) + mS(m) = 2S(m/2) + m S(m)= O(m lg m)S(m)= O(m lg m) T(n) = O(lg n lg lg n)T(n) = O(lg n lg lg n)
nnTnT lg)(2)(
Iteration methodIteration method
Repeatedly expand the recurrence Repeatedly expand the recurrence relation using the same given form relation using the same given form for the recurrence terms on the right for the recurrence terms on the right side, until we reach th base the case, side, until we reach th base the case, for which we can substitute the for which we can substitute the value given for the base case.value given for the base case.
Finally, we have to apply the Finally, we have to apply the summation of the values, to find the summation of the values, to find the boundbound
Iteration methodIteration method
T(n) = 3T(T(n) = 3T(n/4n/4) + ) + nn
O(n) and 3 3log3loglog 444 nnn
Recursion treeRecursion tree
A recursion tree is a way to visualize A recursion tree is a way to visualize what happens when a recursion is what happens when a recursion is iterated.iterated.
E.g., T(n) = 2T(n/2) + nE.g., T(n) = 2T(n/2) + n22
The master methodThe master method
Let a Let a 1 and b >1, f(n) is 1 and b >1, f(n) is monotonically non-negative monotonically non-negative function of n, and T(n) = aT(n/b) + function of n, and T(n) = aT(n/b) + f(n) is a recurrence relation defined f(n) is a recurrence relation defined on the non-negative integers. Then on the non-negative integers. Then T(n) can be bounded asymptotically T(n) can be bounded asymptotically as follows,as follows,
Analysis of divide-and-Analysis of divide-and-conquer algorithmsconquer algorithms
Binary search: Binary search: T(n) = T(n/2) + T(n) = T(n/2) + (1) [O(log n)](1) [O(log n)] Merge sort: Merge sort: T(n)= T(n/2) + T(n/2) + (n) [rule (n) [rule
1]1] Quick sort: Quick sort: T(n) = T(i); + T(n-i-1) + O(n) ?????T(n) = T(i); + T(n-i-1) + O(n) ????? Counting inver prob: Counting inver prob: T(n) ≤2T(n/2) + O(n) [R1: T(n) ≤2T(n/2) + O(n) [R1:
O(n log n]O(n log n] Closest pair: T(n) + O(n log n), where T(n) = Closest pair: T(n) + O(n log n), where T(n) =
2T(n/2) + O(n) [R1: O(n log n)]2T(n/2) + O(n) [R1: O(n log n)] Integer multiplication: T(n) = 3T(n/2) + O(n) Integer multiplication: T(n) = 3T(n/2) + O(n)
[R1][R1] Matrix multiplication:Matrix multiplication:
T(n) = 8T(n/2) + T(n) = 8T(n/2) + (n(n22) [R1: ) [R1: (n(n22) ]) ] T(n) = 7T(n/2) + T(n) = 7T(n/2) + (n(n22) [R1: ) [R1: (n(n22) ] ) ]
Master theoremMaster theorem
T(n) =3T(n/4) + n T(n) =3T(n/4) + n lg nlg n
a=3 and b=4, for a=3 and b=4, for =0.2,=0.2,
)()()( 3log793.03log 44 nnfnOn
Now test the regularity condition, af(n/b) ≤ Now test the regularity condition, af(n/b) ≤ cf(n) for some 0 <c <1cf(n) for some 0 <c <1
3(n/4) lg (n/4) ≤ (3/4) n lg n c=3/4.3(n/4) lg (n/4) ≤ (3/4) n lg n c=3/4.
By base 3, solution is T(n)= By base 3, solution is T(n)= (n lg n)(n lg n)
Master theoremMaster theorem
T(n) =2T(n/2) + n T(n) =2T(n/2) + n lg nlg n
a=2 and b=2, for any a=2 and b=2, for any ,,
)()( 2log2log 22 nnfnn
Approximation by Approximation by integralsintegrals
Let f(x) is a monotonically increasing Let f(x) is a monotonically increasing function and then the sum can be function and then the sum can be expressed as expressed as
Similarly, if f(x) monotonically decreasing Similarly, if f(x) monotonically decreasing function then function then
n
mx
xf )(
n
m
n
mk
n
m
dxxfkfdxxf1
1
)()()(
1
1
)()()(n
m
n
mk
n
m
dxxfkfdxxf
Consider f(k) = 1/k, for k>0Consider f(k) = 1/k, for k>0
n
k
nn
k
dk
kk
dk
1 0
1
1
1
n
k kn
1
1)1ln(
nn
k
nk
dk
k 12
ln1
1ln1
)1ln(1
nk
nn
k
Average case quick sortAverage case quick sort
Assumptions:Assumptions: All elements are distinctAll elements are distinct Number of permutation depends on Number of permutation depends on
order of the input data set. We assume order of the input data set. We assume the data set as {1,2,…,n}.the data set as {1,2,…,n}.
All permutation is equally likelyAll permutation is equally likely Let input list is any random Let input list is any random
permutation of {1, 2, …, n} and permutation of {1, 2, …, n} and pivot element is ‘s’pivot element is ‘s’
Average case quick sortAverage case quick sort
After partition let the situation is asAfter partition let the situation is as
(i(i11, i, i22, …, i, …, is-1s-1, s, j, s, js+1s+1, …., j, …., jnn,),)
(i(i11, i, i22, …, i, …, is-1s-1) is a random ) is a random permutation of {1, 2, …, s-1}permutation of {1, 2, …, s-1}
(j(js+1s+1, j, js+2s+2, …, j, …, jnn) is a random ) is a random permutation of {s+1, s+2, …, n}permutation of {s+1, s+2, …, n}
Average case quick sortAverage case quick sort
QQaa(n) = avg. number of comparison (n) = avg. number of comparison done by ‘Partition( )’ on an input done by ‘Partition( )’ on an input which is a random permutation of which is a random permutation of {1, 2, …, n}, then{1, 2, …, n}, then
QQaa(0) = Q(0) = Qaa(1) = 0(1) = 0 QQaa(2) = 3(2) = 3 Any element can be a pivot element, Any element can be a pivot element,
prob (s: s is pivot)=1/n, 1≤ s ≤ nprob (s: s is pivot)=1/n, 1≤ s ≤ n
Average case quick sortAverage case quick sort
2for )()1()1(1
)(1
nsnQsQnn
nQn
s
aaa
)1(22)1()1()(
)3(3for )(2)1()1()1(
)2(2for )(2)1()(
2for )()1()1()(
2
2
1
2
1
nQnnQnnnQ
nkQnnnQn
nkQnnnnQ
nsnQsQnnnnQ
aaa
n
k
aa
n
k
aa
n
s
aaa
Average case quick sortAverage case quick sort
1
41 )1
2
1
3
1(21)2(
3
112)(
1
1
3for 1
2)1(
1)(
1
1
n
kn
aa
aa
HQk
nQn
nn
nQn
nQn
)lg(3for )3
4)(1(2)( 1 nnnHnnQ n
a