introduction to algorithms
DESCRIPTION
Introduction to Algorithms. 6.046J/18.401J/SMA5503 Lecture 4 Prof. Charles E. Leiserson. Quicksort. • Proposed by C.A.R. Hoare in 1962. • Divide-and-conquer algorithm. • Sorts “ in place ” (like insertion sort, but not like merge sort). • Very practical (with tuning). - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/1.jpg)
Introduction to Algorithms
6.046J/18.401J/SMA5503
Lecture 4Prof. Charles E. Leiserson
![Page 2: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/2.jpg)
Quicksort
• Proposed by C.A.R. Hoare in 1962.
• Divide-and-conquer algorithm.
• Sorts “in place” (like insertion sort, but not
like merge sort).
• Very practical (with tuning).
![Page 3: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/3.jpg)
Divide and conquer
Quicksort an n-element array:1. Divide: Partition the array into two subarraysaround a pivot x such that elements in lowersubarray ≤ x ≤ elements in upper subarray.
2. Conquer: Recursively sort the two subarrays.3. Combine: Trivial.Key: Linear-time partitioning subroutine.
![Page 4: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/4.jpg)
Partitioning subroutine
PARTITION(A, p, q) ⊳ A[p . . q]
x ← A[p] ⊳ pivot = A[p] Running time i ← p = O(n) for n for j ← p + 1 to q elements.
do if A[ j] ≤ x then i ← i + 1
exchange A[i] ↔ A[ j] exchange A[p] ↔ A[i] return i
invariant:
![Page 5: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/5.jpg)
Example of partitioning
![Page 6: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/6.jpg)
Example of partitioning
![Page 7: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/7.jpg)
Example of partitioning
![Page 8: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/8.jpg)
Example of partitioning
![Page 9: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/9.jpg)
Example of partitioning
![Page 10: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/10.jpg)
Example of partitioning
![Page 11: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/11.jpg)
Example of partitioning
![Page 12: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/12.jpg)
Example of partitioning
![Page 13: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/13.jpg)
Example of partitioning
![Page 14: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/14.jpg)
Example of partitioning
![Page 15: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/15.jpg)
Example of partitioning
![Page 16: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/16.jpg)
Example of partitioning
![Page 17: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/17.jpg)
Pseudocode for quicksort
QUICKSORT(A, p, r)
if p < r
then q ← PARTITION(A, p, r)
QUICKSORT(A, p, q–1)
QUICKSORT(A, q+1, r)
Initial call: QUICKSORT(A, 1, n)
![Page 18: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/18.jpg)
Analysis of quicksort
• Assume all input elements are distinct.
• In practice, there are better partitioning
algorithms for when duplicate input
elements may exist.
• Let T(n) = worst-case running time on
an array of n elements.
![Page 19: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/19.jpg)
Worst-case of quicksort
• Input sorted or reverse sorted.
• Partition around min or max element.
• One side of partition always has no elements.
T(n) = T(0) +T(n-1) + Θ(n)
= Θ(1) +T(n-1) + Θ(n)
= T(n-1) + Θ(n)
= Θ(n2) (arithmetic series)
![Page 20: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/20.jpg)
Worst-case recursion tree
T(n) = T(0) +T(n-1) + cn
![Page 21: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/21.jpg)
Worst-case recursion tree
T(n) = T(0) +T(n-1) + cn
T(n)
![Page 22: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/22.jpg)
Worst-case recursion tree
T(n) = T(0) +T(n-1) + cn
cn
T(0) T(n-1)
![Page 23: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/23.jpg)
Worst-case recursion tree
T(n) = T(0) +T(n-1) + cn
cn
T(0) c(n-1)
T(0) T(n-2)
![Page 24: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/24.jpg)
Worst-case recursion tree
T(n) = T(0) +T(n-1) + cn
cn
T(0) c(n-1)
T(0) T(n-2)
T(0)
Θ(1)
![Page 25: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/25.jpg)
Worst-case recursion tree
T(n) = T(0) +T(n-1) + cn
![Page 26: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/26.jpg)
Worst-case recursion tree
T(n) = T(0) +T(n-1) + cn
![Page 27: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/27.jpg)
Best-case analysis(For intuition only!)
If we’re lucky, PARTITION splits the array evenly:
T(n) = 2T(n/2) + Θ(n)
= Θ(n lg n) (same as merge sort)
What if the split is always 1/10:9/10?
T(n) = T(1/10n) + T(9/10n)+Θ(n)
What is the solution to this recurrence?
![Page 28: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/28.jpg)
Analysis of “almost-best” case
T(n)
![Page 29: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/29.jpg)
Analysis of “almost-best” case
cn
T(1/10n) T(9/10n)
![Page 30: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/30.jpg)
Analysis of “almost-best” case
cn
T(1/10n) T(9/10n)
T(1/100n ) T(9/100n) T(9/100n) T(81/100n)
![Page 31: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/31.jpg)
Analysis of “almost-best” case
![Page 32: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/32.jpg)
Analysis of “almost-best” case
![Page 33: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/33.jpg)
More intuition
Suppose we alternate lucky, unlucky,lucky, unlucky, lucky, ….
L(n) = 2U(n/2) + Θ(n) luckyU(n) = L(n – 1) + Θ(n) unlucky
Solving:L(n) = 2(L(n/2 – 1) + Θ(n/2)) + Θ(n)
= 2L(n/2 – 1) + Θ(n) = Θ(n lg n)
How can we make sure we are usually lucky?
Lucky!
![Page 34: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/34.jpg)
Randomized quicksort
IDEA: Partition around a random element.• Running time is independent of the input order.• No assumptions need to be made about the input distribution.• No specific input elicits the worst-case behavior.• The worst case is determined only by the output of a random-number generator.
![Page 35: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/35.jpg)
Randomized quicksortanalysis
Let T(n) = the random variable for the runningtime of randomized quicksort on an input of sizen, assuming random numbers are independent.For k = 0, 1, …, n–1, define the indicatorrandom variableXk = if PARTITION generates a k : n–k–1 split,
otherwise.E[Xk] = Pr{Xk = 1} = 1/n, since all splits areequally likely, assuming elements are distinct.
![Page 36: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/36.jpg)
Analysis (continued)
![Page 37: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/37.jpg)
Calculating expectation
Take expectations of both sides.
![Page 38: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/38.jpg)
Calculating expectation
Linearity of expectation.
![Page 39: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/39.jpg)
Calculating expectation
Independence of Xk from other random
choices.
![Page 40: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/40.jpg)
Calculating expectation
Linearity of expectation; E[Xk] = 1/n .
![Page 41: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/41.jpg)
Calculating expectation
![Page 42: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/42.jpg)
Hairy recurrence
(The k = 0, 1 terms can be absorbed in the Θ(n).)
Prove: E[T(n)] ≤ anlg n for constant a > 0 .
• Choose a large enough so that anlg n
dominates E[T(n)] for sufficiently small n ≥ 2.
![Page 43: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/43.jpg)
Substitution method
Substitute inductive hypothesis.
![Page 44: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/44.jpg)
Substitution method
Use fact.
![Page 45: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/45.jpg)
Substitution method
Express as desired – residual.
![Page 46: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/46.jpg)
Substitution method
if a is chosen large enough so that
an/4 dominates the Θ(n).
![Page 47: Introduction to Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f9d550346895dbd5b11/html5/thumbnails/47.jpg)
Quicksort in practice
• Quicksort is a great general-purpose
sorting algorithm.
• Quicksort is typically over twice as fast
as merge sort.
• Quicksort can benefit substantially from
code tuning.
• Quicksort behaves well even with
caching and virtual memory.