randomized algorithms & quicksortcarola/teaching/cs5633/spring12/slides/lecture-04.pdf ·...
TRANSCRIPT
![Page 1: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/1.jpg)
2/2/12 CS 5633 Analysis of Algorithms 1
CS 5633 – Spring 2012
Randomized Algorithms & Quicksort Carola Wenk
Slides courtesy of Charles Leiserson with small
changes by Carola Wenk
![Page 2: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/2.jpg)
2/2/12 CS 5633 Analysis of Algorithms 2
Deterministic Algorithms
Runtime for deterministic algorithms with input size n:
• Best-case runtime
Attained by one input of size n
• Worst-case runtime
Attained by one input of size n
• Average runtime
Averaged over all possible inputs of size n
![Page 3: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/3.jpg)
2/2/12 CS 5633 Analysis of Algorithms 3
Deterministic Algorithms:
Insertion Sort Best-case runtime: O(n), input [1,2,3,…,n]
Attained by one input of size n
• Worst-case runtime: O(n2), input [n, n-1, …,2,1]
Attained by one input of size n
• Average runtime : O(n2); see book for analysis
Averaged over all possible inputs of size n
•What kind of inputs are there?
• How many inputs are there?
![Page 4: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/4.jpg)
2/2/12 CS 5633 Analysis of Algorithms 4
Average Runtime
• What kind of inputs are there?
• Do [1,2,…,n] and [5,6,…,n+5] cause different behavior of Insertion Sort?
• No. Therefore it suffices to only consider all permutations of [1,2,…,n] .
• How many inputs are there?
• There are n! different permutations of [1,2,…,n]
![Page 5: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/5.jpg)
2/2/12 CS 5633 Analysis of Algorithms 5
Average Runtime
Insertion Sort: n=4
• Runtime is proportional to: 3 + #times in while loop
• Best: 3+0, Worst: 3+6=9, Average: 3+72/24 = 6
• Inputs: 4!=24
[1,2,3,4] [4,1,2,3] [4,1,3,2] [4,3,2,1]
[2,1,3,4] [1,4,2,3] [1,4,3,2] [3,4,2,1]
[1,3,2,4] [1,2,4,3] [1,3,4,2] [3,2,4,1]
[3,1,2,4] [4,2,1,3] [4,3,1,2] [4,2,3,1]
[3,2,1,4] [2,1,4,3] [3,4,1,2] [2,4,3,1]
[2,3,1,4] [2,4,1,3] [3,1,4,2] [2,3,4,1]
0 3 4 6
1 2 3 5
1 1 2 4
2 4 5 5
3 2 4 4
2 3 3 3
![Page 6: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/6.jpg)
2/2/12 CS 5633 Analysis of Algorithms 6
Average Runtime: Insertion
Sort • The average runtime averages runtimes over all n! different input permutations
• Disadvantage of considering average runtime:
• There are still worst-case inputs that will have the worst-case runtime
• Are all inputs really equally likely? That depends on the application
Better: Use a randomized algorithm
![Page 7: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/7.jpg)
2/2/12 CS 5633 Analysis of Algorithms 7
Randomized Algorithm:
Insertion Sort • Randomize the order of the input array:
• Either prior to calling insertion sort,
• or during insertion sort (insert random element)
• This makes the runtime depend on a probabilistic experiment (sequence of numbers obtained from random number generator)
Runtime is a random variable (maps sequence of random numbers to runtimes)
• Expected runtime = expected value of runtime random variable
![Page 8: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/8.jpg)
2/2/12 CS 5633 Analysis of Algorithms 8
Randomized Algorithm:
Insertion Sort • Runtime is independent of input order ([1,2,3,4] may have good or bad runtime, depending on sequence of random numbers)
•No assumptions need to be made about input distribution
• No one specific input elicits worst-case behavior
• The worst case is determined only by the output of a random-number generator.
When possible use expected runtimes of randomized algorithms instead of average case analysis of deterministic algorithms
![Page 9: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/9.jpg)
2/2/12 CS 5633 Analysis of Algorithms 9
Quicksort
• Proposed by C.A.R. Hoare in 1962.
• Divide-and-conquer algorithm.
• Sorts “in place” (like insertion sort, but not like merge sort).
• Very practical (with tuning).
• We are going to perform an expected runtime analysis on randomized quicksort
![Page 10: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/10.jpg)
2/2/12 CS 5633 Analysis of Algorithms 10
Quicksort: Divide and conquer
Quicksort an n-element array:
1. Divide: Partition the array into two subarrays around a pivot x such that elements in lower subarray x elements in upper subarray.
2. Conquer: Recursively sort the two subarrays.
3. Combine: Trivial.
x x x
Key: Linear-time partitioning subroutine.
![Page 11: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/11.jpg)
2/2/12 CS 5633 Analysis of Algorithms 11
Running time = O(n) for n elements.
Partitioning subroutine
PARTITION(A, p, q) ⊳ A[ p . . q] x A[ p] ⊳ pivot = A[ p] i p for j p + 1 to q
do if A[ j] x then i i + 1 exchange A[i] A[ j]
exchange A[ p] A[i] return i
x x x ?
p i q j
Invariant:
![Page 12: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/12.jpg)
2/2/12 CS 5633 Analysis of Algorithms 12
Example of partitioning
i j
6 10 13 5 8 3 2 11
![Page 13: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/13.jpg)
2/2/12 CS 5633 Analysis of Algorithms 13
Example of partitioning
i j
6 10 13 5 8 3 2 11
![Page 14: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/14.jpg)
2/2/12 CS 5633 Analysis of Algorithms 14
Example of partitioning
i j
6 10 13 5 8 3 2 11
![Page 15: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/15.jpg)
2/2/12 CS 5633 Analysis of Algorithms 15
Example of partitioning
6 10 13 5 8 3 2 11
i j
6 5 13 10 8 3 2 11
![Page 16: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/16.jpg)
2/2/12 CS 5633 Analysis of Algorithms 16
Example of partitioning
6 10 13 5 8 3 2 11
i j
6 5 13 10 8 3 2 11
![Page 17: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/17.jpg)
2/2/12 CS 5633 Analysis of Algorithms 17
Example of partitioning
6 10 13 5 8 3 2 11
i j
6 5 13 10 8 3 2 11
![Page 18: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/18.jpg)
2/2/12 CS 5633 Analysis of Algorithms 18
Example of partitioning
6 10 13 5 8 3 2 11
i j
6 5 3 10 8 13 2 11
6 5 13 10 8 3 2 11
![Page 19: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/19.jpg)
2/2/12 CS 5633 Analysis of Algorithms 19
Example of partitioning
6 10 13 5 8 3 2 11
i j
6 5 3 10 8 13 2 11
6 5 13 10 8 3 2 11
![Page 20: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/20.jpg)
2/2/12 CS 5633 Analysis of Algorithms 20
Example of partitioning
6 10 13 5 8 3 2 11
6 5 3 10 8 13 2 11
6 5 13 10 8 3 2 11
i j
6 5 3 2 8 13 10 11
![Page 21: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/21.jpg)
2/2/12 CS 5633 Analysis of Algorithms 21
Example of partitioning
6 10 13 5 8 3 2 11
6 5 3 10 8 13 2 11
6 5 13 10 8 3 2 11
i j
6 5 3 2 8 13 10 11
![Page 22: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/22.jpg)
2/2/12 CS 5633 Analysis of Algorithms 22
Example of partitioning
6 10 13 5 8 3 2 11
6 5 3 10 8 13 2 11
6 5 13 10 8 3 2 11
i j
6 5 3 2 8 13 10 11
![Page 23: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/23.jpg)
2/2/12 CS 5633 Analysis of Algorithms 23
Example of partitioning
6 10 13 5 8 3 2 11
6 5 3 10 8 13 2 11
6 5 13 10 8 3 2 11
6 5 3 2 8 13 10 11
i
2 5 3 6 8 13 10 11
![Page 24: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/24.jpg)
2/2/12 CS 5633 Analysis of Algorithms 24
Pseudocode for quicksort
QUICKSORT(A, p, r)
if p < r
then q PARTITION(A, p, r)
QUICKSORT(A, p, q–1)
QUICKSORT(A, q+1, r)
Initial call: QUICKSORT(A, 1, n)
![Page 25: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/25.jpg)
2/2/12 CS 5633 Analysis of Algorithms 25
Analysis of quicksort
• Assume all input elements are distinct.
• In practice, there are better partitioning algorithms for when duplicate input elements may exist.
• Let T(n) = worst-case running time on an array of n elements.
![Page 26: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/26.jpg)
2/2/12 CS 5633 Analysis of Algorithms 26
Worst-case of
quicksort
• Input sorted or reverse sorted.
• Partition around min or max element.
• One side of partition always has no elements.
)(
)()1(
)()1()1(
)()1()0()(
2n
nnT
nnT
nnTTnT
(arithmetic series)
![Page 27: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/27.jpg)
2/2/12 CS 5633 Analysis of Algorithms 27
Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
![Page 28: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/28.jpg)
2/2/12 CS 5633 Analysis of Algorithms 28
Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
T(n)
![Page 29: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/29.jpg)
2/2/12 CS 5633 Analysis of Algorithms 29
cn
T(0) T(n–1)
Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
![Page 30: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/30.jpg)
2/2/12 CS 5633 Analysis of Algorithms 30
cn
T(0) c(n–1)
Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
T(0) T(n–2)
![Page 31: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/31.jpg)
2/2/12 CS 5633 Analysis of Algorithms 31
cn
T(0) c(n–1)
Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
T(0) c(n–2)
T(0)
(1)
![Page 32: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/32.jpg)
2/2/12 CS 5633 Analysis of Algorithms 32
cn
T(0) c(n–1)
Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
T(0) c(n–2)
T(0)
T(0)
2
1
nkk
height
height = n
![Page 33: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/33.jpg)
2/2/12 CS 5633 Analysis of Algorithms 33
cn
T(0) c(n–1)
Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
T(0) c(n–2)
T(0)
T(0)
2
1
nkk
n
height = n
![Page 34: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/34.jpg)
2/2/12 CS 5633 Analysis of Algorithms 34
cn
c(n–1)
Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
c(n–2)
(1)
2
1
nkk
n
height = n
(1)
(1)
(1) T(n) = (n) + (n2)
= (n2)
![Page 35: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/35.jpg)
2/2/12 CS 5633 Analysis of Algorithms 35
Best-case analysis (For intuition only!)
If we’re lucky, PARTITION splits the array evenly:
T(n) = 2T(n/2) + (n)
= (n log n) (same as merge sort)
What if the split is always 10
9
10
1 : ?
)()(10
9
101 nnTnTnT
What is the solution to this recurrence?
![Page 36: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/36.jpg)
2/2/12 CS 5633 Analysis of Algorithms 36
Analysis of “almost-best” case
)(nT
![Page 37: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/37.jpg)
2/2/12 CS 5633 Analysis of Algorithms 37
Analysis of “almost-best” case
cn
nT101 nT
10
9
![Page 38: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/38.jpg)
2/2/12 CS 5633 Analysis of Algorithms 38
Analysis of “almost-best” case
cn
cn101 cn
10
9
nT100
1 nT100
9 nT100
9 nT100
81
![Page 39: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/39.jpg)
2/2/12 CS 5633 Analysis of Algorithms 39
Analysis of “almost-best” case
cn
cn101 cn
10
9
cn100
1 cn100
9 cn100
9 cn100
81
(1)
(1)
log10/9n
cn
cn
cn
…
O(n) leaves
![Page 40: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/40.jpg)
2/2/12 CS 5633 Analysis of Algorithms 40
log10
n
Analysis of “almost-best” case
cn
cn101 cn
10
9
cn100
1 cn100
9 cn100
9 cn100
81
(1)
(1)
log10/9n
cn
cn
cn
T(n) cn log10/9n + O(n)
…
cn log10n
O(n) leaves
(n log n)
![Page 41: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/41.jpg)
2/2/12 CS 5633 Analysis of Algorithms 41
Quicksort Runtimes
• Best case runtime Tbest(n) O(n log n)
• Worst case runtime Tworst(n) O(n2)
• Worse than mergesort? Why is it called quicksort then?
• Its average runtime Tavg(n) O(n log n )
• Better even, the expected runtime of randomized quicksort is O(n log n)
![Page 42: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/42.jpg)
2/2/12 CS 5633 Analysis of Algorithms 42
Average Runtime
The average runtime Tavg(n) for Quicksort is the average runtime over all possible inputs of length n.
• Tavg(n) has to average the runtimes over all n! different input permutations.
• There are still worst-case inputs that will have a O(n2) runtime
Better: Use randomized quicksort
![Page 43: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/43.jpg)
2/2/12 CS 5633 Analysis of Algorithms 43
Randomized quicksort
IDEA: Partition around a random element.
• Running time is independent of the input order. It depends only on the sequence s of random numbers.
• No assumptions need to be made about the input distribution.
• No specific input elicits the worst-case behavior.
• The worst case is determined only by the sequence s of random numbers.
![Page 44: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/44.jpg)
2/2/12 CS 5633 Analysis of Algorithms 44
Randomized quicksort
analysis • T(n,s) = random variable for the running time of randomized quicksort on an input of size n, with sequence s of random numbers which are assumed to be independent.
• E(T(n)) = expected value of T(n,s), the “expected runtime” of randomized quicksort.
T(n,s) =
T(0,s) + T(n–1,s) + (n) if 0 : n–1 split,
T(1,s) + T(n–2,s) + (n) if 1 : n–2 split,
T(n–1,s) + T(0,s) + (n) if n–1 : 0 split,
![Page 45: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/45.jpg)
2/2/12 CS 5633 Analysis of Algorithms 45
Randomized quicksort
analysis
For k = 0, 1, …, n–1, define the indicator random variable
Xk (s)= 1 if PARTITION generates a k : n–k–1 split,
0 otherwise.
E[Xk] = Pr{Xk = 1} = 1/n, since all splits are equally likely, assuming elements are distinct.
![Page 46: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/46.jpg)
2/2/12 CS 5633 Analysis of Algorithms 46
Analysis (continued)
1
0
)(),1(),()(n
k
k nsknTskTsX
T(n,s) =
T(0,s) + T(n–1,s) + (n) if 0 : n–1 split,
T(1,s) + T(n–2,s) + (n) if 1 : n–2 split,
T(n–1,s) + T(0,s) + (n) if n–1 : 0 split,
![Page 47: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/47.jpg)
2/2/12 CS 5633 Analysis of Algorithms 47
Calculating expectation
1
0
)()1()()]([n
kk nknTkTXEnTE
Take expectations of both sides.
![Page 48: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/48.jpg)
2/2/12 CS 5633 Analysis of Algorithms 48
Calculating expectation
1
0
1
0
)()1()(
)()1()()]([
n
kk
n
kk
nknTkTXE
nknTkTXEnTE
Linearity of expectation.
![Page 49: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/49.jpg)
2/2/12 CS 5633 Analysis of Algorithms 49
Calculating expectation
1
0
1
0
1
0
)()1()(
)()1()(
)()1()()]([
n
kk
n
kk
n
kk
nknTkTEXE
nknTkTXE
nknTkTXEnTE
Independence of Xk from other random choices.
![Page 50: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/50.jpg)
2/2/12 CS 5633 Analysis of Algorithms 50
Calculating expectation
1
0
1
0
1
0
1
0
1
0
1
0
)(1)1(1)(1
)()1()(
)()1()(
)()1()()]([
n
k
n
k
n
k
n
kk
n
kk
n
kk
nn
knTEn
kTEn
nknTkTEXE
nknTkTXE
nknTkTXEnTE
Linearity of expectation; E[Xk] = 1/n .
![Page 51: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/51.jpg)
2/2/12 CS 5633 Analysis of Algorithms 51
Calculating expectation
)()(2
)(1
)1(1
)(1
)()1()(
)()1()(
)()1()()]([
1
0
1
0
1
0
1
0
1
0
1
0
1
0
nkTEn
nn
knTEn
kTEn
nknTkTEXE
nknTkTXE
nknTkTXEnTE
n
k
n
k
n
k
n
k
n
k
k
n
k
k
n
k
k
Summations have identical terms.
![Page 52: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/52.jpg)
2/2/12 CS 5633 Analysis of Algorithms 52
Hairy recurrence
)()(2)]([1
2
nkTEn
nTEn
k
(The k = 0, 1 terms can be absorbed in the (n).)
Prove: E[T(n)] a n log n for constant a > 0 .
Use fact: 21
2
812
21 loglog nnnkk
n
k
(exercise).
• Choose a large enough so that a n log n dominates E[T(n)] for sufficiently small n 2.
![Page 53: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/53.jpg)
2/2/12 CS 5633 Analysis of Algorithms 53
Substitution method
)(log2
)(1
2
nkakn
nTEn
k
Substitute inductive hypothesis.
![Page 54: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/54.jpg)
2/2/12 CS 5633 Analysis of Algorithms 54
Substitution method
)(8
1log
2
12
)(log2
)(
22
1
2
nnnnn
a
nkakn
nTEn
k
Use fact.
![Page 55: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/55.jpg)
2/2/12 CS 5633 Analysis of Algorithms 55
Substitution method
)(4
log
)(8
1log
2
12
)(log2
)(
22
1
2
nan
nan
nnnnn
a
nkakn
nTEn
k
Express as desired – residual.
![Page 56: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/56.jpg)
2/2/12 CS 5633 Analysis of Algorithms 56
Substitution method
nan
nan
nan
nnnnn
a
nkakn
nTEn
k
log
)(4
log
)(8
1log
2
12
)(log2
)(
22
1
2
if a is chosen large enough so that an/4 dominates the (n).
,
![Page 57: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/57.jpg)
2/2/12 CS 5633 Analysis of Algorithms 57
Quicksort in practice
• Quicksort is a great general-purpose sorting algorithm.
• Quicksort is typically over twice as fast as merge sort.
• Quicksort can benefit substantially from code tuning.
• Quicksort behaves well even with caching and virtual memory.
![Page 58: Randomized Algorithms & Quicksortcarola/teaching/cs5633/spring12/slides/Lecture-04.pdf · Quicksort •Proposed by C.A.R. Hoare in 1962. •Divide-and-conquer algorithm. •Sorts](https://reader030.vdocument.in/reader030/viewer/2022040501/5e2219f36f660c55fc7eb2d7/html5/thumbnails/58.jpg)
2/2/12 CS 5633 Analysis of Algorithms 58
Average Runtime vs. Expected
Runtime • Average runtime is averaged over all inputs of a deterministic algorithm.
• Expected runtime is the expected value of the runtime random variable of a randomized algorithm. It effectively “averages” over all sequences of random numbers.
• De facto both analyses are very similar. However in practice the randomized algorithm ensures that not one single input elicits worst case behavior.