2-11 heapsort - nanjing universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf ·...
TRANSCRIPT
![Page 1: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/1.jpg)
2-11 Heapsort
Hengfeng Wei
May 12, 2020
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 1 / 24
![Page 2: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/2.jpg)
Obama in a job interview at Google
“What is most efficient way to sort a million 32-bit integers?”
Obama: “The bubblesort would be the wrong way to go.”
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 2 / 24
![Page 3: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/3.jpg)
Obama: “The bubblesort would be the wrong way to go.”
O Ω Θ
Best case Worst case Average case
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 3 / 24
![Page 4: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/4.jpg)
Worst-case of Max-Heapify (TC 6.2-6)Show that the worst-case running time of Max-Heapify on ann-element heap is Ω(log n).
By Example.
1
5
2
6
34
Compare vs. Exchange
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 4 / 24
![Page 5: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/5.jpg)
Worst-case of Max-Heapify (Section 6.2 of CLRS)Show that the worst-case running time of Max-Heapify on ann-element heap is O(log n).
W (n) ≤ H(n)
No Examples Here!Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 5 / 24
![Page 6: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/6.jpg)
Worst-case of Max-HeapifyShow that the worst-case running time of Max-Heapify on ann-element heap is Θ(log n).
O Ω Θ
Worst-case “power” of A by example O = Ω
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 6 / 24
![Page 7: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/7.jpg)
Worst-case of Heapsort (TC 6.4-4)Show that the worst-case running time of Heapsort is Ω(n log n).
By Example.
Non-proof.
Θ(n)︸ ︷︷ ︸Extract-Max
× Ω(log n)︸ ︷︷ ︸Max-Heapify
= Ω(n log n)
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 7 / 24
![Page 8: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/8.jpg)
Worst-case of Heapsort (TC 6.4-4)Show that the worst-case running time of Heapsort is Ω(n log n).
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 8 / 24
![Page 9: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/9.jpg)
Heap in decreasing order?
7
5
12
6
34
T (7) = 2 + 1 + 1 + 1 + 0 + 0 = 5
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 9 / 24
![Page 10: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/10.jpg)
7
6
13
5
42
T (7) = 2 + 2 + 2 + 1 + 1 + 0 = 8
(Ex. 23, Section 5.2.3, TAOCP Vol 3)
8
7
63
5
42
1
n−1∑r=1
⌊log r⌋ = n⌊log n⌋ − 2⌊log n⌋+1 + 2 = Ω(n log n)
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 10 / 24
![Page 11: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/11.jpg)
Worst-case of HeapsortShow that the worst-case running time of Heapsort is O(n log n).
n−1∑r=1
⌊log r⌋ = n⌊log n⌋ − 2⌊log n⌋+1 + 2 = O(n log n)
No Examples Here!
Θ(n)︸ ︷︷ ︸Extract-Max
× O(log n)︸ ︷︷ ︸Max-Heapify
= O(n log n)
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 11 / 24
![Page 12: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/12.jpg)
Worst-case of HeapsortShow that the worst-case running time of Heapsort is Θ(n log n).
O Ω Θ
Worst-case “power” of A by example O = Ω
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 12 / 24
![Page 13: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/13.jpg)
Algorithm A
Inputs I of size n
O Ω Θ
Best-case by example “weakness” of A O = Ω
Worst-case “power” of A by example O = Ω
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 13 / 24
![Page 14: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/14.jpg)
Best-case of Heapsort (TC 6.4-5⋆)Show that when all elements are distinct, the best-case running time ofHeapsort is Ω(n log n).
Best-case of Heapsort (Ex. 32, Section 5.2.3, TAOCP Vol 3)Prove that the number of heapsort promotions, B, is always at least12N log N + O(N), if the keys being sorted are distinct.
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 14 / 24
![Page 15: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/15.jpg)
Best-case of Heapsort (TC 6.4-5⋆)Show that when all elements are distinct, the best-case running time ofHeapsort is Ω(n log n).
Consider the largest m = ⌈n/2⌉ elements.The largest m elements form a subtree.
≥ ⌊m/2⌋ of m must be nonleaves of that subtree.
≥ ⌊m/2⌋ of m appear in the first ⌊n/2⌋ positions.
They must be promoted to the root before being Extract-Max.
⌊m/2⌋∑k=1
⌊log k⌋ = 12
m log m + O(m)
B(n) ≥ 14
n log n + O(n) + B(⌊n/2⌋) =⇒ B(n) ≥ 12
n log n + O(n)Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 15 / 24
![Page 16: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/16.jpg)
Best-case of HeapsortShow that when all elements are distinct, the best-case running time ofHeapsort is O(n log n).
O Ω Θ
Best-case by example “weakness” of A O = Ω
Worst-case “power” of A by example O = Ω
12
n log n + O(n) ≤ ? ≤ n log n
O Ω Θ
Best-case ? ∼ 12n log n + O(n) O = Ω
Worst-case ∼ n log n ∼ n log n O = Ω
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 16 / 24
![Page 17: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/17.jpg)
Best-case of HeapsortShow that when all elements are distinct, the best-case running time ofHeapsort is
B(n) ≤ 12
n log n + O(n log log n).
By Example.
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 17 / 24
![Page 18: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/18.jpg)
“On the Best Case of Heapsort” (1994)
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 18 / 24
![Page 19: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/19.jpg)
Best-case of HeapsortShow that when all elements are distinct, the best-case running time ofHeapsort is Θ(n log n).
O Ω Θ
Best-case by example “weakness” of A O = Ω
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 19 / 24
![Page 20: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/20.jpg)
Algorithm A
Inputs I of size n
O Ω Θ
Best-case by example “weakness” of A O = Ω
Worst-case “power” of A by example O = Ω
Average-case ≤ ≥ O = Ω
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 20 / 24
![Page 21: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/21.jpg)
Average-case of HeapsortAssume that all elements are distinct. Show that the average-caserunning time of Heapsort is Θ(n log n).
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 21 / 24
![Page 22: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/22.jpg)
“By a surprisingly short counting argument.”“The Analysis of Heapsort” (Sedgewick ; 1992)
Robert Sedgewick D. E. Knuth
“It is elegant. see exercise 30.”Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 22 / 24
![Page 23: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/23.jpg)
Heap Identity (Additional)
∀h ≥ 1 : ⌈log(⌊12
h⌋ + 1)⌉ + 1 = ⌈log(h + 1)⌉
⌈log(h + 1)⌉ = ⌊log h⌋ + 1, ∀h ≥ 1
⌊log⌊12
h⌋⌋ + 1 = ⌈log(⌊12
h⌋ + 1)⌉ = ⌈log(h + 1)⌉ − 1 = ⌊log h⌋
(Depth of the parent of h) + 1 = Depth of h
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 23 / 24
![Page 24: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/24.jpg)
k-way Merging (TC 6.5-9)Give an O(n log k)-time algorithm to merge k sorted lists with nelements in total into one sorted list.
k = 2 =⇒ O(n)
Always maintain a min-heap of size k
whose root contains the next smallest element.
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 24 / 24
![Page 25: 2-11 Heapsort - Nanjing Universitycslabcms.nju.edu.cn/.../6/6c/2019-2-11-heapsort-handout.pdf · 2020. 5. 18. · Heapsort is Ω(nlogn). Best-case of Heapsort (Ex. 32, Section 5.2.3,](https://reader034.vdocument.in/reader034/viewer/2022052022/6037511ca2cd47721859672a/html5/thumbnails/25.jpg)
Hengfeng Wei ([email protected]) 2-11 Heapsort May 12, 2020 24 / 24