![Page 1: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/1.jpg)
CSC148 Week 11Larry Zhang
1
![Page 2: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/2.jpg)
Sorting Algorithms
2
![Page 3: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/3.jpg)
Selection Sort
def selection_sort(lst):
for each index i in the list lst:
swap element at index i with the smallest element to the right of i
3Image source: https://medium.com/@notestomyself/how-to-implement-selection-sort-in-swift-c3c981c6c7b3
![Page 4: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/4.jpg)
Selection Sort: Code
4
Worst-case runtime:
O(n²)
![Page 5: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/5.jpg)
Insertion Sort
5
def insertion_sort(lst):
for each index from 2 to the end of the list lst
insert element with index i in the proper place in lst[0..i]
image source: http://piratelearner.com/en/C/course/computer-science/algorithms/trivial-sorting-algorithms-insertion-sort/29/
![Page 6: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/6.jpg)
Insertion Sort: Code
6
Worst-case runtime:
O(n²)
![Page 7: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/7.jpg)
Bubble Sort
7
swap adjacent elements if they are “out of order”
go through the list over and over again, keep swapping until all elements are sorted
Image source: http://piratelearner.com/en/C/course/computer-science/algorithms/trivial-sorting-algorithms-bubble-sort/27/
Worst-case runtime:
O(n²)
![Page 8: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/8.jpg)
Summary
O(n²) sorting algorithms are considered slow.
We can do better than this, like O(n log n).
We will discuss a recursive fast sorting algorithms is called Quicksort.
● Its worst-case runtime is still O(n²)● But “on average”, it is O(n log n)
8
![Page 9: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/9.jpg)
Quicksort
9
![Page 10: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/10.jpg)
BackgroundInvented by Tony Hoare in 1960
Very commonly used sorting algorithm. When implemented well, can be about 2-3 times faster than merge sort and heapsort.
Invented NULL reference in 1965.Apologized for it in 2009
http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare10
![Page 11: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/11.jpg)
Quicksort: the idea
➔ Partition a list
2 8 7 1 3 5 6 4
pick a pivot
2 1 3 4 7 5 6 8
smaller than pivot larger than or equal to pivot 11
![Page 12: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/12.jpg)
2 1 3 4 7 5 6 8
Recursively partition the sub-lists before and after the pivot.
Base case:
1 sorted12
![Page 13: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/13.jpg)
Partition
def partition(lst, left, right, pivot)
’’’(list, int, int, int) -> int
Rearrange lst[left:right+1] so that elements >= pivot
come after elements < pivot;
return index of first element >= pivot
’’’
After partition([6, 2, 12, 6, 10, 15, 2, 13], 0, 8, 5)
lst becomes something like [2, 2, 6, 12, 6, 10, 15, 13]
return value is 2
13
![Page 14: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/14.jpg)
Partition
14
Trace:
partition([6, 2, 12, 6, 10, 15, 2, 13], 0, 8, 5)
See partition.py
![Page 15: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/15.jpg)
Quicksort Procedure
Quicksort(lst, left, right) would sort the segment
lst[left:right+1]
1. pick lst[right] as the pivot
2. Partition the list around the pivot
3. Recursive Quicksort the partitions on the left and right of
pivot.
Things to be careful about:
1. Is the subproblem getting smaller?
2. Are the subproblems containing the correct values?
15
![Page 16: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/16.jpg)
Quicksort Attempt #1
16
Recursive call not guaranteed to run on a smaller subproblem, e.g., i=0.Pivot should be excluded.
2 1 3 4 7 5 6 8
![Page 17: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/17.jpg)
Quicksort Attempt #2
17
We are excluding the element at index i, this element is supposed to be the pivot, but it may not be true here. How to fix?
![Page 18: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/18.jpg)
Quicksort Correct Attempt
18
pivot will be at index “right” after the partition.
This swap moves the pivot to index i
Exclude index i (the pivot) when making recursive call. Subproblem guaranteed to be smaller!
![Page 19: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/19.jpg)
DEMO
Trace an example in quicksort.py
Compare the runtimes: time_sorting.py
19
![Page 20: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/20.jpg)
Think
What kind of input list is “good” for Quicksort?
What kind of input list is “bad” for Quicksort?
20
![Page 21: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/21.jpg)
Runtime for Partition
O(n)
more precisely,
right - left, i.e., the length of the range to sort
(the value of j goes from left to right)
21
![Page 22: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/22.jpg)
Best-case runtime of Quicksort
If every time, the pivot happens to be the median of the range to sort.
22
Total amount of work at each level
● n
How many levels?
● log(n)
Total runtime is
● O(n log(n))
![Page 23: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/23.jpg)
Worst-case runtime of Quicksort
If every time, the pivot is the maximum/minimum of the range to sort
23
n
n-11
n-2 1
n-3
...1
Total amount of work at each level
● n, n-1, n-2, n-3, …, 1
How many levels?
● n
Total runtime is
● 1+2+3+4+...+n● O(n²)
Example of a worst-case input?
● [0,1,2,3,4,5,6,7,8]● an already-sorted list!
![Page 24: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/24.jpg)
Summary of Quicksort
Quicksort isn’t always “quick”, theoretically.
● In the worst case, it’s still quite slow (O(n²))
However, in practice, Quicksort performs very well.
● The worst case is rare, and on average the performance is good (O(n logn)).
What if someone maliciously keep passing the worst-case input to Quicksort?
● We could shuffle the input before calling Quicksort; or randomly pick the pivot rather than picking a fixed position.
● This “randomization” guarantees the average-case performance. It is also a major topic in algorithm design and analysis.
24
![Page 25: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/25.jpg)
Think
What if …
the pivot is not the median (dividing the range into half-half),
but divides the range by 90%-10% ratio.
What’s the big-Oh runtime for this case?
● The depth of recursion would be log base 10/9 rather base 2● Overall, still O(n log n)
25
![Page 26: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/26.jpg)
Merge Sort
26
recursive, worst-case O(n logn)
![Page 27: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/27.jpg)
Merge Sort
It is a recursive algorithm that works as follows:
● divide the input list into two halves● recursively sort left half and the right
half, then we obtain two sorted lists for the two halves.
● merge these two sorted lists into one big sorted list, that’s the answer
● Base case: size of the input list 1, already sorted.
27
Image source: https://github.com/heray1990/AlgorithmVisualization
![Page 28: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/28.jpg)
28
Merge two sorted lists
3
5
8
11
4
9
12
20
30
3
4
5
8
9
11
12
20
30
The procedure● compare the two pointed by
arrows● add the smaller one to the
output● advance the arrow of the
smaller one● When reaching the end of
one list, append everything left in the other
A B
Worst-case runtime:O(len(A) + len(B))or O(n)
![Page 29: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/29.jpg)
Merge two sorted lists
29
merge.py
![Page 30: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/30.jpg)
Merge Sort (returning a sorted copy of lst)
30
mergesort.py
![Page 31: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/31.jpg)
Merge Sort V2 (in-place mutating lst)
31
mergesort2.py
![Page 32: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/32.jpg)
Runtime of Merge Sort
Every recursive call is working on the subproblem of half the size
32
Total amount of work at each level
● n
How many levels?
● log(n)
Total runtime is
● O(n log(n))
This is true even in the worst case!
![Page 33: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/33.jpg)
Summary of Merge Sort
Merge Sort is a typical “divide-and-conquer” algorithm.
You will learn more about this algorithm design technique in CSC236.
33
![Page 34: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/34.jpg)
Bogosortjust for fun
34
![Page 35: CSC148 Week 11 - Department of Computer Science ...ylzhang/csc148w18/files/lec11.pdf · Summary of Quicksort Quicksort isn’t always “quick”, theoretically. In the worst case,](https://reader034.vdocument.in/reader034/viewer/2022042314/5f02b04e7e708231d40583d9/html5/thumbnails/35.jpg)
Bogosort codeWhat’s the best-case runtime?
What’s the worst-case runtime?
What’s the average-case runtime?
Bogosort is also known as
● Permutation sort● Slow Sort● Stupid Sort● Shotgun sort● Monkey sort
35