cse 326: data structures: sorting
DESCRIPTION
CSE 326: Data Structures: Sorting. Lecture 16: Friday, Feb 14, 2003. Review: QuickSort. procedure quickSortRecursive (Array A, int left, int right) if (left == right) return ; int pivot = choosePivot(A, left, right); /* partition A s.t.: A[left], A[left+1], …, A[i] pivot - PowerPoint PPT PresentationTRANSCRIPT
1
CSE 326: Data Structures: Sorting
Lecture 16: Friday, Feb 14, 2003
2
Review: QuickSort
procedure quickSortRecursive (Array A, int left, int right)
if (left == right) return;int pivot = choosePivot(A, left, right);
/* partition A s.t.: A[left], A[left+1], …, A[i] pivot A[i+1], A[i+2], …, A[right] pivot*/
quickSortRecursive(A, left, i);quickSortRecursive(A, i+1, right);}
procedure quickSortRecursive (Array A, int left, int right)
if (left == right) return;int pivot = choosePivot(A, left, right);
/* partition A s.t.: A[left], A[left+1], …, A[i] pivot A[i+1], A[i+2], …, A[right] pivot*/
quickSortRecursive(A, left, i);quickSortRecursive(A, i+1, right);}
3
Review: The Partitioni = left; j = right;repeat { while (A[i] < pivot) i++; while (A[j] > pivot) j--;
if (i<j) {swap(A[i], A[j]); i++; j++;} else break;}
i = left; j = right;repeat { while (A[i] < pivot) i++; while (A[j] > pivot) j--;
if (i<j) {swap(A[i], A[j]); i++; j++;} else break;}
There exists a sentinel A[k] pivot
A[left] … A[i-1] A[i] … A[j] … A[right]
pivot pivot
Why do we need i++, j+
+ ?
4
Review: The Partition
pivot
A[left] … A[j] … A[i] … A[right]
pivot
At the end:
Q: How are these elements ? A: They are = pivot !
quickSortRecursive(A, left, j);quickSortRecursive(A, i, right);
quickSortRecursive(A, left, j);quickSortRecursive(A, i, right);
5
Why is QuickSort Faster than Merge Sort?
• Quicksort typically performs more comparisons than Mergesort, because partitions are not always perfectly balanced– Mergesort – n log n comparisons– Quicksort – 1.38 n log n comparisons on average
• Quicksort performs many fewer copies, because on average half of the elements are on the correct side of the partition – while Mergesort copies every element when merging– Mergesort – 2n log n copies (using “temp array”) n log n copies (using “alternating array”)– Quicksort – n/2 log n copies on average
6
Stable Sorting Algorithms
Typical sorting scenario:
• Given N records: R[1], R[2], ..., R[N]
• They have N keys: R[1].A, ..., R[N].A
• Sort the records s.t.:R[1].A R[2].A ... R[N].A
A sorting algorithm is stable if:
• If i < j and R[i].A = R[j].A then R[i] comes before R[j] in the output
7
Stable Sorting Algorithms
Which of the following are stable sorting algorithms ?
• Bubble sort• Insertion sort• Selection sort• Heap sort• Merge sort• Quick sort
8
Stable Sorting Algorithms
Which of the following are stable sorting algorithms ?
• Bubble sort yes• Insertion sort yes• Selection sort yes• Heap sort no• Merge sort no• Quick sort no
We can always transform a non-stable sorting algorithm into a stable oneHow ?
9
Detour: Computing the Median
• The median of A[1], A[2], …, A[N] is some A[k] s.t.:– There exists N/2 elements A[k]– There exists N/2 elements A[k]
• Think of it as the perfect pivot !
• Very important in applications:– Median income v.s. average income– Median grade v.s. average grade
• To compute: sort A[1], …, A[N], then median=A[N/2]– Time O(N log N)
• Can we do it in O(N) time ?
10
Detour: Computing the Median
int medianRecursive(Array A, int left, int right){ if (left==right) return A[left];
. . . Partition . . .
if N/2 j return medianRecursive(A, left, j); if N/2 i return medianRecursive(A, i, right); return pivot}
Int median(Array A, int N) { return medianRecursive(A, 0, N-1); }
int medianRecursive(Array A, int left, int right){ if (left==right) return A[left];
. . . Partition . . .
if N/2 j return medianRecursive(A, left, j); if N/2 i return medianRecursive(A, i, right); return pivot}
Int median(Array A, int N) { return medianRecursive(A, 0, N-1); }
pivot
A[left] … A[j] … A[i] … A[right]
pivot
Why ?
11
Detour: Computing the Median• Best case running time:
T(N) = T(N/2) + cN = T(N/4) + cN(1 + 1/2) = T(N/8) + cN(1 + 1/2 + 1/4) = . . . = T(1) + cN (1 + 1/2 + 1/4 + … 1/2k) = O(N)
• Worst case = O(N2)• Average case = O(N)• Question: how can you compute the median in O(N) worst
case time ? Note: it’s tricky.
12
Back to Sorting
• Naïve sorting algorithms:– Bubble sort, insertion sort, selection sort– Time = O(N2)
• Clever sorting algorithms:– Merge sort, heap sort, quick sort– Time = O(N log N)
• I want to sort in O(N) !– Is this possible ?
13
Could We Do Better?
• Consider any sorting algorithm based on comparisons
• Run it on A[1], A[2], ..., A[N]– Assume they are distinct
• At each step it compares some A[i] with some A[j]– If A[i] < A[j] then it does something...– If A[i] > A[j] then it does something else...
Decision Tree !
14
Decision tree to sort list A,B,C
A<B
B<C
A<C C<A
C<B
B<A
A<C C<A
B<C C<B
A<B B<A
A<BC <B
A,B ,C .
A ,C ,B . C ,A ,B .
B ,A ,C . B<AC <A
B,C ,A . C ,B ,A
Legendfacts In ternal node, w ith facts known so far
A,B ,C Leaf node, w ith ordering of A ,B ,CC<A Edge, w ith result o f one com parison
Every possible execution of the algorithm corresponds to a root-to-leafpath in the tree.
15
Max depth of the decision tree
• How many permutations are there of N numbers?
• How many leaves does the tree have?
• What’s the shallowest tree with a given number of leaves?
• What is therefore the worst running time (number of comparisons) by the best possible sorting algorithm?
16
Max depth of the decision tree
• How many permutations are there of N numbers?
N!
• How many leaves does the tree have?
N!
• What’s the shallowest tree with a given number of leaves?
log(N!)
• What is therefore the worst running time (number of comparisons) by the best possible sorting algorithm?
log(N!)
17
Stirling’s approximation
n
e
nnn
2!
log( !) log 2
log( 2 ) lo ( log )g
n
n
nn n
e
nn n n
e
At least onebranch in thetree has this
depth
18
If you forget Stirling’s formula...
1
1
1
(log !) (ln !) ( ln ) ( lo
ln ! ln1 ln
g
2 ... ln
ln ln
ln ln 1
)
n n
k
n
n n
k x dx
x
n n n n n
x
n
n n n
Theorem:Every algorithm that sorts by comparing keys takes (n log n) time
Theorem:Every algorithm that sorts by comparing keys takes (n log n) time
19
Bucket Sort
• Now let’s sort in O(N)• Assume:
A[0], A[1], …, A[N-1] {0, 1, …, M-1}M = not too big
• Example: sort 1,000,000 person records on the first character of their last names:– Hence M = 128 (in practice: M = 27)
20
Bucket Sortint bucketSort(Array A, int N) { for k = 0 to M-1 Q[k] = new Queue;
for j = 0 to N-1 Q[A[j]].enqueue(A[j]);
Result = new Queue; for k = 0 to M-1 Result = Result.append(Q[k]);
return Result;}
int bucketSort(Array A, int N) { for k = 0 to M-1 Q[k] = new Queue;
for j = 0 to N-1 Q[A[j]].enqueue(A[j]);
Result = new Queue; for k = 0 to M-1 Result = Result.append(Q[k]);
return Result;}
Stablesorting !
21
Bucket Sort• Running time: O(M+N)
• Space: O(M+N)
• Recall that M << N, hence time = O(N)
• What about the Theorem that says sorting takes (N log N) ??
This is not realsorting, because
it’s for trivial keys
22
Radix Sort• I still want to sort in time O(N): non-trivial keys
• A[0], A[1], …, A[N-1] are strings– Very common in practice
• Each string is:cd-1cd-2…c1c0, where c0, c1, …, cd-1 {0, 1, …, M-1}M = 128
• Other example: decimal numbers
23
RadixSort• Radix = “The base of a
number system” (Webster’s dictionary)– alternate terminology: radix is
number of bits needed to represent 0 to base-1; can say “base 8” or “radix 3”
• Used in 1890 U.S. census by Hollerith
• Idea: BucketSort on each digit, bottom up.
24
The Magic of RadixSort
• Input list: 126, 328, 636, 341, 416, 131, 328
• BucketSort on lower digit:341, 131, 126, 636, 416, 328, 328
• BucketSort result on next-higher digit:416, 126, 328, 328, 131, 636, 341
• BucketSort that result on highest digit:126, 131, 328, 328, 341, 416, 636
25
Inductive Proof that RadixSort Works
• Keys: d-digit numbers, base B– (that wasn’t hard!)
• Claim: after ith BucketSort, least significant i digits are sorted. – Base case: i=0. 0 digits are sorted.– Inductive step: Assume for i, prove for i+1.
Consider two numbers: X, Y. Say Xi is ith digit of X:• Xi+1 < Yi+1 then i+1th BucketSort will put them in order• Xi+1 > Yi+1 , same thing• Xi+1 = Yi+1 , order depends on last i digits. Induction hypothesis
says already sorted for these digits because BucketSort is stable
26
Radix Sort
int radixSort(Array A, int N) { for k = 0 to d-1 A = bucketSort(A, on position k)}
int radixSort(Array A, int N) { for k = 0 to d-1 A = bucketSort(A, on position k)}
Running time: T = O(d(M+N)) = O(dN) = O(Size)
27
Radix Sort35 53 55 33 52 32 25
Q[0] Q[1] Q[2] Q[3] Q[4] Q[5] Q[6] Q[7] Q[8] Q[9]
53 33 35 55 25
52 32 53 33 35 55 25
52 32
Q[0] Q[1] Q[2] Q[3] Q[4] Q[5] Q[6] Q[7] Q[8] Q[9]
32 33 3525 52 53 55
25 32 33 35 52 53 55
A=
A=
A=
28
Running time of Radixsort
• N items, D digit keys of max value M
• How many passes?
• How much work per pass?
• Total time?
29
Running time of Radixsort
• N items, D digit keys of max value M
• How many passes? D
• How much work per pass? N + M – just in case M>N, need to account for time to empty out
buckets between passes
• Total time? O( D(N+M) )
30
Radix Sort
• What is the size of the input ? Size = DN
• Radix sort takes time O(Size) !!
cD-1 cD-2 … c0
A[0] ‘S’ ‘m’ ‘i’ ‘t’ ‘h’
A[1] ‘J’ ‘o’ ‘n’ ‘e’ ‘s’
…
A[N-1]
31
Radix Sort
• Variable length strings:
• Can adapt Radix Sort to sort in time O(Size) !– What about our Theorem ??
A[0]
A[1]
A[2]
A[3]
A[4]
32
Radix Sort
• Suppose we want to sort N distinct numbers
• Represent them in decimal:– Need D=log N digits
• Hence RadixSort takes time O(DN) = O(N log N)
• The total Size of N keys is O(N log N) !
• No conflict with theory
33
Sorting HUGE Data Sets• US Telephone Directory:
– 300,000,000 records • 64-bytes per record
– Name: 32 characters– Address: 54 characters– Telephone number: 10 characters
– About 2 gigabytes of data– Sort this on a machine with 128 MB RAM…
• Other examples?
34
Merge Sort Good for Something!
• Basis for most external sorting routines
• Can sort any number of records using a tiny amount of main memory– in extreme case, only need to keep 2 records in
memory at any one time!
35
External MergeSort• Split input into two “tapes” (or areas of disk)• Merge tapes so that each group of 2 records is
sorted• Split again• Merge tapes so that each group of 4 records is
sorted• Repeat until data entirely sorted
log N passes
36
Better External MergeSort
• Suppose main memory can hold M records.
• Initially read in groups of M records and sort them (e.g. with QuickSort).
• Number of passes reduced to log(N/M)