all sorting
TRANSCRIPT
-
8/12/2019 All Sorting
1/138
By:
Vishal Kumar Arora
AP,CSE Department,
Shaheed Bhagat Singh State Technical Campus,Ferozepur.
Different types of SortingTechniques used in DataStructures
-
8/12/2019 All Sorting
2/138
Sorting: Definition
Sorting: an operation that segregatesitems into groups according to specified
criterion.
A = { 3 1 6 2 1 3 4 5 9 0 }
A = { 0 1 1 2 3 3 4 5 6 9 }
-
8/12/2019 All Sorting
3/138
Sorting
Sorting = ordering. Sorted = ordered based on a particular
way.
Generally, collections of data are presentedin a sorted manner. Examples of Sorting: Words in a dictionary are sorted (and
case distinctions are ignored).
Files in a directory are often listed insorted order. The index of a book is sorted (and case
distinctions are ignored).
-
8/12/2019 All Sorting
4/138
Sorting: Contd
Many banks provide statementsthat list checks in increasing order(by check number).
In a newspaper, the calendar of events
in a schedule is generally sorted by date. Musical compact disks in a record store
are generally sorted by recording artist. Why? Imagine finding the phone number of
your friend in your mobile phone, but thephone book is not sorted.
-
8/12/2019 All Sorting
5/138
Review of Complexity
Most of the primary sorting algorithmsrun on different space and timecomplexity.
Time Complexity is defined to be the timethe computer takes to run a program (oralgorithm in our case).
Space complexity is defined to be theamount of memory the computer needs
to run a program.
-
8/12/2019 All Sorting
6/138
Complexity (cont.)
Complexity in general, measures thealgorithms efficiency in internal factorssuch as the time needed to run an
algorithm.
External Factors (not related tocomplexity):
Size of the input of the algorithm
Speed of the Computer
Quality of the Compiler
-
8/12/2019 All Sorting
7/138
An algorithm or function T(n) is O(f(n)) wheneverT(n)'s rate of growth is less than or equal to f(n)'s
rate.
An algorithm or function T(n) is (f(n)) wheneverT(n)'s rate of growth is greater than or equal to
f(n)'s rate.An algorithm or function T(n) is (f(n)) if andonly if the rate of growth of T(n) is equal to f(n).
O(n), (n), & (n)
-
8/12/2019 All Sorting
8/138
Types of Sorting Algorithms
There are many, many different types of
sorting algorithms, but the primary onesare:
Bubble SortSelection SortInsertion SortMerge SortQuick SortShell Sort
Radix SortSwap SortHeap Sort
-
8/12/2019 All Sorting
9/138
Bubble Sort: Idea
Idea: bubble in water.
Bubble in water moves upward. Why?
How? When a bubble moves upward, the water
from above will move downward to fill inthe space left by the bubble.
-
8/12/2019 All Sorting
10/138
Bubble Sort Example9, 6, 2, 12, 11, 9, 3, 7
6, 9, 2, 12, 11, 9, 3, 7
6, 2, 9, 12, 11, 9, 3, 7
6, 2, 9, 12, 11, 9, 3, 7
6, 2, 9, 11, 12, 9, 3, 7
6, 2, 9, 11, 9, 12, 3, 76, 2, 9, 11, 9, 3, 12, 7
6, 2, 9, 11, 9, 3, 7, 12The 12 is greater than the 7 so they are exchanged.The 12 is greater than the 3 so they are exchanged.
The twelve isgreater than the 9 so they are exchanged
The 12 is larger than the 11 so they are exchanged.
In the third comparison, the 9 is not larger than the 12 so no
exchange is made. We move on to compare the next pair without
any change to the list.
Now the next pair of numbers are compared. Again the 9 is the
larger and so this pair is also exchanged.
Bubblesort compares the numbers in pairs from left to right
exchanging when necessary. Here the first number is compared
to the second and as it is larger they are exchanged.
The end of the list has been reached so this is the end of the first pass. The
twelve at the end of the list must be largest number in the list and so is now in
the correct position. We now start a new pass from left to right.
-
8/12/2019 All Sorting
11/138
Bubble Sort Example
6, 2, 9, 11, 9, 3, 7, 122, 6, 9, 11, 9, 3, 7, 122, 6, 9, 9, 11, 3, 7, 122, 6, 9, 9, 3, 11, 7, 122, 6, 9, 9, 3, 7, 11, 126, 2, 9, 11, 9, 3, 7, 12
Notice that this time we do not have to compare the last two
numbers as we know the 12 is in position. This pass therefore only
requires 6 comparisons.
First Pass
Second Pass
-
8/12/2019 All Sorting
12/138
Bubble Sort Example
2, 6, 9, 9, 3, 7, 11, 122, 6, 9, 3, 9, 7, 11, 122, 6, 9, 3, 7, 9, 11, 12
6, 2, 9, 11, 9, 3, 7, 122, 6, 9, 9, 3, 7, 11, 12
Second Pass
First Pass
Third Pass
This time the 11 and 12 are in position. This pass therefore only
requires 5 comparisons.
-
8/12/2019 All Sorting
13/138
Bubble Sort Example
2, 6, 9, 3, 7, 9, 11, 122, 6, 3, 9, 7, 9, 11, 122, 6, 3, 7, 9, 9, 11, 12
6, 2, 9, 11, 9, 3, 7, 122, 6, 9, 9, 3, 7, 11, 12
Second Pass
First Pass
Third Pass
Each pass requires fewer comparisons. This time only 4 are needed.
2, 6, 9, 3, 7, 9, 11, 12Fourth Pass
-
8/12/2019 All Sorting
14/138
Bubble Sort Example
2, 6, 3, 7, 9, 9, 11, 122, 3, 6, 7, 9, 9, 11, 12
6, 2, 9, 11, 9, 3, 7, 122, 6, 9, 9, 3, 7, 11, 12
Second Pass
First Pass
Third Pass
The list is now sorted but the algorithm does not know this until it
completes a pass with no exchanges.
2, 6, 9, 3, 7, 9, 11, 12Fourth Pass2, 6, 3, 7, 9, 9, 11, 12
Fifth Pass
-
8/12/2019 All Sorting
15/138
Bubble Sort Example
2, 3, 6, 7, 9, 9, 11, 12
6, 2, 9, 11, 9, 3, 7, 122, 6, 9, 9, 3, 7, 11, 12
Second Pass
First Pass
Third Pass
2, 6, 9, 3, 7, 9, 11, 12Fourth Pass2, 6, 3, 7, 9, 9, 11, 12
Fifth Pass
Sixth Pass 2, 3, 6, 7, 9, 9, 11, 12
This pass no exchanges are made so the algorithm knows the list is
sorted. It can therefore save time by not doing the final pass. With
other lists this check could save much more work.
-
8/12/2019 All Sorting
16/138
Bubble Sort ExampleQuiz Time
1. Which number is definitely in its correct position at the
end of the first pass?
Answer: The last number must be the largest.
Answer: Each pass requires one fewer comparison than the last.
Answer: When a pass with no exchanges occurs.
2. How does the number of comparisons required change as
the pass number increases?
3. How does the algorithm know when the list is sorted?
4. What is the maximum number of comparisons requiredfor a list of 10 numbers?
Answer: 9 comparisons, then 8, 7, 6, 5, 4, 3, 2, 1 so total 45
-
8/12/2019 All Sorting
17/138
Bubble Sort: Example
40 2 1 43 3 65 0 -1 58 3 42 4
652 1 40 3 43 0 -1 58 3 42 4
65581 2 3 40 0 -1 43 3 42 4
1 2 3 400 65-1 43 583 42 4
1
2
3
4
Notice that at least one element will bein the correct position each iteration.
-
8/12/2019 All Sorting
18/138
1 0 -1 32 653 43 5842404
Bubble Sort: Example
0 -1 1 2 653 43 58424043
-1 0 1 2 653 43 58424043
6
7
8
1 2 0 3-1 3 40 6543 584245
-
8/12/2019 All Sorting
19/138
Bubble Sort: Analysis
Running time:
Worst case: O(N2)
Best case: O(N)
Variant:
bi-directional bubble sort
original bubble sort: only works to one
direction bi-directional bubble sort: works back and
forth.
-
8/12/2019 All Sorting
20/138
Selection Sort: Idea
1. We have two group of items:
sorted group, and
unsorted group
2. Initially, all items are in the unsortedgroup. The sorted group is empty.
We assume that items in the unsorted
group unsorted. We have to keep items in the sorted
group sorted.
-
8/12/2019 All Sorting
21/138
Selection Sort: Contd
1. Select the best (eg. smallest) itemfrom the unsorted group, then putthe best item at the end of thesorted group.
2. Repeat the process until the unsortedgroup becomes empty.
-
8/12/2019 All Sorting
22/138
Selection Sort
5 1 3 4 6 2
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
23/138
Selection Sort
5 1 3 4 6 2
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
24/138
Selection Sort
5 1 3 4 6 2
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
25/138
Selection Sort
5 1 3 4 6 2
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
26/138
Selection Sort
5 1 3 4 6 2
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
27/138
Selection Sort
5 1 3 4 6 2
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
28/138
Selection Sort
5 1 3 4 6 2
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
29/138
Selection Sort
5 1 3 4 6 2
Comparison
Data Movement
Sorted
Largest
-
8/12/2019 All Sorting
30/138
Selection Sort
5 1 3 4 2 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
31/138
Selection Sort
5 1 3 4 2 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
32/138
Selection Sort
5 1 3 4 2 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
33/138
Selection Sort
5 1 3 4 2 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
34/138
Selection Sort
5 1 3 4 2 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
35/138
Selection Sort
5 1 3 4 2 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
36/138
Selection Sort
5 1 3 4 2 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
37/138
Selection Sort
5 1 3 4 2 6
Comparison
Data Movement
Sorted
Largest
-
8/12/2019 All Sorting
38/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
39/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
40/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
41/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
42/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
43/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
44/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
Largest
-
8/12/2019 All Sorting
45/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
46/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
47/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
48/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
49/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
50/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
Largest
-
8/12/2019 All Sorting
51/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
52/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
53/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
54/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
55/138
Selection Sort
2 1 3 4 5 6
Comparison
Data Movement
Sorted
Largest
-
8/12/2019 All Sorting
56/138
Selection Sort
1 2 3 4 5 6
Comparison
Data Movement
Sorted
-
8/12/2019 All Sorting
57/138
Selection Sort
1 2 3 4 5 6
Comparison
Data Movement
Sorted
DONE!
-
8/12/2019 All Sorting
58/138
Selection Sort: Example
-
8/12/2019 All Sorting
59/138
4240 2 1 3 3 4 0 655843-1
42-1 2 1 3 3 4 0 65584340
42-1 2 1 3 3 4 655843400
42-1 2 1 0 3 4 655843403
Selection Sort: Example
Selection Sort: Example
-
8/12/2019 All Sorting
60/138
1
42-1 2 1 3 4 6558434030
42-1 0 3 4 6558434032
1 42-1 0 3 4 6558434032
1 420 3 4 6558434032-1
1 420 3 4 6558434032-1
Selection Sort: Example
-
8/12/2019 All Sorting
61/138
Selection Sort: Analysis
Running time:
Worst case: O(N2)
Best case: O(N2)
-
8/12/2019 All Sorting
62/138
Insertion Sort: Idea
Idea: sorting cards.
8 | 5 9 2 6 3
5 8 | 9 2 6 3
5 8 9 | 2 6 3
2 5 8 9 | 6 3
2 5 6 8 9 | 3
2 3 5 6 8 9 |
-
8/12/2019 All Sorting
63/138
Insertion Sort: Idea
1. We have two group of items:
sorted group, and
unsorted group
2. Initially, all items in the unsortedgroup and the sorted group is empty.
We assume that items in the unsorted
group unsorted. We have to keep items in the sorted
group sorted.
3. Pick any item from, then insert the
-
8/12/2019 All Sorting
64/138
40 2 1 43 3 65 0 -1 58 3 42 4
2 40 1 43 3 65 0 -1 58 3 42 4
1 2 40 43 3 65 0 -1 58 3 42 4
40
Insertion Sort: Example
-
8/12/2019 All Sorting
65/138
1 2 3 40 43 65 0 -1 58 3 42 4
1 2 40 43 3 65 0 -1 58 3 42 4
1 2 3 40 43 65 0 -1 58 3 42 4
Insertion Sort: Example
-
8/12/2019 All Sorting
66/138
1 2 3 40 43 65 0 -1 58 3 42 4
1 2 3 40 43 650 -1 58 3 42 4
1 2 3 40 43 650 58 3 42 41 2 3 40 43 650-1
Insertion Sort: Example
-
8/12/2019 All Sorting
67/138
1 2 3 40 43 650 58 3 42 41 2 3 40 43 650-1
1 2 3 40 43 650 58 42 41 2 3 3 43 650-1 5840 43 65
1 2 3 40 43 650 42 41 2 3 3 43 650-1 5840 43 65
Insertion Sort: Example
1 2 3 40 43 650 421 2 3 3 43 650-1 584 43 6542 5840 43 65
-
8/12/2019 All Sorting
68/138
Insertion Sort: Analysis
Running time analysis:
Worst case: O(N2)
Best case: O(N)
-
8/12/2019 All Sorting
69/138
A Lower Bound
Bubble Sort, Selection Sort, InsertionSort all have worst case of O(N2).
Turns out, for any algorithm thatexchanges adjacent items, this is thebest worst case: (N2)
In other words, this is a lower bound!
-
8/12/2019 All Sorting
70/138
Mergesort
Mergesort (divide-and-conquer)
Divide array into two halves.
A L G O R I T H M S
divideA L G O R I T H M S
-
8/12/2019 All Sorting
71/138
Mergesort
Mergesort (divide-and-conquer)
Divide array into two halves.
Recursively sort each half.
sort
A L G O R I T H M S
divideA L G O R I T H M S
A G L O R H I M S T
-
8/12/2019 All Sorting
72/138
Mergesort
Mergesort (divide-and-conquer)
Divide array into two halves.
Recursively sort each half.
Merge two halves to make sorted whole.
merge
sort
A L G O R I T H M S
divideA L G O R I T H M S
A G L O R H I M S T
A G H I L M O R S T
-
8/12/2019 All Sorting
73/138
auxiliary array
smallest smallest
A G L O R H I M S T
Merging
Merge. Keep track of smallest element in each sorted half.
Insert smallest of two elements into auxiliary array.
Repeat until done.
A
-
8/12/2019 All Sorting
74/138
auxiliary array
smallest smallest
A G L O R H I M S T
A
Merging
Merge.
Keep track of smallest element in each sorted half.
Insert smallest of two elements into auxiliary array.
Repeat until done.
G
-
8/12/2019 All Sorting
75/138
auxiliary array
smallest smallest
A G L O R H I M S T
A G
Merging
Merge. Keep track of smallest element in each sorted half.
Insert smallest of two elements into auxiliary array.
Repeat until done.
H
-
8/12/2019 All Sorting
76/138
auxiliary array
smallest smallest
A G L O R H I M S T
A G H
Merging
Merge. Keep track of smallest element in each sorted half.
Insert smallest of two elements into auxiliary array.
Repeat until done.
I
-
8/12/2019 All Sorting
77/138
auxiliary array
smallest smallest
A G L O R H I M S T
A G H I
Merging
Merge. Keep track of smallest element in each sorted half.
Insert smallest of two elements into auxiliary array.
Repeat until done.
L
-
8/12/2019 All Sorting
78/138
auxiliary array
smallest smallest
A G L O R H I M S T
A G H I L
Merging
Merge. Keep track of smallest element in each sorted half.
Insert smallest of two elements into auxiliary array.
Repeat until done.
M
-
8/12/2019 All Sorting
79/138
auxiliary array
smallest smallest
A G L O R H I M S T
A G H I L M
Merging
Merge. Keep track of smallest element in each sorted half.
Insert smallest of two elements into auxiliary array.
Repeat until done.
O
-
8/12/2019 All Sorting
80/138
auxiliary array
smallest smallest
A G L O R H I M S T
A G H I L M O
Merging
Merge. Keep track of smallest element in each sorted half.
Insert smallest of two elements into auxiliary array.
Repeat until done.
R
-
8/12/2019 All Sorting
81/138
auxiliary array
first halfexhausted smallest
A G L O R H I M S T
A G H I L M O R
Merging
Merge. Keep track of smallest element in each sorted half.
Insert smallest of two elements into auxiliary array.
Repeat until done.
S
-
8/12/2019 All Sorting
82/138
auxiliary array
first halfexhausted smallest
A G L O R H I M S T
A G H I L M O R S
Merging
Merge. Keep track of smallest element in each sorted half.
Insert smallest of two elements into auxiliary array.
Repeat until done.
T
-
8/12/2019 All Sorting
83/138
Notes on Quicksort
Quicksort is more widely used thanany other sort.
Quicksort is well-studied, not difficult
to implement, works well on a varietyof data, and consumes fewerresources that other sorts in nearly all
situations. Quicksort is O(n*log n) time, and
O(log n) additional space due torecursion.
-
8/12/2019 All Sorting
84/138
Quicksort Algorithm
Quicksort is a divide-and-conquermethod for sorting. It works bypartitioning an array into parts, then
sorting each part independently. The crux of the problem is how to
partition the array such that the
following conditions are true: There is some element, a[i], where
a[i] is in its final position.
For all l < i, a[l] < a[i].
For all i < r, a[i] < a[r].
-
8/12/2019 All Sorting
85/138
Quicksort Algorithm (cont)
As is typical with a recursive program, onceyou figure out how to divide your probleminto smaller subproblems, the
implementation is amazingly simple.int partition(Item a[], int l, int r);
void quicksort(Item a[], int l, int r)
{ int i;
if (r
-
8/12/2019 All Sorting
86/138
-
8/12/2019 All Sorting
87/138
-
8/12/2019 All Sorting
88/138
Partitioning in Quicksort
How do we partition the arrayefficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element exchange
repeat until pointers cross
Q U I C K S O R T I S C O O L
partitioned
partition element left
right
unpartitioned
-
8/12/2019 All Sorting
89/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
exchange
repeat until pointers crossswap me
Q U I C K S O R T I S C O O L
partitioned
partition element left
right
unpartitioned
-
8/12/2019 All Sorting
90/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
exchange
repeat until pointers cross
partitioned
partition element left
right
unpartitioned
swap me
Q U I C K S O R T I S C O O L
-
8/12/2019 All Sorting
91/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
exchange
repeat until pointers cross
partitioned
partition element left
right
unpartitioned
swap me
Q U I C K S O R T I S C O O L
-
8/12/2019 All Sorting
92/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
exchange
repeat until pointers cross
partitioned
partition element left
right
unpartitioned
swap me
Q U I C K S O R T I S C O O L
swap me
-
8/12/2019 All Sorting
93/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
exchange
repeat until pointers cross
partitioned
partition element left
right
unpartitioned
C U I C K S O R T I S Q O O L
-
8/12/2019 All Sorting
94/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
exchange
repeat until pointers crossswap me
partitioned
partition element left
right
unpartitioned
C U I C K S O R T I S Q O O L
-
8/12/2019 All Sorting
95/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
exchange
repeat until pointers cross
partitioned
partition element left
right
unpartitioned
swap me
C U I C K S O R T I S Q O O L
-
8/12/2019 All Sorting
96/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
exchange
repeat until pointers cross
partitioned
partition element left
right
unpartitioned
swap me
C U I C K S O R T I S Q O O L
swap me
-
8/12/2019 All Sorting
97/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
exchange
repeat until pointers cross
partitioned
partition element left
right
unpartitioned
C I I C K S O R T U S Q O O L
-
8/12/2019 All Sorting
98/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
exchange
repeat until pointers cross
partitioned
partition element left
right
unpartitioned
C I I C K S O R T U S Q O O L
-
8/12/2019 All Sorting
99/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
exchange
repeat until pointers cross
partitioned
partition element left
right
unpartitioned
C I I C K S O R T U S Q O O L
-
8/12/2019 All Sorting
100/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
Exchange and repeat until pointers cross
partitioned
partition element left
right
unpartitioned
C I I C K S O R T U S Q O O L
-
8/12/2019 All Sorting
101/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
Exchange and repeat until pointers cross
swap me
partitioned
partition element left
right
unpartitioned
C I I C K S O R T U S Q O O L
-
8/12/2019 All Sorting
102/138
-
8/12/2019 All Sorting
103/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
Exchange and repeat until pointers cross
partitioned
partition element left
right
unpartitioned
swap me
C I I C K S O R T U S Q O O L
-
8/12/2019 All Sorting
104/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
Exchange and repeat until pointers cross
partitioned
partition element left
right
unpartitioned
swap me
C I I C K S O R T U S Q O O L
-
8/12/2019 All Sorting
105/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
Exchange and repeat until pointers cross
pointers cross
swap with
partitioning
element
partitioned
partition element left
right
unpartitioned
C I I C K S O R T U S Q O O L
-
8/12/2019 All Sorting
106/138
Partitioning in Quicksort
How do we partition the array efficiently? choose partition element to be rightmost element
scan from left for larger element
scan from right for smaller element
Exchange and repeat until pointers cross
partitioned
partition element left
right
unpartitioned
partition iscomplete
C I I C K L O R T U S Q O O S
-
8/12/2019 All Sorting
107/138
Quicksort Demo
Quicksortillustrates the operation ofthe basic algorithm. When the arrayis partitioned, one element is in place
on the diagonal, the left subarray hasits upper corner at that element, andthe right subarray has its lowercorner at that element. The originalfile is divided into two smaller partsthat are sorted independently. Theleft subarray is always sorted first, so
the sorted result emerges as a line of
http://www.cs.princeton.edu/courses/archive/spring03/cs226/demo/qsort/QuickSort.htmlhttp://www.cs.princeton.edu/courses/archive/spring03/cs226/demo/qsort/QuickSort.html -
8/12/2019 All Sorting
108/138
Why study Heapsort?
It is a well-known, traditionalsorting algorithm you will beexpected to know
Heapsort is alwaysO(n log n)
Quicksort is usually O(n log n) butin the worst case slows to O(n2)
Quicksort is generally faster, butHeapsort is better in time-criticalapplications
-
8/12/2019 All Sorting
109/138
What is a heap?
Definitions of heap:1. A large area of memory from which
the programmer can allocate
blocks as needed, and deallocatethem (or allow them to be garbagecollected) when no longer needed
2. A balanced, left-justified binary
tree in which no node has a valuegreater than the value in its parent
Heapsort uses the second
-
8/12/2019 All Sorting
110/138
Balanced binary trees
Recall: The depth of a node is its distance from the root
The depth of a tree is the depth of the deepest node
A binary tree of depth nis balanced if all the nodes at depths 0through n-2have two children
Balanced Balanced Not balanced
n-2
n-1n
-
8/12/2019 All Sorting
111/138
Left-justified binary trees
A balanced binary tree is left-justified if:
all the leaves are at the samedepth, or
all the leaves at depth n+1are tothe left of all the nodes at depth n
Left-justified Not left-justified
-
8/12/2019 All Sorting
112/138
The heap property
A node has the heap property if thevalue in the node is as large as orlarger than the values in its children
All leaf nodes automatically have the heapproperty
A binary tree is a heap if allnodes in it
have the heap property
12
8 3
Blue node has
heap property
12
8 12
Blue node has
heap property
12
8 14
Blue node does not
have heap property
-
8/12/2019 All Sorting
113/138
siftUp Given a node that does not have the heap
property, you can give it the heapproperty by exchanging its value with thevalue of the larger child
This is sometimes called sifting up
Notice that the child may have lostthe
heap property
14
8 12
Blue node has
heap property
12
8 14
Blue node does not
have heap property
-
8/12/2019 All Sorting
114/138
Constructing a heap I
A tree consisting of a single node is automatically a heap We construct a heap by adding nodes one at a time:
Add the node just to the right of the rightmost node in thedeepest level
If the deepest level is full, start a new level
Examples:
Add a new
node here
Add a new
node here
-
8/12/2019 All Sorting
115/138
Constructing a heap II
Each time we add a node, we may destroy the heap property of itsparent node
To fix this, we sift up
But each time we sift up, the value of the topmost node in the sift mayincrease, and this may destroy the heap property of itsparent node
We repeat the sifting up process, moving up in the tree, until either
We reach nodes whose values dont need to be swapped (becausethe parent is stilllarger than both children), or
We reach the root
-
8/12/2019 All Sorting
116/138
Constructing a heap III
8 8
10
10
8
10
8 5
10
8 5
12
10
12 5
8
12
10 5
8
1 2 3
4
-
8/12/2019 All Sorting
117/138
Other children are not affected
The node containing 8 is not affected because itsparent gets larger, not smaller
The node containing 5 is not affected because itsparent gets larger, not smaller
The node containing 8 is still not affected because,although its parent got smaller, its parent is stillgreater than it was originally
12
10 5
8 14
12
14 5
8 10
14
12 5
8 10
-
8/12/2019 All Sorting
118/138
A sample heap Heres a sample binary tree after it has
been heapified
Notice that heapified does notmeansorted
Heapifying does notchange the shape of
the binary tree; this binary tree is
19
1418
22
321
14
119
15
25
1722
h
-
8/12/2019 All Sorting
119/138
Removing the root Notice that the largest number is now in
the root
Suppose we discardthe root:
How can we fix the binary tree so it isonce again balanced and left-justified?
Solution: remove the rightmost leaf at the
deepest level and use it for the new root
19
1418
22
321
14
119
15
1722
11
h h d
-
8/12/2019 All Sorting
120/138
The reHeapmethod I Our tree is balanced and left-justified, but
no longer a heap
However, only the rootlacks the heapproperty
We can siftUp()the root
After doing this, one and only one of itschildren may have lost the heap property
19
1418
22
321
14
9
15
1722
11
Th h d II
-
8/12/2019 All Sorting
121/138
The reHeapmethod II Now the left child of the root (still the
number 11) lacks the heap property
We can siftUp()this node
After doing this, one and only one of itschildren may have lost the heap property
19
1418
22
321
14
9
15
1711
22
Th h d III
-
8/12/2019 All Sorting
122/138
The reHeapmethod III Now the right child of the left child of the
root (still the number 11) lacks the heapproperty:
We can siftUp()this node
After doing this, one and only one of itschildren may have lost the heap property
but it doesnt because its a leaf
19
1418
11
321
14
9
15
1722
22
Th th d IV
-
8/12/2019 All Sorting
123/138
The reHeapmethod IV Our tree is once again a heap, because
every node in it has the heap property
Once again, the largest (oralargest) value isin the root
We can repeat this process until the tree
becomes em t
19
1418
21
311
14
9
15
1722
22
S ti
-
8/12/2019 All Sorting
124/138
Sorting
What do heaps have to do withsorting an array?
Heres the neat part:
Because the binary tree is balancedandleft justified,it can be represented as anarray
All our operations on binary trees can be
represented as operations on arrays To sort:heapify the array;
while the array isnt empty {
remove and replace the root;
M i i t
-
8/12/2019 All Sorting
125/138
Mapping into an array
Notice: The left child of indexi is at index2*i+1
The right child of index iis at index2*i+2
Example: the children of node 3 (19) are 7
19
1418
22
321
14
119
15
25
1722
25 22 17 19 22 14 15 18 14 21 3 9 11
0 1 2 3 4 5 6 7 8 9 10 11 12
Removing and replacing thet
-
8/12/2019 All Sorting
126/138
root
The root is the first element in the array The rightmost node at the deepest level
is the last element
Swap them...
...And pretend that the last element in thearray no longer existsthat is, the lastindex is 11(9)
25 22 17 19 22 14 15 18 14 21 3 9 11
0 1 2 3 4 5 6 7 8 9 10 11 12
11 22 17 19 22 14 15 18 14 21 3 9 25
0 1 2 3 4 5 6 7 8 9 10 11 12
R h d t
-
8/12/2019 All Sorting
127/138
Reheap and repeat
Reheap the root node (index 0, containing11)...
...And again, remove and replace the rootnode
Remember, though, that the last array
22 22 17 19 21 14 15 18 14 11 3 9 25
0 1 2 3 4 5 6 7 8 9 10 11 12
9 22 17 19 22 14 15 18 14 21 3 22 25
0 1 2 3 4 5 6 7 8 9 10 11 12
11 22 17 19 22 14 15 18 14 21 3 9 25
0 1 2 3 4 5 6 7 8 9 10 11 12
A l i I
-
8/12/2019 All Sorting
128/138
Analysis I
Heres how the algorithm starts:heapify the array;
Heapifying the array: we add each ofnnodes
Each node has to be sifted up, possiblyas far as the root
Since the binary tree is perfectly balanced,sifting up a single node takesO(log n)time
Since we do this ntimes, heapifyingtakes n*O(log n)time, that is,O(n log n)time
A l i II
-
8/12/2019 All Sorting
129/138
Analysis II
Heres the rest of the algorithm:while the array isnt empty {
remove and replace the root;
reheap the new root node;
} We do the while loop ntimes (actually,
n-1times), because we remove one ofthe nnodes each time
Removing and replacing the root takesO(1)time
Therefore, the total time is ntimes
An l i III
-
8/12/2019 All Sorting
130/138
Analysis III
To reheap the root node, we have tofollow one pathfrom the root to a leafnode (and we might stop before we
reach a leaf) The binary tree is perfectly balanced
Therefore, this path is O(log n)long
And we only do O(1)operations at eachnode
Therefore, reheaping takes O(log n)times
Analysis IV
-
8/12/2019 All Sorting
131/138
Analysis IV
Heres the algorithm again:heapify the array;
while the array isnt empty {
remove and replace the root;
reheap the new root node;}
We have seen that heapifying takesO(n log n)time
The while loop takes O(n log n)time
The total time is therefore O(n log n) +O(n log n)
The End
-
8/12/2019 All Sorting
132/138
The End
Shell Sort: Idea
-
8/12/2019 All Sorting
133/138
40 2 1 43 3 65 0 -1 58 3 42 4
Original:
5-sort: Sort items with distance 5 element:
40 2 1 43 3 65 0 -1 58 3 42 4
Shell Sort: IdeaDonald Shell (1959): Exchange items that are far apart!
O i i lShell Sort: Example
-
8/12/2019 All Sorting
134/138
40 2 1 43 3 65 0 -1 58 3 42 4
Original:
40 0 -1 43 3 42 2 1 58 3 65 4
After 5-sort:
2 0 -1 3 1 4 40 3 42 43 65 58
After 3-sort:
Shell Sort: Example
After 1-sort:
1 2 3 40 43 650 421 2 3 3 43 650-1 584 43 6542 5840 43 65
Shell Sort: Gap Values
-
8/12/2019 All Sorting
135/138
Shell Sort: Gap Values
Gap: the distance between itemsbeing sorted.
As we progress, the gapdecreases. Shell Sort is alsocalled Diminishing Gap Sort.
Shell proposed starting gap ofN/2, halving at each step.
There are many ways of choosing
the next gap
Shell Sort: Analysis
-
8/12/2019 All Sorting
136/138
Shell's Odd Gaps Only Dividing by 2.21000 122 11 11 9
2000 483 26 21 23
4000 1936 61 59 54
8000 7950 153 141 114
16000 32560 358 322 26932000 131911 869 752 575
64000 520000 2091 1705 1249
Shellsort
N
Insertion
Sort
O(N3/2)? O(N5/4)? O(N7/6)?
Shell Sort: Analysis
So we have 3 nested loops, but Shell Sort is still betterthan Insertion Sort! Why?
Generic Sort
-
8/12/2019 All Sorting
137/138
Generic Sort
So far we have methods to sortintegers. What about Strings?Employees? Cookies?
A new method for each class? No! In order to be sorted, objects should
be comparable (less than, equal,
greater than). Solution:
use an interfacethat has a method tocompare two objects.
Other kinds of sort
-
8/12/2019 All Sorting
138/138
Other kinds of sort
Heap sort. We will discuss this aftertree.
Postman sort / Radix Sort.
etc.