priority queues, heaps & leftist trees cse, postech
TRANSCRIPT
Priority Queues, Heaps & Leftist Trees
CSE, POSTECH
2 2
Priority Queues
A priority queue is a collection of zero or more elements each element has a priority or value
Unlike the FIFO queues, the order of deletion from a priority queue (e.g., who gets served next) is determined by the element priority
Elements are deleted by increasing or decreasing order of priority rather than by the order in which they arrived in the queue
3 3
Priority Queues
Operations performed on priority queues1) Find an element, 2) insert a new element, 3) delete an
element, etc.
Two kinds of (Min, Max) priority queues exist In a Min priority queue, find/delete operation
finds/deletes the element with minimum priority In a Max priority queue, find/delete operation
finds/deletes the element with maximum priority Two or more elements can have the same priority
4 4
Priority Queues
See ADT 12.1 & Program 12.1 for max priority queue specification
What would be different for min priority queue specification?
Read Examples 12.1, 12.2 What are other examples in our daily lives that
utilize the priority queue concept?
5 5
Implementation of Priority Queues
Implemented using heaps and leftist trees
Heap is a complete binary tree that is efficiently stored using the array-based representation
Leftist tree is a linked data structure suitable for the implementation of a priority queue
6 6
Max (Min) Tree
A max tree (min tree) is a tree in which the value in each node is greater (less) than or equal to those in its children (if any)– See Figure 12.1, 12.2 for examples– Nodes of a max or min tree may have more than two
children (i.e., may not be binary tree)
7 7
Max Tree Example
8 8
Min Tree Example
9
A max heap (min heap) is a max (min) tree that is also a complete binary tree– Figure 12.1 (a) & (c) are max heap– Figure 12.2 (a) & (c) are min heap– Why aren’t Figure 12.1 (b) & 12.2 (b) max/min heap?– How can you change the Figure 12.1 (b) & 12.2 (b) so
that they are max/min heap?
9
Heaps - Definitions
30
25
14
12 7
10 8 6
9
6
5
10 10
Max Heap with 9 Nodes
11 11
Min Heap with 9 Nodes
12 12
Array Representation of Heap
A heap is efficiently represented as an array.
13 13
Heap Operations
When n is the number of elements (heap size), Insertion O(log2n)
Deletion O(log2n) Initialization O(n)
14 14
Insertion into a Max Heap
9
8
6
7
7 2 6
5 1 5• New element is 5• Are we finished?
15 15
9
8
6
7
7 2 6
5 1 20
• New element is 20• Are we finished?
Insertion into a Max Heap
16 16
9
8
6
7
2 6
5 1 7
20
• Exchange the positions with 7• Are we finished?
Insertion into a Max Heap
17 17
9
6
7
2 6
5 1 7
8
20
Insertion into a Max Heap
• Exchange the positions with 8• Are we finished?
18 18
6
7
2 6
5 1 7
8
9
20
Insertion into a Max Heap
• Exchange the positions with 9• Are we finished?
19 19
Complexity of Insertion
See also Figure 12.3 for another insertion example
At each level, we do (1) work Thus the time complexity is O(height) = O(log2n),
where n is the heap size
20 20
Deletion from a Max Heap
• Max element is in the root• What happens when we delete an element?
20
6
7
2 6
5 1 7
15
8
9
21 21
Deletion from a Max Heap
• After the max element is removed.• Are we finished?
6
7
2 6
5 1 7
15
8
9
22 22
Deletion from a Max Heap
• Heap with 10 nodes. • Reinsert 8 into the heap.
6
7
2 6
5 1 7
15
8
9
23 23
Deletion from a Max Heap
• Reinsert 8 into the heap.• Are we finished?
6
7
2 6
5 1 7
15
9
8
24 24
Deletion from a Max Heap
• Exchange the position with 15• Are we finished?
6
7
2 6
5 1 7
9
15
8
25 25
Deletion from a Max Heap
6
7
2 6
5 1 7
8
15
9
• Exchange the position with 9• Are we finished?
26 26
Complexity of Deletion
See also Figure 12.4 for another deletion example
The time complexity of deletion is the same as insertion
At each level, we do (1) work Thus the time complexity is O(height) = O(log2n),
where n is the heap size
27 27
Max Heap Initialization
• Heap initialization means to construct a heap by adjusting the tree if necessary• Example: input array = [-,1,2,3,4,5,6,7,8,9,10,11]
28 28
Max Heap Initialization
- Start at rightmost array position that has a child.- Index is floor(n/2).
29 29
Max Heap Initialization
30 30
Max Heap Initialization
31 31
Max Heap Initialization
32 32
Max Heap Initialization
33 33
Max Heap Initialization
•Are we finished?•Done!
34 34
Complexity of Initialization
• See Figure 12.5 for another initialization example
• Height of heap = h.• Number of nodes at level j is <= 2j-1.• Time for each node at level j is O(h-j+1).• Time for all nodes at level j is <= 2j-1(h-j+1) = t(j).• Total time is t(1) + t(2) + … + t(h) = O(2h) = O(n).
35 35
The Class MaxHeap
See Program 12.2 for Insertion into a MaxHeap See Program 12.3 for Deletion from a MaxHeap See Program 12.4 for Initializing a nonempty
MaxHeap
36
PUSH OPERATIONtemplate<class T>void maxHeap<T>::push(const T& theElement){// Add theElement to heap.
// increase array length if necessaryif (heapSize == arrayLength - 1) {// double array lengthchangeLength1D(heap, arrayLength, 2 * arrayLength); arrayLength *= 2; }
// find place for theElement // currentNode starts at new leaf and moves up tree int currentNode = ++heapSize; while (currentNode != 1 && heap[currentNode / 2] < theElement) { // cannot put theElement in heap[currentNode] heap[currentNode] = heap[currentNode / 2]; // move element down currentNode /= 2; // move to parent }
heap[currentNode] = theElement;}
36
37
POP OPERATIONtemplate<class T>void maxHeap<T>::pop(){// Remove max element. // if heap is empty return null if (heapSize == 0) // heap empty throw queueEmpty();
// Delete max element heap[1].~T();
// Remove last element and reheapify T lastElement = heap[heapSize--];
// find place for lastElement starting at root int currentNode = 1, child = 2; // child of currentNode while (child <= heapSize) { // heap[child] should be larger child of currentNode if (child < heapSize && heap[child] < heap[child + 1]) child++;
// can we put lastElement in heap[currentNode]? if (lastElement >= heap[child]) break; // yes
// no heap[currentNode] = heap[child]; // move child up currentNode = child; // move down a level child *= 2; } heap[currentNode] = lastElement;}3
7
38
INITIALIZEtemplate<class T>void maxHeap<T>::initialize(T *theHeap, int theSize){// Initialize max heap to element array theHeap[1:theSize]. delete [] heap; heap = theHeap; heapSize = theSize;
// heapify for (int root = heapSize / 2; root >= 1; root--) { T rootElement = heap[root];
// find place to put rootElement int child = 2 * root; // parent of child is target // location for rootElement while (child <= heapSize) { // heap[child] should be larger sibling if (child < heapSize && heap[child] < heap[child + 1]) child++;
// can we put rootElement in heap[child/2]? if (rootElement >= heap[child]) break; // yes
// no heap[child / 2] = heap[child]; // move child up child *= 2; // move down a level } heap[child / 2] = rootElement; }}
38
39 39
Exercise 12.7
Do Exercise 12.7 – theHeap = [-, 10, 2, 7, 6, 5, 9, 12, 35, 22, 15, 1, 3, 4]
40 40
Exercise 12.7 (a)
12.7 (a) – complete binary tree
41 41
Exercise 12.7 (b)
12.7 (b) – The heapified tree
42 42
Exercise 12.7 (c)
12.7 (c) – The heap after 15 is inserted is:
43 43
Exercise 12.7 (c)
12.7 (c) – The heap after 20 is inserted is:
44 44
Exercise 12.7 (c)
12.7 (c) – The heap after 45 is inserted is:
45 45
Exercise 12.7 (d) 12.7 (d) – The heap after the first remove max operation is:
46 46
Exercise 12.7 (d) 12.7 (d) – The heap after the second remove max operation is:
47 47
Exercise 12.7 (d) 12.7 (d) – The heap after the third remove max operation is:
48 48
Leftist Trees
Despite heap structure being both space and time efficient, it is NOT suitable for all applications of priority queues
Leftist tree structures are useful for applications– to meld (i.e., combine) pairs of priority queues– using multiple queues of varying size
Leftist tree is a linked data structure suitable for the implementation of a priority queue
A tree which tends to “lean” to the left.
49 49
Leftist Trees
External node – a special node that replaces each empty subtree
Internal node – a node with non-empty subtrees Extended binary tree – a binary tree with external
nodes added (see Figure 12.6)
50 50
Extended Binary Tree
Figure 12.6 s and w values
51 51
Height-Biased Leftist Tree (HBLT) Let s(x) be the length (height) of a shortest path from
node x to an external node in its subtree If x is an external node, s(x) = 0 If x is an internal node, s(x) = min {s(L), s(R)} + 1, where L
and R are left and right children of x A binary tree is a height-biased leftist tree (HBLT) iff at
every internal node, the s value of the left child is greater than or equal to the s value of the right child
Is Figure 12.6(a) an HBLT? If not, how can we change it to become an HBLT?
52 52
Max/Min HBLT
A max HBLT is an HBLT that is also a max tree– Are the trees of Figure 12.1 are also max HBLTs?– YES!
A min HBLT is an HBLT that is also a min tree– Are the trees of Figure 12.2 are also min HBLTs?– YES!
53 53
Weight-Biased Leftist Tree (WBLT)
Let the weight, w(x), of node x to be the number of internal nodes in the subtree with root x
If x is an external node, w(x) = 0 If x is an internal node, its weight is one more than the sum
of the weights of its children A binary tree is a weight-biased leftist tree (WBLT) iff at
every internal node, the w value of the left child is greater than or equal to the w value of the right child
54 54
Weight-Biased Leftist Tree (WBLT)
A max (min) WBLT is a max (min) tree that is also a WBLT
Is Figure 12.6(a) an WBLT? If not, how can we change it to become an WBLT?
55 55
Operations on a Max HBLT
Read Section 12.5.2 for Insertion into a Max HBLT Read Section 12.5.3 for Deletion from a Max HBLT Read Section 12.5.4 for Melding Two Max HBLTs See Figure 12.7 and read Example 12.3 for Melding max
HBLTs Read Section 12.5.5 for Initialization of a Max HBLT See Figure 12.8 for Initializing a max HBLT
56 56
Melding max HBLTs
Figure 12.7 Melding (combining) max HBLTs
57 57
The Class maxHBLT
See Program 12.5 for Melding of two leftist trees See Program 12.6 for meld, push and pop
methods See Program 12.7 for Initializing a max HBLT
Do Exercise 12.19
58 58
Exercise 12.19
(a) The first six calls to meld create the following six max leftist trees.
5
3
7
6
20
8
9
2
15
12
30
17
The next three calls to meld combine pairs of these trees to create the following three trees:
5
3
7
6 9
2
20
8 15
12
30
17
What would be next?
59 59
Applications of Heaps
Sort (heap sort) Machine scheduling Huffman codes
60 60
Heap Sort
use element key as priority
Algorithm put elements to be sorted into a priority queue
(i.e., initialize a heap) extract (delete) elements from the priority queue
– if a min priority queue is used, elements are extracted in non-decreasing order of priority
– if a max priority queue is used, elements are extracted in non-increasing order of priority
61 61
Heap Sort Example
After putting into a max priority queue
62 62
Sorting Example
After first remove max operation
63 63
Sorting Example
After second remove max operation
64 64
Sorting Example
After third remove max operation
65 65
Sorting Example
After fourth remove max operation
66 66
Sorting Example
After fifth remove max operation
67 67
Complexity Analysis of Heap Sort
See Program 12.8 for Heap Sort See Figure 12.9 for another Heap Sort example
Heap sort n elements.– Initialization operation takes O(n) time– Each deletion operation takes O(log n) time– Thus, the total time is O(n log n) - Why?
The heap has to be reinitialized (melded) after each delete operation
– compare with O(n2) for sort methods of Chapter 2
68 68
Machine Scheduling Problem
m identical machines n jobs to be performed The machine scheduling problem is to assign jobs
to machines so that the time at which the last job completes is minimum
69 69
Machine Scheduling Example
3 machines and 7 jobs job times are [6,2,3,5,10,7,14] What are some possible schedules? A possible schedule:
What algorithm did we use for the above scheduling? What are other scheduling algorithms
70 70
Machine Scheduling Example
What is the finish time (length) of the schedule?
21 Objective: Find schedules with minimum finish time Minimum finish time scheduling is NP-hard.
71 71
The class of problems for which no one has developed a polynomial time algorithm.
No algorithm whose complexity is O(nk ml) is known for any NP-hard problem (for any constants k and l)
NP stands for Nondeterministic Polynomial NP-hard problems are often solved by heuristics (or
approximation algorithms), which do not guarantee optimal solutions
Longest Processing Time (LPT) rule is a good heuristic for minimum finish time scheduling.
NP-hard Problems
72 72
LPT Schedule & Example
Longest Processing Time (LPT) first Jobs are scheduled in the descending order
14, 10, 7, 6, 5, 3, 2 Each job is scheduled on the machine
on which it finishes earliest
finish time is 16!
73 73
LPT Schedule & Example
What is the minimum finish time with thee machines for jobs (2, 14, 4, 16, 6, 5, 3)?
See Figure 12.10
74 74
LPT using a Min Heap
Min Heap has the finish times of the m machines. Initial finish times are all 0. To schedule a job, remove the machine with
minimum finish time from the heap. Update the finish time of the selected machine and
put the machine back into the min heap. See Program 12.9 for LPT scheduler
75 75
Complexity Analysis of LPT
When n m (i.e., more machines than jobs), LPT takes (1) time
When n m, (i.e., more jobs than machines), the heap sort takes O(n log n) time
Heap initialization takes O(m) time DeleteMin operation takes O(log m) time Insert operation takes O(log m) time n DeleteMin and n Insert takes O(n log m) time Thus, the total time is O(n log n + n log m) = O(n log n)
time (as n > m)
76 76
Huffman Codes For text compression, the LZW method relies on the
recurrence of substrings in a text Huffman codes is another text compression method,
which relies on the relative frequency (i.e., the number of occurrences of a symbol) with which different symbols appear in a text
Uses extended binary trees Variable-length codes that satisfy the property, where no
code is a prefix of another Huffman tree is a binary tree with minimum weighted
external path length for a given set of frequencies (weights)
77 77
Huffman Codes
READ Section 12.6.3
READ all of Chapter 12