CISC 235: Topic 7
Priority Queues and Binary Heaps
CISC 235 Topic 7 2
Outline
• Priority Queues• Binary Heaps
– Ordering Properties– Structural Property
• Array Representation• Algorithms and Analysis of Complexity
• insert• minimin• extractMin• buildHeap
CISC 235 Topic 7 3
Priority Queues
A queue that is ordered according to some priority value
Standard Queue Priority Queue
enqueue dequeue enqueue dequeueAdd at Remove Add Remove end from front according from
frontto priorityvalue
CISC 235 Topic 7 4
Example Applications
Line-up of Incoming Planes at AirportPossible Criteria for Priority?
Operating Systems Priority Queues?
Several criteria could be mapped to a priority status
CISC 235 Topic 7 5
Min-Priority Queue Operations
insert( S, x ) – Inserts element x into set S, according to its priority
minimum( S ) – Returns, but does not remove, element of S with the smallest key
extractMin( S ) – Removes and returns the element of S with the smallest key
CISC 235 Topic 7 6
Possible Implementations?
CISC 235 Topic 7 7
Binary Heaps
A binary tree with an ordering property and a structural property
Ordering Property: Min Heap– The element in the root is less than or equal to all elements in
both its sub-trees– Both of its sub-trees are Min Heaps
Ordering Property: Max Heap– The element in the root is greater than or equal to all elements in
both its sub-trees– Both of its sub-trees are Max Heaps
CISC 235 Topic 7 8
Binary Heap Structural Property
A binary heap is a binary tree that is completely filled, with the possible exception of the bottom level, which is filled from left to right with no missing nodes.
CISC 235 Topic 7 9
Which Tree is a Min Heap?
68
1621
13
14
65
31 19
26 32
68
1621
13
24
65
31 19
26 32
CISC 235 Topic 7 10
A Min Heap and its Array Representation
68
1621
13
24
65
31 19
26 32
13 21 16 24 31 19 68 65 26 32 0 1 2 3 4 5 6 7 8 9 10 11 12
1
32
75 64
8 109
Note: Only the array exists in memory
heap
CISC 235 Topic 7 11
Locating Parents & Children
parent( i )
return i / 2
left( i )
return 2i
right( i )
return 2i + 1
CISC 235 Topic 7 12
Min Heap Operations: Insert
1. Find the next position at which to insert – after the “last” element in the heap – and create a “hole” at that postion
2. “Bubble” the hole up the tree by moving its parent element into it until the hole is at a position where the new element ≤ its children and ≥ its parent:
the percolateUp() operation3. Insert the new element in the hole
CISC 235 Topic 7 13
Insert Step 1: Create hole at next insert position
68
1621
13
24
65
31 19
26 32
13 21 16 24 31 19 68 65 26 32 0 1 2 3 4 5 6 7 8 9 10 11 12
1
32
75 64
8 109
Insert 14
14
11
heap
CISC 235 Topic 7 14
Insert Step 2: Bubble hole up to where element ≤ its children and ≥ its parent
68
1621
13
24
65 31
19
26 32
13 21 16 24 19 68 65 26 32 31 0 1 2 3 4 5 6 7 8 9 10 11 12
1
32
75 64
8 109
Insert 14
14
11
heap
CISC 235 Topic 7 15
Insert Step 2: Bubble hole up to where element ≤ its children and ≥ its parent
68
16
21
13
24
65 31
19
26 32
13 16 24 21 19 68 65 26 32 31 0 1 2 3 4 5 6 7 8 9 10 11 12
1
32
75 64
8 109
Insert 14
14
11
heap
CISC 235 Topic 7 16
Insert Step 3: Insert the new element in the hole
68
16
21
13
24
65 31
19
26 32
13 14 16 24 21 19 68 65 26 32 31 0 1 2 3 4 5 6 7 8 9 10 11 12
1
32
75 64
8 109
Insert 14
11
14
heap
14
CISC 235 Topic 7 17
Analysis of Insert?
Worst-case complexity?
CISC 235 Topic 7 18
Min Heap Operations: MinimumReturn minimum at top of heap: O(1)
68
1621
13
24
65
31 19
26 32
13 21 16 24 31 19 68 65 26 32 0 1 2 3 4 5 6 7 8 9 10 11 12
1
32
75 64
8 109
13
heap
return heap[1];
CISC 235 Topic 7 19
Min Heap Operations: ExtractMin
1. Copy the root (the minimum) to a temporary variable, thus creating a “hole” at the root.
2. Delete the last item in the heap by copying its element to a temporary location.
3. Find a place for that element by bubbling the hole down by moving its smallest child up until the element ≤ its children and ≥ its parent:
the percolateDown() operation.
4. Place the former last element in that hole and return the former root.
CISC 235 Topic 7 20
ExtractMin Step 1:Copy the root to a temporary variable
68
1621
13
24
65
31 19
26 32
13 21 16 24 31 19 68 65 26 32 0 1 2 3 4 5 6 7 8 9 10 11 12
1
32
75 64
8 109
13
minimum
heap
minimum = heap[1];
CISC 235 Topic 7 21
ExtractMin Step 1:thus creating a “hole” at the root
68
1621
24
65
31 19
26 32
21 16 24 31 19 68 65 26 32 0 1 2 3 4 5 6 7 8 9 10 11 12
1
32
75 64
8 109
13
minimum
heap
minimum = heap[1];
CISC 235 Topic 7 22
ExtractMin Step 2: Delete last item in heap by copying it to a temporary location
68
1621
24
65
31 19
26 32
21 16 24 31 19 68 65 26 32 0 1 2 3 4 5 6 7 8 9 10 11 12
1
32
75 64
8 109
13
minimum
heap
32
temp
temp = heap[10];
CISC 235 Topic 7 23
68
1621
24
65
31 19
26
21 16 24 31 19 68 65 26 0 1 2 3 4 5 6 7 8 9 10 11 12
1
32
75 64
8 9
13
minimum
heap
32
temp
ExtractMin Step 2: Delete last item in heap by copying it to a temporary location
CISC 235 Topic 7 24
68
16
21
24
65
31 19
26
16 21 24 31 19 68 65 26 0 1 2 3 4 5 6 7 8 9 10 11 12
1
32
75 64
8 9
13
minimum
heap
32
temp
ExtractMin Step 3: Bubble hole down until temp ≤ its children and ≥ its parent
CISC 235 Topic 7 25
68
16
21
24
65
31
19
26
16 21 19 24 31 68 65 26 0 1 2 3 4 5 6 7 8 9 10 11 12
1
32
75 64
8 9
13
minimum
heap
32
temp
ExtractMin Step 3: Bubble hole down until temp ≤ its children and ≥ its parent
CISC 235 Topic 7 26
68
16
21
24
65
31
19
26
16 21 19 24 31 32 68 65 26 0 1 2 3 4 5 6 7 8 9 10 11 12
1
32
75 64
8 9
13
minimum
heap
32
temp
ExtractMin Step 4: Copy temp to the hole and return the minimum
32
heap[6] = temp;
return minimum;
CISC 235 Topic 7 27
Analysis of ExtractMin?
Worst-case complexity?
CISC 235 Topic 7 28
Insert n Elements into an Initially Empty Heap
Worst-case complexity?
CISC 235 Topic 7 29
Bottom-Up Heap Construction: BuildHeap
If all the keys are given in advance and stored in an array, we can build a heap in worst-case O(n) time.
Note: For simplicity, we’ll describe bottom-up heap construction for a full tree
Algorithm: Call percolateDown() on nodes in non-heap tree in reverse level-order traversal to convert tree to a heap.
CISC 235 Topic 7 30
BuildHeap Algorithm
1. Build (n+1)/2 elementary heaps, with one key each, of the leaves of the tree (no work)
2. Build (n+1)/4 heaps, each with 3 keys, by joining pairs of elementary heaps with their parent as the root. Call percolateDown()
3. Build (n+1)/8 heaps, each with 7 keys, by joining pairs of 3-key heaps with their parent as the root. Call percolateDown()
4. Continue until reach root of entire tree.
CISC 235 Topic 7 31
BuildHeap at Start: Elements are Unordered: Not in Heap Order
14 9 8 25 5 11 27 16 15 4 12 6 7 23 20
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
9
25
16
5
15 4
8
11
6
27
7 2312 20
CISC 235 Topic 7 32
BuildHeap Step 1: Build (n+1)/2 elementary heaps, with one key each
14 9 8 25 5 11 27 16 15 4 12 6 7 23 20
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
9
25
16
5
15 4
8
11
6
27
7 2312 20
CISC 235 Topic 7 33
BuildHeap Step 2: Build (n+1)/4 heaps, with 3 keys each
14 9 8 25 5 11 27 16 15 4 12 6 7 23 20
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
9
25
16
5
15 4
8
11
6
27
7 2312 20
CISC 235 Topic 7 34
BuildHeap Step 2: Call percolateDown( ) for each 3-key heap
14 9 8 25 5 11 27 16 15 4 12 6 7 23 20
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
9
25
16
5
15 4
8
11
6
27
7 2312 20
2725
5 11
CISC 235 Topic 7 35
BuildHeap Step 2: Call percolateDown( ) for each 3-key heap
14 9 8 16 15 4 12 6 7 23 20
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
9
16 15 4
8
6 7 2312 20
2725
5 11
CISC 235 Topic 7 36
BuildHeap Step 2: Call percolateDown( ) for each 3-key heap
14 9 8 15 4 6 20 16 12 7 23
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
9
16
15 4
8
6
7 2312
20 2725
5 11
CISC 235 Topic 7 37
BuildHeap Step 2: Call percolateDown( ) for each 3-key heap
14 9 8 15 4 6 20 16 12 7 23
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
9
16
15 4
8
6
7 2312
20 2725
5 11
CISC 235 Topic 7 38
BuildHeap Step 2: Now we have (n+1)/4 heaps, with 3 keys each
14 9 8 15 4 6 20 16 25 5 12 11 7 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
9
16
15 4
8
6
7 2312
20
25 2711 5
CISC 235 Topic 7 39
BuildHeap Step 3: Build (n+1)/8 heaps, with 7 keys each
14 9 8 15 4 6 20 16 25 5 12 11 7 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
9
16
15 4
8
6
7 2312
20
25 2711 5
CISC 235 Topic 7 40
BuildHeap Step 3: Call percolateDown( ) for each 7-key heap
14 9 8 15 4 6 20 16 25 5 12 11 7 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
9
16
15 4
8
6
7 2312
20
25 2711 5
8 9
CISC 235 Topic 7 41
BuildHeap Step 3: Call percolateDown( ) for each 7-key heap
14 15 4 6 20 16 25 5 12 11 7 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
16
15 4 6
7 2312
20
25 2711 5
8 9
CISC 235 Topic 7 42
BuildHeap Step 3: Call percolateDown( ) for each 7-key heap
14 4 6 15 20 16 25 5 12 11 7 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
16
15
4 6
7 2312
20
25 2711 5
8 9
CISC 235 Topic 7 43
BuildHeap Step 3: Call percolateDown( ) for each 7-key heap
14 4 6 15 5 7 20 16 25 12 11 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
16
15
4 6
7
2312
20
25 2711
5
8 9
CISC 235 Topic 7 44
BuildHeap Step 3: Call percolateDown( ) for each 7-key heap
14 4 6 15 5 7 20 16 25 12 11 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
16
15
4 6
7
2312
20
25 2711
5
8 9
CISC 235 Topic 7 45
BuildHeap Step 3: Now we have (n+1)/8 heaps, with 7 keys each
14 4 6 15 5 7 20 16 25 9 12 11 8 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
16
15
4 6
7
2312
20
25 2711
5
8 9
CISC 235 Topic 7 46
BuildHeap Step 4: Build (n+1)/16 heaps, with 15 keys each (only one here)
14 4 6 15 5 7 20 16 25 9 12 11 8 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
16
15
4 6
7
2312
20
25 2711
5
8 9
CISC 235 Topic 7 47
BuildHeap Step 4: Call percolateDown( ) for each 15-key heap
14 4 6 15 5 7 20 16 25 9 12 11 8 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
14
16
15
4 6
7
2312
20
25 2711
5
8 9
14
CISC 235 Topic 7 48
BuildHeap Step 4: Call percolateDown( ) for each 15-key heap
4 6 15 5 7 20 16 25 9 12 11 8 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
16
15
4 6
7
2312
20
25 2711
5
8 9
14
CISC 235 Topic 7 49
BuildHeap Step 4: Call percolateDown( ) for each 15-key heap
4 6 15 5 7 20 16 25 9 12 11 8 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
16
15
4
6
7
2312
20
25 2711
5
8 9
14
CISC 235 Topic 7 50
BuildHeap Step 4: Call percolateDown( ) for each 15-key heap
4 5 6 15 7 20 16 25 9 12 11 8 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
16
15
4
6
7
2312
20
25 2711
5
8 9
14
CISC 235 Topic 7 51
BuildHeap Step 4: Call percolateDown( ) for each 15-key heap
4 5 6 15 9 7 20 16 25 12 11 8 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
16
15
4
6
7
2312
20
25 2711
5
8
9
14
CISC 235 Topic 7 52
BuildHeap Step 4: Call percolateDown( ) for each 15-key heap
4 5 6 15 9 7 20 16 25 12 11 8 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
16
15
4
6
7
2312
20
25 2711
5
8
9
14
CISC 235 Topic 7 53
BuildHeap Step 4: The Entire Tree is Now a Heap!
4 5 6 15 9 7 20 16 25 14 12 11 8 23 27
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
16
15
4
6
7
2312
20
25 2711
5
8
9
14
CISC 235 Topic 7 54
Heapsort
How could we use a heap to sort an array?