1 theory i algorithm design and analysis (9 – priority queues: fibonacci heaps) t. lauer
TRANSCRIPT
1
Theory I
Algorithm Design and Analysis
(9 – Priority queues: Fibonacci heaps)
T. Lauer
2
Priority Queues
• Data structures for managing elements (insert, delete)
– Stack: “last in – first out”
– Queue: “first in – first out”
– Dictionary: “find any element” (lookup)
• New structure: priority queue
– a priority is assigned to each inserted element
– the element with highest priority can be accessed and removed
– Real-world example: “to-do” list:
• New tasks are added
• The most urgent/important one is carried out first
3
Priority queues: elements
The key of an element (node) in a priority queue represents its assigned priority.
The priority has to be of an ordered type (e.g. int, long, float, …) so the can be compared.
The priority is not a unique identifier of an element (e.g. for lookup). Hence, several elements
could have the same priority.
We store the key as an int (a smaller value represents a higher priority).
class Node { Object content; // content int key; // priority}
4
Priority queues: operations
Operations on a priority queue Q:
Q.insert(int k): Insert a new element with key (= priority) k.
Q.accessmin(): Return the element with minimum key (= highest priority).
Q.deletemin(): Remove the element with minimum key.
Q.decreasekey(Node N, int k): Decrease the key of node N to the value k.
Q.delete(Node N): Delete the given node N.
Q.meld(PriorityQueue P): Merge Q with the priority queue P.
Q.isEmpty(): Return true iff Q is empty.
Remark: Efficient lookup of an element with a given key is not supported in priority queues. Hence, decreasekey and delete require direct access to the respective element N (not just the key).
5
Fibonacci heaps: idea
• List of multiway trees which are all heap-ordered.
Definition:
A tree is called heap-ordered if the key of each node is greater than or equal to
the key of its parent (if there is a parent).
• The roots of the trees are organized in a doubly-connected circular list (root list).
• The Fibonacci heap can be accessed through a pointer to a (the) node with
minimum key.
2 19
13 45 8
36 21
24
15 83 52
79
117
6
Representation of trees
• Child-sibling representation (cf. Exercise sheet 2)
• To be able to go upward in the tree, a pointer to the parent node is added.
• In order to realize deletion of a child (and the concatenation of two child lists) in
O(1), we use doubly-connected circular lists.
• Hence, each node contains 4 pointers: child, parent, left, right
2
13 45 8
36 2115
2
13 45 8
36 2115
2
13 45 8
36 2115
7
Fibonacci heaps: node format
class FibNode { Object content; // the content int key; // priority FibNode parent, child; // pointers to parent and one child FibNode left, right; // pointers to left and right sibling
int rank; // number of children of this node boolean mark; // marker}
rank indicates the number of children of the node (the outdegree of the node)
The meaning of mark will become clear later. It is true if the node has lost any of its children since it last became the child of another node.
8
Fibonacci heap: detailed figure
2 19
13 45 8
36 21
24
15 83 52
79
117
9
Fibonacci heap: simplified figure
2 19
13 45 8
36 21
24
15 83 52
79
117
10
Fibonacci heaps: operations
• Q.accessmin(): Return the node Q.min (or null if Q is empty).
• Q.insert(int k): Create a new node N with key k and insert it in the root list
of Q. If k < Q.min.key, update the minimum pointer (set Q.min = N).
5 19
13 45 8
36 21
24
15 83 52
79
117
11
Manipulation of trees in Fibonacci heaps
There are only three ways how a tree can change in a Fibonacci heap:
link: “growth” of trees
Two trees are linked into a new tree.
cut: cut out a subtree
A subtree is cut out of a tree and inserted in the root list as a new tree.
remove: decompose a tree at the root
Removes the root of a tree from the root list and replaces it by the list of its
children, i.e. the children become new roots.
12
Tree manipulation: link
link:
Input: two root nodes of the same rank k
Method: joins two trees of the same rank by making the root with larger key a child of
the root with smaller key. Unmark the new child if it was marked.
The total number of trees in the Fibonacci heap is reduced by 1, the number of
nodes does not change.
Output: one root node of rank k+1
Cost:
2
13 8
36 21
24
83 52
2
13 8
36 21
24
83 52
13
Tree manipulation: cut
cut:
Input: one non-root node
Method: separate the node (including the subtree rooted in it) from its parent and
insert it as a new tree in the root list.
The total number of trees is increased by 1, the number of nodes does not change.
Cost:
2
13 45 8
36 2115
26
2
13 45 8
36
21
15
26
14
Tree manipulation: remove
remove:
Input: one root node (with rank k)
Method: delete the root of a tree and inserts its k children in the root list. The number
of trees is increased by k-1, the number of nodes is decreased by 1.
Cost: [if the parent pointers of the children are not
deleted!]
2
13 45 8
36 2115
26
13 45 8
36 2115
26
15
Further operations
• By combining the basic methods
– link
– cut
– remove
we can construct the missing operations
– deletemin
– decreasekey
– delete
16
Deletion of the minimum node
Q.deletemin():
• If Q is empty, return null.
• Otherwise:
– Delete the minimal node (using remove).
– „Consolidate“ (clean up) the root list:
• Join (link) two root nodes of the same rank, until only nodes with distinct rank appear in the root list.
• While doing this, remove all “dangling” parent pointers of root nodes.
– Find the new minimum among the root nodes.
– Return the deleted node.
17
deletemin: example
2 19
13 45 8
36 21
24
15 83 52
117
64
18
deletemin: example
2
1913 45 8
36 21 2415
83 52
117
64
19
deletemin: example
1913 45 8
36 21 2415
83 52
117
64
20
deletemin: example
19
13
45 8
36 21 24
15 83 52
117
64
21
deletemin: example
19
13
45
8
36 21
24
15 83 52
117
64
22
deletemin: example
19
13 458
36 21
24
15 83 52
117
64
23
deletemin: example
1913 458
36 21 2415
83 52
117
64
24
decreasekey
Q.decreasekey(FibNode N, int k):
• Set the key value of N to k.
• If the heap order is violated (k < N.parent.key):
– Cut N off its parent (using cut)
– If the parent is marked (N.parent.mark == true), also cut it from its own
parent; if that parent is marked, also cut it, etc. (“cascading cuts”)
– Mark the node whose child was last cut (unless it is a root node).
– Update the minimum pointer (if k < min.key).
25
decreasekey: example 1
2 19
13 45 8
36 21
24
15 83 52
79
117
26
decreasekey: example 1
2 19
13 45 8
36 21
24
15 83 1
79
117
27
decreasekey: example 1
2 19
13 45 8
36 21
24
15 83 1
79
117
28
decreasekey: example 1
2 19
13 45 8
36 21
24
15 83
1
79
117
29
decreasekey: example 1
2 19
13 45 8
36 21
24
15 83
1
79
117
30
decreasekey: example 2
2 19
13 45 8
36 21
24
15 83
1
79
117
31
decreasekey: example 2
2 19
13 45 8
36 5
24
15 83
1
79
117
32
decreasekey: example 2
2 19
13 45 8
36
5
24
15 83
1
79
117
33
decreasekey: example 2
2 19
13 45 8
36
5
24
15 83
1
79
117
34
decreasekey: example 2
2 19
13 45
8
36
5
24
15 83
1
79
117
35
Marking history of a node
• A node N is unmarked when it becomes the child of another node
(via link).
• When N loses a child node (via cut), N is marked (unless N is a root).
• When a marked node loses another child, it is cut off itself and becomes
a root.
Hence, if a node N is marked we know that N has lost one child since N
itself was last made the child of another node.
Also, a node that is not a root cannot have lost more than one child!
36
Deletion of a given node
Q.delete(FibNode N):
• Set the key of N to a value smaller than the current minimum, e.g.
Q.decreasekey(N, -)
• Then perform Q.deletemin().
37
delete: example
2 19
13 45 8
36 21
24
15 83 52
79
7
38
delete: example
2 19
13 45 8
36 21
-
15 83 52
79
7
39
delete: example
2 19
13 45 8
36 21
-
15
83 52
79
7
40
delete: example
2 19
13 45 8
36 2115
83 52
79
7
41
delete: example
2 19
13 45 8
36 2115
83
52
79
7
42
delete: example
2 19
13 45 8
36 2115
83 52
79
7
43
delete: example
2 19
13 45 8
36 2115
83 52
79
7
44
Other operations
• Q.meld(FibHeap P): Append the root list of P to the root list of Q. Then update
the minimum pointer of Q: if P.min.key < Q.min.key, set Q.min = P.min.
• Q.isEmpty(): If Q.size == 0, return true; otherwise false.
45
Analysis
Complexity of deletemin():
remove: O(1)
consolidate: ?
updatemin: O(#root nodes after consolidate) =
After consolidating there is only one root with any given rank (otherwise, they would
have been linked).
We define maxRank(n) as the largest possible rank a root node can have in a
Fibonacci heap of size n. (We will calculate maxRank(n) later.)
Now we have to determine the complexity of consolidate.
46
deletemin: consolidating the root list
• How can we implement consolidate efficiently?
• Observations:
– Obviously, each root node has to be visited at least once
– At the end, each rank may occur at most once
• Idea: Insert the root nodes in a temporary array. Each node is inserted at the
position corresponding to its rank. If a position is occupied, we know that we have
already seen another node with the same rank. We can now link the two nodes
and (try to) insert the root of the resulting tree at the next higher array position.
47
consolidate: example
1913 45 8
36 21 2415
52
117
2
48
consolidate: example
1913 45 8
36 21 2415
52
117
0 1 2 3 4 5Rank array:
49
consolidate: example
1913 45 8
36 21 2415
117
0 1 2 3 4 5Rank array:
52
50
consolidate: example
19
13 45 8
36 21
24
15
117
0 1 2 3 4 5Rank array:
52
51
consolidate: example
19
13
45 8
36 21
24
15
117
0 1 2 3 4 5Rank array:
52
52
consolidate: example
19
13 45
8
36 21
24
15
117
0 1 2 3 4 5Rank array:
52
53
consolidate: example
19
13 45
8
36 21
24
15
117
52
54
Analysis of consolidate
rankArray = new FibNode[maxRank(n)+1]; // create the array
for “each FibNode N in rootlist” {
while (rankArray[N.rank] != null) { // position
occupied
N = link(N, rankArray[N.rank]); // link tree roots
rankArray[N.rank-1] = null; // delete old pos.
}
rankArray[N.rank] = N; // insert in array
}
55
Analysis of the for-loop
for “each FibNode N in rootlist” { while (rankArray[N.rank] != null) { N = link(N, rankArray[N.rank]); rankArray[N.rank-1] = null; } rankArray[N.rank] = N;}
Let rbefore = #root nodes before consolidate and rafter = #root nodes after consolidate.
• The total number of link operations in the loop is rbefore- rafter.
• The total number of array modifications in the loop is #links + rbefore
(because each link and each iteration of the loop ends with one modification).Since rbefore = rafter + #links, we get 2·#links + rafter array modifications.
Therefore, the complexity of the for-loop is#links·O(1) + (2·#links + rafter)·O(1) = O(#links) + O(maxRank(n))
56
Complexity of deletemin
remove: O(1)
Creating the rank array: O(maxRank(n))
for-loop: O(#links) + O(maxRank(n))
Update minimum pointer: O(maxRank(n))
Total cost: O(#links) + O(maxRank(n))
57
Analysis
Complexity of decreasekey():
Set key to new value: O(1)
cut: O(1)
Cascading cuts: #cuts · O(1)
Mark: O(1)
Total cost: O(#cuts)
58
Analysis
Complexity of delete():
• Sum of the costs for decreasekey and deletemin
Total cost: O(#cuts) + O(#links) + O(maxRank(n))
59
Amortized analysis
Observations:
For deletemin, the number of link operations affects the total cost.
For decreasekey, it is the number of cascading cuts.
Idea: Pay those costs from savings (“accounting method”)!
Assumption: the cost for each link and each cut is 1€.
(1) Make sure that for each root node there is always 1€ on the savings account,
so we can pay for the link operation if that node becomes the child of another
one.
(2) Make sure that for each marked node, there are always 2€ on the savings
account, so we can
– pay the cut operation for that node during “cascading cuts” and
– still have 1€ left as savings for the new root node
60
Example
913 45 3
36 21 24
75
52
79
107
1147
36
3214
50 39
61
61
Increase the savings
For which operations do we have to pay extra (in order to gain savings for later)?
New root nodes originate from:
– insert: attach 1€ to each new node inserted in the root list
– decreasekey: pay 1€ extra for the cut-off node.
– deletemin: during remove, pay 1€ for each child of the removed node,
i.e. up to maxRank(n) €.
Marked nodes may only result at the end of decreasekey:
– For each mark operation pay an extra 2€ for the marked node.
62
Amortized cost of insert
Create the node: O(1)
Insert into root list: O(1) + O(1)
Total amortized cost: O(1)
63
Amortized cost of deletemin
remove: O(1) + O(maxRank(n))
Create the rank array: O(maxRank(n))
link operations: #links · O(1) paid from savings!
Other insertions: O(maxRank(n))
Update minimum pointer: O(maxRank(n))
Total amortized cost: O(maxRank(n))
64
Example
913 45 3
36 21 24
75
52
79
107
1147
36
3214
50 39
61
65
Example
913 45 36 21
24
75
52
79
107
1147
36
3214
50 39
61
remove: Pay 1€ extra for each child of the removed node.
66
Example
913 45
36
21
24
75
52
79
107
1147
36
3214
50 39
61
link: Pay all link operations from the savings.
67
Example
913 45
36
2124
75
52
79
107
1147
36
3214
50 39
61
link: Pay all link operations from the savings.
68
Example
913
45
36
2124
75
52
79
107
1147
36
3214
50 39
61
link: Pay all link operations from the savings.
69
Example
9
1345
36
2124
75
52
79
107
1147
36 3214
50 39
61
link: Pay all link operations from the savings.
70
Example
9
13
45
36
21
24
75
52
79
10
7
1147
36
3214
50 39
61
link: Pay all link operations from the savings.
71
Amortized cost of decreasekey
Set key to new value: O(1)
cut: O(1) + O(1)
Cascading cuts: #cuts · O(1) paid from savings!
Mark: O(1) + 2 · O(1)
Total amortized cost: O(1)
72
Example
913 45 3
36 21 24
75
52
79
7
1147
36
3214
50 39
73
Example
913 45 3
36 21 24
20
52
79
7
1147
36
3214
50 39
Pay 1€ extra for the initial cut operation.
74
Example
913 45 3
36 21 24
20
52
79
7
1147
36
3214
50 39
All cascading cuts are paid form savings.
75
Example
913 45 3
36 21 24
20
52
79
7
1147
36
3214
50
39
All cascading cuts are paid form savings.
76
Example
913 45 3
36 21 24
20
52
79
7
1147
36
3214
50
39
When marking the last parent, pay an additional 2€.
77
Amortized cost of delete
• Sum of the costs of decreasekey and deletemin:
O(1) + O(maxRank(n))
Total amortized cost: O(maxRank(n))
78
Amortized Analysis
Amortized costs
• Insert: O(1)
• Accessmin: O(1)
• Deletemin: O(maxRank(n))
• Decreasekey: O(1)
• Delete: O(maxRank(n))
• Meld: O(1)
To do: show that maxRank(n) = O(log n). I.e. the largest possible rank of a node in a
Fibonacci heap is logarithmic in the size n of the Fibonacci heaps.
79
Calculation of maxRank(n)
Lemma 1:
Let N be a node in a Fibonacci heap and k = N.rank. Consider the children
C1, ..., Ck of N in the order, in which they have beed added to N (with link).
Then:
(1) C1.rank ≥ 0
(2) Ci.rank ≥ i - 2 for i = 2, ..., k
Proof: (1) obvious
(2) When Si was made achild of N, C1, ..., Ci-1 had already been children of N,
i.e. we had N.rank ≥ i-1. Since link always links two nodes of the same rank,
at the time of linking Ci.rank ≥ i-1. Since then, Ci cannot have lost more than
one of its children (otherwise it would have been cut off too), hence:
Ci.rank ≥ i - 2
80
Calculation of maxRank(n)
Lemma 2:
Let N be a node in a Fibonacci heap and k = N.rank.
Let size(N) be the number of nodes in the subtree rooted in N.
Then: size(N) ≥ Fk+2 ≥ 1.618k (Fk = k-th Fibonacci number)
i.e. a node with k children is root of a subtree with at least Fk+2 nodes.
Proof: Let S(k) = min {size(N) | N with N.rank = k}, i.e. the smallest possible size
of a tree whose root has rank k. (Obviously, S(0) = 1.)
Again, let C1, ..., Ck be the children of N in the order of their linking to N.
Then
(Lemma
1)
k
i
k
iS
kSS
rankCSrankCSrankCSNsize
2
21
)2(2
)2(...)22(11
).(...).().(1)(
81
Calculation of maxRank(n)
Theorem:
The maximum rank maxRank(n) of any node in a Fibonacci heap with n nodes
is bounded by O(log n).
Proof: Let N be a node in a Fibonacci heap with n nodes and let k = N.rank.
Then n ≥ size(N) ≥ 1.618k (cf. Lemma 2)
Hence k ≤ log1.618(n) = O(log n)
82
Conclusion
Linear list (Min-)heap Fibonacci heap
insert: O(1) O(log n) O(1)
accessmin: O(1) O(1) O(1)
deletemin: O(n) O(log n) O(log n)*
decreasekey: O(1) O(log n) O(1)*
delete: O(n) O(log n) O(log n)*
meld: O(1) O(m log(n+m)) O(1)
*Amortized cost