fibonacci heaps

Last updated on Sep 8, 2013 7:00 AM

Copyright © 2013 Kevin Waynehttp://www.cs.princeton.edu/~wayne/kleinberg-tardos

FIBONACCI HEAPS

‣ preliminaries

‣ insert

‣ extract the minimum

‣ decrease key

‣ bounding the rank

‣ meld and delete

2

Ahead. O(1) INSERT and DECREASE-KEY, O(log n) EXTRACT-MIN.

Priority queues performance cost summary

operation linked list binary heap binomial heap Fibonacci heap †

MAKE-HEAP O(1) O(1) O(1) O(1)

IS-EMPTY O(1) O(1) O(1) O(1)

INSERT O(1) O(log n) O(log n) O(1)

EXTRACT-MIN O(n) O(log n) O(log n) O(log n)

DECREASE-KEY O(1) O(log n) O(log n) O(1)

DELETE O(1) O(log n) O(log n) O(log n)

MELD O(1) O(n) O(log n) O(1)

FIND-MIN O(n) O(1) O(log n) O(1)

† amortized

3

Fibonacci heaps

Theorem. [Fredman-Tarjan 1986] Starting from an empty Fibonacci heap,

any sequence of m INSERT, EXTRACT-MIN, and DECREASE-KEY operations

involving n INSERT operations takes O(m + n log n) time.

Fibonacci Heaps and Their Uses in Improved Network Optimization Algorithms

MICHAEL L. FREDMAN

University of California, San Diego, L.a Jolla, California

AND

ROBERT ENDRE TARJAN

AT&T Bell Laboratories, Murray HilI, New Jersey

Abstract. In this paper we develop a new data structure for implementing heaps (priority queues). Our structure, Fibonacci heaps (abbreviated F-heaps), extends the binomial queues proposed by Vuillemin and studied further by Brown. F-heaps support arbitrary deletion from an n-item heap in qlogn) amortized time and all other standard heap operations in o( 1) amortized time. Using F-heaps we are able to obtain improved running times for several network optimization algorithms. In particular, we obtain the following worst-case bounds, where n is the number of vertices and m the number of edges in the problem graph:

( 1) O(n log n + m) for the single-source shortest path problem with nonnegative edge lengths, improved from O(m logfmh+2)n);

(2) O(n*log n + nm) for the all-pairs shortest path problem, improved from O(nm lo&,,,+2,n); (3) O(n*logn + nm) for the assignment problem (weighted bipartite matching), improved from

O(nm log0dn+2)n); (4) O(mj3(m, n)) for the minimum spanning tree problem, improved from O(mloglo&,,.+2,n), where

j3(m, n) = min {i 1 log% 5 m/n). Note that B(m, n) 5 log*n if m 2 n.

Of these results, the improved bound for minimum spanning trees is the most striking, although all the results give asymptotic improvements for graphs of appropriate densities.

Categories and Subject Descriptors: E.l [Data]: Data Structures--trees; graphs; F.2.2 [Analysis of Algorithms and Problem Complexity]: Nonnumerical Algorithms and Problems-computations on discrete structures; sorting and searching; G.2.2 [Discrete Mathematics]: Graph Theory-graph algorithms; network problems; trees General Terms: Algorithms, Theory Additional Key Words and Phrases: Heap, matching, minimum spanning tree, priority queue, shortest Pati

’ A preliminary version of this paper appeared in the Proceedings of the 25th IEEE Symposium on the Foundations of Computer Science (Singer Island, Fla., Oct. 24-26). IEEE, New York, 1984, pp. 338- 346, 0 IEEE. Any portion of this paper that appeared in the preliminary version is reprinted with permission. This research was supported in part by the National Science Foundation under grant MCS 82-0403 1. Authors’ addresses: M. L. Fredman, Electrical Engineering and Computer Science Department, Uni- versity of California, San Diego, La Jolla, CA 92093; R. E. Tarjan, AT&T Bell Laboratories, Murray Hill, NJ 07974. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. 0 1987 ACM 0004-541 l/87/0700-0596 $1.50

Journal ofthe Association for Computing Machinery, Vol. 34, No. 3, July 1987, Pages 596-615.

this statement is a bit weakerthan the actual theorem

4

Fibonacci heaps

Theorem. [Fredman-Tarjan 1986] Starting from an empty Fibonacci heap,

any sequence of m INSERT, EXTRACT-MIN, and DECREASE-KEY operations

involving n INSERT operations takes O(m + n log n) time.

History.

・Ingenious data structure and application of amortized analysis.

・Original motivation: improve Dijkstra's shortest path algorithm

from O(m log n) to O(m + n log n).

・Also improved best-known bounds for all-pairs shortest paths,

assignment problem, minimum spanning trees.

SECTION 19.1

FIBONACCI HEAPS

‣ structure

‣ insert


‣ decrease key


‣ meld and delete

6

Fibonacci heaps

Basic idea.

・Similar to binomial heaps, but less rigid structure.

・Binomial heap: eagerly consolidate trees after each INSERT;

implement DECREASE-KEY by repeatedly exchanging node with its parent.

・Fibonacci heap: lazily defer consolidation until next EXTRACT-MIN;

implement DECREASE-KEY by cutting off node and splicing into root list.

Remark. Height of Fibonacci heap is Θ(n) in worst case, but it doesn't use

sink or swim operations.

・Set of heap-ordered trees.

7

723

30

17

35

26 46

24

39

4118 52

3

44

Fibonacci heap: structure

heap-ordered tree

each child no smallerthan its parent

root

heap H

・Set of heap-ordered trees.

・Set of marked nodes.

8

723

30

17

35

26 46

24

39

4118 52

3

44


min

used to keep trees bushy(stay tuned)

39

18

markedheap H

26

Heap representation.

・Store a pointer to the minimum node.

・Maintain tree roots in a circular, doubly-linked list.

9

723

30

17

35

26 46

24

39

4118 52

3

44


min

39

18

heap H

26

Node representation. Each node stores:

・A pointer to its parent.

・A pointer to any of its children.

・A pointer to its left and right siblings.

・Its rank = number of children.

・Whether it is marked.

10

723

30

17

35

26 46

24

39

4118 52

3

44

Fibonacci heap: representation

min

39

18

heap H children are in a circular doubly-linked list

26

rank = 3

Operations we can do in constant time:

・Determine rank of a node.

・Find the minimum element.

・Merge two root lists together.

・Add or remove a node from the root list.

・Remove a subtree and merge into root list.

・Link the root of a one tree to root of another tree.

11

723

30

17

35

26 46

24

39

4118 52

3

44

Fibonacci heap: representation

min

39

18

heap H

26

rank = 3

12

723

30

17

35

26 46

24

39

4118 52

3

44

Fibonacci heap: notation

min

39

18

rank(H) = 3 trees(H) = 5 marks(H) = 3n = 14

notation meaning

n number of nodes

rank(x) number of children of node x

rank(H) max rank of any node in heap H

trees(H) number of trees in heap H

marks(H) number of marked nodes in heap H

heap H

26

rank = 3rank = 1

Potential function.

13

723

30

17

35

26 46

24

39

4118 52

3

44

Fibonacci heap: potential function

min

39

18

trees(H) = 5 marks(H) = 3Φ(H) = 5 + 2⋅3 = 11

heap H

Φ(H) = trees(H) + 2 ⋅ marks(H)

26

SECTION 19.2

FIBONACCI HEAPS

‣ preliminaries

‣ insert


‣ decrease key


‣ meld and delete

・Create a new singleton tree.

・Add to root list; update min pointer (if necessary).

15

723

30

17

35

26 46

24

39

4118 52

3

44

Fibonacci heap: insert

21

insert 21

min

heap H

39

1826

・Create a new singleton tree.

・Add to root list; update min pointer (if necessary).

16

723

30

17

35

26 46

24

39

4118 52

3

44

Fibonacci heap: insert

21

insert 21

min

heap H

39

1826

Actual cost. ci = O(1).

Change in potential. ∆Φ = Φ(Hi) – Φ(Hi–1) = +1.

Amortized cost. ĉi = ci + ∆Φ = O(1).

17

723

30

17

35

26 46

24

39

4118 52

3

44

Fibonacci heap: insert analysis

min

heap H

21

39

18


26

one more tree;no change in marks

SECTION 19.2

FIBONACCI HEAPS

‣ preliminaries

‣ insert


‣ decrease key


‣ meld and delete

19

Linking operation

Useful primitive. Combine two trees T1 and T2 of rank k.

・Make larger root be a child of smaller root.

・Resulting tree T ' has rank k + 1.

39

4118 52

3

4477

56 24

15

tree T1 tree T2

39

4118 52

3

44

77

56 24

15

tree T'

still heap-ordered

33

33

・Delete min; meld its children into root list; update min.

・Consolidate trees so that no two roots have same rank.

20

Fibonacci heap: extract the minimum

39

4118 52

3

44

1723

30

7

35

26 46

24

min

2626



21


39

411723 18 52

30

7

35

26 46

24

44

min

2626

22




39

411723 18 52

30

7

35

26 46

24

44

mincurrent

2626

23




39

411723 18 52

30

7

35

26 46

24

44

rank

mincurrent

0 1 2 3

2626

24




39

411723 18 52

30

7

35

26 46

24

44

mincurrent

rank

2626

0 1 2 3

25




39

411723 18 52

30

7

35

26 46

24

44

mincurrent

rank

2626

0 1 2 3

26




39

411723 18 52

30

7

35

26 46

24

44

mincurrent

ranklink 23 to 17

2626

0 1 2 3

27




39

4118 52

30

7

35

26 46

24

44

mincurrent

ranklink 17 to 7

17

232626

0 1 2 3

28




39

417

30

18 52

17

35

26 46

24

44

23

min current

ranklink 24 to 7

2626

0 1 2 3

29




39

417

30

18 52

23

17

35

26 46

24 44

min current

rank

2626

0 1 2 3

30




39

417

30

18 52

23

17

35

26 46

24 44

min current

rank

2626

0 1 2 3

31




39

417

30

18 52

23

17

35

26 46

24 44

min current

rank

2626

0 1 2 3

32




39

417

30

18 52

23

17

35

26 46

24 44

min current

ranklink 41 to 18

2626

0 1 2 3

33




3941

7

30

1852

23

17

35

26 46

24

44

min current

rank

2626

0 1 2 3

34




7

30

52

23

17

35

26 46

24 3941

18

44

min current

rank

2626

0 1 2 3

35




7

30

52

23

17

35

26 46

24 3941

18

44

min

stop (no two trees have same rank)

2626

36

Fibonacci heap: extract the minimum analysis

Actual cost. ci = O(rank(H)) + O(trees(H)).

・O(rank(H)) to meld min's children into root list.

・O(rank(H)) + O(trees(H)) to update min.

・O(rank(H)) + O(trees(H)) to consolidate trees.

Change in potential. ∆Φ ≤ rank(H') + 1 – trees(H).

・No new nodes become marked.

・trees(H') ≤ rank(H') + 1.

Amortized cost. O(log n).

・ĉi = ci + ∆Φ = O(rank(H)) + O(rank(H')).

・The rank of a Fibonacci heap with n elements is O(log n).


≤ rank(H) + trees(H) – 1 root nodes

number of roots decreases by 1 aftereach linking operation

≤ rank(H) children

no two trees have same rank after consolidation

Fibonacci lemma(stay tuned)

37

Observation. If only INSERT and EXTRACT-MIN operations, then all trees are

binomial trees.

Binomial heap property. This implies rank(H) ≤ log2 n.

Fibonacci heap property. Our DECREASE-KEY implementation will not preserve

this property, but we will implement it in such a way that rank(H) ≤ logφ n.

Fibonacci heap vs. binomial heaps

B0 B1 B2 B3

we link only trees of equal rank

SECTION 19.3

FIBONACCI HEAPS

‣ preliminaries

‣ insert


‣ decrease key


‣ meld and delete

39

Intuition for deceasing the key of node x.

・If heap-order is not violated, decrease the key of x.

・Otherwise, cut tree rooted at x and meld into root list.

Fibonacci heap: decrease key

decrease-key of x from 30 to 7

24

23 22

50

48 31 17

448 29 10

6

32

30

55

45




40



24

23 22

50

48 31 17

448 29 10

6

32

7

55

45




41


decrease-key of 22 to 4decrease-key of 48 to 3decrease-key of 31 to 2decrease-key of 17 to 1

24

5

22

50

48 31 17

448 29 10

6

32

7

55

45




・Problem: number of nodes not exponential in rank.

42


24

5 4

50

3 2 1

448 29 10

6

32

7

55

45

rank = 4, nodes = 5

43




・Solution: as soon as a node has its second child cut,

cut it off also and meld into root list (and unmark it).

24

46

17

30

23

7

88

26

21

52

39

18

41

38

72


35

marked node:one child already cut

min

44

Case 1. [heap order not violated]

・Decrease key of x.

・Change heap min pointer (if necessary).

24

46

17

30

23

7

88

26

21

52

39

18

41

38

72


35

min

x


45

Case 1. [heap order not violated]


・Change heap min pointer (if necessary).

24

29

17

30

23

7

88

26

21

52

39

18

41

38

72


35

min

x


46

Case 2a. [heap order violated]


・Cut tree rooted at x, meld into root list, and unmark.

・If parent p of x is unmarked (hasn't yet lost a child), mark it;

Otherwise, cut p, meld into root list, and unmark

(and do so recursively for all ancestors that lose a second child).

24

29

17

30

23

7

88

26

21

52

39

18

41

38

72


35

min

x

p


47







24

15

17

30

23

7

88

26

21

52

39

18

41

38

72


35

min

x

p


48







24

15

17

30

23

7

88

26

21

52

39

18

41

38

72


35

min

x

p


49







24 17

30

23

7

88

26

21

52

39

18

41

38


35

15

72

min

p

x


50







24 17

30

23

7

88

26

21

52

39

18

41

38


35

15

72

min

p

x


24

Case 2b. [heap order violated]






5135

24

15

17

30

23

7

88

26

21

52

39

18

41

38

72 24


min


p

x







525

24

15

17

30

23

7

88

26

21

52

39

18

41

38

72 24


min


p

x

53







24

15

17

30

23

7

88

26

21

52

39

18

41

38

72 24



p

x

5

min

54







24

15

17

30

23

7

88

26

21

52

39

18

41

38

72 24



p

x

5

min

second child cut

55







24

26

17

30

23

7

21

52

39

18

41

38

88 24

5


15

72

min


x p

56







24

26

17

30

23

7

21

52

39

18

41

38

88 24

5


15

72

min


x p

second child cut

p'

57







26

17

30

23

7

21

52

39

18

41

38

88

5


15

72

min


x p p'

24

but don'tmark parentif it's a root

p''

58

Actual cost. ci = O(c), where c is the number of cuts.

・O(1) time for changing the key.

・O(1) time for each of c cuts, plus melding into root list.

Change in potential. ∆Φ = O(1) – c.

・trees(H') = trees(H) + c.

・marks(H') ≤ marks(H) – c + 2.

・ΔΦ ≤ c + 2 ⋅ (-c + 2) = 4 – c.


Fibonacci heap: decrease key analysis


each cut (except first) unmarks a nodelast cut may or may not mark a node

SECTION 19.4

FIBONACCI HEAPS

‣ preliminaries

‣ insert


‣ decrease key


‣ meld and delete

60

Analysis summary

Insert. O(1).Delete-min. O(rank(H)) amortized.

Decrease-key. O(1) amortized.

Fibonacci lemma. Let H be a Fibonacci heap with n elements.

Then, rank(H) = O(log n).

number of nodes isexponential in rank

61

Bounding the rank

Lemma 1. Fix a point in time. Let x be a node of rank k, and let y1, …, yk

denote its current children in the order in which they were linked to x. Then:

Pf.

・When yi was linked into x, x had at least i – 1 children y1, …, yi–1.

・Since only trees of equal rank are linked, at that time

rank(yi) = rank(x) ≥ i – 1.

・Since then, yi has lost at most one child (or yi would have been cut).

・Thus, right now rank(yi) ≥ i – 2. ▪

x

y1 y2 yk…

rank(yi) ��

0 i = 1

i � 2 i � 2

62

Bounding the rank



Def. Let Tk be smallest possible tree of rank k satisfying property.

T0 T1 T2 T3 T4 T5

F2 = 1 F3 = 2 F4 = 3 F5 = 5 F6 = 8 F7 = 13

x

y1 y2 yk…

rank(yi) ��

0 i = 1

i � 2 i � 2

63

Bounding the rank



Def. Let Tk be smallest possible tree of rank k satisfying property.

T4 T5

F6 = 8 F7 = 13

T6

F8 = F6 + F7 = 8 + 13 = 21

x

y1 y2 yk…

rank(yi) ��

0 i = 1

i � 2 i � 2

64

Bounding the rank

Lemma 2. Let sk be minimum number of elements in any Fibonacci heap of

rank k. Then sk ≥ Fk+2, where Fk is the kth Fibonacci number.

Pf. [by strong induction on k]

・Base cases: s0 = 1 and s1 = 2.

・Inductive hypothesis: assume si ≥ Fi+2 for i = 0, …, k – 1.

・As in Lemma 1, let let y1, …, yk denote its current children in the order in

which they were linked to x.

sk ≥ 1 + 1 + (s0 + s1 + … + sk–2) (Lemma 1)

≥ (1 + F1) + F2 + F3 + … + Fk (inductive hypothesis)

= Fk+2. ▪ (Fibonacci fact 1)

65

Bounding the rank

Fibonacci lemma. Let H be a Fibonacci heap with n elements.

Then, rank(H) ≤ logφ n, where φ is the golden ratio = (1 + √5) / 2 ≈ 1.618.

Pf.

・Let H is a Fibonacci heap with n elements and rank k.

・Then n ≥ Fk+2 ≥ φk.

・Taking logs, we obtain rank(H) = k ≤ logφ n. ▪

Lemma 2 FibonacciFact 2

66

Fibonacci fact 1

Def. The Fibonacci sequence is: 0, 1, 1, 2, 3, 5, 8, 13, 21, …

Fibonacci fact 1. For all integers k ≥ 0, Fk+2 = 1 + F0 + F1 + … + Fk.

Pf. [by induction on k]

・Base case: F2 = 1 + F0 = 2.

・Inductive hypothesis: assume Fk+1 = 1 + F0 + F1 + … + Fk–1.

Fk+2 = Fk + Fk+1 (definition)

= Fk + (1 + F0 + F1 + … + Fk–1) (inductive hypothesis)

= 1 + F0 + F1 + … + Fk–1 + Fk. ▪ (algebra)

Fk =

��

��

0 k = 0

1 k = 1

Fk�1 + Fk�2 k � 2

67

Fibonacci fact 2

Def. The Fibonacci sequence is: 0, 1, 1, 2, 3, 5, 8, 13, 21, …

Fibonacci fact 2. Fk+2 ≥ φk, where φ = (1 + √5) / 2 ≈ 1.618.

Pf. [by induction on k]

・Base cases: F2 = 1 ≥ 1, F3 = 2 ≥ φ.

・Inductive hypotheses: assume Fk ≥ φk and Fk+1 ≥ φk + 1

Fk+2 = Fk + Fk+1 (definition)

≥ φk – 1 + φk – 2 (inductive hypothesis)

= φk – 2 (1 + φ) (algebra)

= φk – 2 φ2 (φ2 = φ + 1)

= φk. ▪ (algebra)

Fk =

��

��

0 k = 0

1 k = 1

Fk�1 + Fk�2 k � 2

Fibonacci numbers arise both in nature and algorithms.

68

Fibonacci numbers and nature

pinecone

cauliflower

SECTION 19.2, 19.3

FIBONACCI HEAPS

‣ preliminaries

‣ insert


‣ decrease key


‣ meld and delete

70

Fibonacci heap: meld

Meld. Combine two Fibonacci heaps (destroying old heaps).

Recall. Root lists are circular, doubly-linked lists.

39

41

717

18 52

3

30

23

35

26 46

24

44

21

heap H1

min

heap H2

min

71

Fibonacci heap: meld

Meld. Combine two Fibonacci heaps (destroying old heaps).

Recall. Root lists are circular, doubly-linked lists.

39

41

717

18 52

3

30

23

35

26 46

24

44

21

heap H

min

72

Fibonacci heap: meld analysis

Actual cost. ci = O(1).Change in potential. ∆Φ = 0.


39

41

717

18 52

3

30

23

35

26 46

24

44

21

heap H

min


73

Delete. Given a handle to an element x, delete it from heap H.

・DECREASE-KEY(H, x, -∞).

・EXTRACT-MIN(H).

Amortized cost. ĉi = O(rank(H)).

・O(1) amortized for DECREASE-KEY.

・O(rank(H)) amortized for EXTRACT-MIN.

Fibonacci heap: delete


74

Accomplished. O(1) INSERT and DECREASE-KEY, O(log n) EXTRACT-MIN.


operation linked list binary heap binomial heap Fibonacci heap †

MAKE-HEAP O(1) O(1) O(1) O(1)

IS-EMPTY O(1) O(1) O(1) O(1)

INSERT O(1) O(log n) O(log n) O(1)

EXTRACT-MIN O(n) O(log n) O(log n) O(log n)

DECREASE-KEY O(1) O(log n) O(log n) O(1)

DELETE O(1) O(log n) O(log n) O(log n)

MELD O(1) O(n) O(log n) O(1)

FIND-MIN O(n) O(1) O(log n) O(1)

† amortized

PRIORITY QUEUES

‣ binary heaps

‣ d-ary heaps

‣ binomial heaps

‣ Fibonacci heaps

‣ advanced topics

・b-heaps.

・Fat heaps.

・2-3 heaps.

・Leaf heaps.

・Thin heaps.

・Skew heaps.

・Splay heaps.

・Weak heaps.

・Leftist heaps.

・Quake heaps.

・Pairing heaps.

・Violation heaps.

・Run-relaxed heaps.

・Rank-pairing heaps.

・Skew-pairing heaps.

・Rank-relaxed heaps.

・Lazy Fibonacci heaps.76

Heaps of heaps

Q. Can we achieve same running time as for Fibonacci heap but with

worst-case bounds per operation (instead of amortized)?

Theory. [Brodal 1996] Yes.

Practice. Ever implemented? Constants are high (and requires RAM model).

77

Brodal queues

Q. Can we achieve same running time as for Fibonacci heap but with

worst-case bounds per operation (instead of amortized) in pointer model?

Theory. [Brodal-Lagogiannis-Tarjan 2012] Yes.

78

Strict Fibonacci heaps

Strict Fibonacci Heaps

Gerth Stølting BrodalMADALGO!

Dept. of Computer ScienceAarhus University

Åbogade 34, 8200 Aarhus NDenmark

[email protected]

George LagogiannisAgricultural University

of AthensIera Odos 75, 11855 Athens

[email protected]

Robert E. Tarjan†

Dept. of Computer SciencePrinceton University

and HP Labs35 Olden Street, PrincetonNew Jersey 08540, [email protected]

ABSTRACTWe present the first pointer-based heap implementation withtime bounds matching those of Fibonacci heaps in the worstcase. We support make-heap, insert, find-min, meld anddecrease-key in worst-case O(1) time, and delete and delete-min in worst-case O(lg n) time, where n is the size of theheap. The data structure uses linear space.

A previous, very complicated, solution achieving the sametime bounds in the RAM model made essential use of arraysand extensive use of redundant counter schemes to maintainbalance. Our solution uses neither. Our key simplificationis to discard the structure of the smaller heap when doinga meld. We use the pigeonhole principle in place of theredundant counter mechanism.

Categories and Subject DescriptorsE.1 [Data Structures]: Trees; F.2.2 [Theory of Compu-tation]: Analysis of Algorithms and Problem Complexity—Nonnumerical Algorithms and Problems

General TermsAlgorithms

KeywordsData structures, heaps, meld, decrease-key, worst-case com-plexity

!Center for Massive Data Algorithmics, a Center of the Dan-ish National Research Foundation.†Partially supported by NSF grant CCF-0830676, US-IsraelBinational Science Foundation Grant 2006204, and theDistinguished Visitor Program of the Stanford UniversityComputer Science Department. The information containedherein does not necessarily reflect the opinion or policy ofthe federal government and no o!cial endorsement shouldbe inferred.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.STOC.12,May 19–22, 2012, New York, New York, USA.Copyright 2012 ACM 978-1-4503-1245-5/12/05 ...$10.00.

1. INTRODUCTIONWilliams in 1964 introduced binary heaps [25]. Since then

the design and analysis of heaps has been thoroughly inves-tigated. The most common operations supported by theheaps in the literature are those listed below. We assumethat each item stored contains an associated key. No itemcan be in more than one heap at a time.

makeheap() Create a new, empty heap and return a ref-erence to it.

insert(H, i) Insert item i, not currently in a heap, intoheap H , and return a reference to where i is storedin H .

meld(H1,H2) Return a reference to a new heap containingall items in the two heaps H1 and H2 (H1 and H2

cannot be accessed after meld).

find-min(H) Return a reference to where the item withminimum key is stored in the heap H .

delete-min(H) Delete the item with minimum key fromthe heap H .

delete(H,e) Delete an item from the heap H given a ref-erence e to where it is stored.

decrease-key(H,e, k) Decrease the key of the item givenby the reference e in heap H to the new key k.

There are many heap implementations in the literature,with a variety of characteristics. We can divide them intotwo main categories, depending on whether the time boundsare worst case or amortized. Most of the heaps in the lit-erature are based on heap-ordered trees, i.e. tree structureswhere the item stored in a node has a key not smaller thanthe key of the item stored in its parent. Heap-ordered treesgive heap implementations that achieve logarithmic time forall the operations. Early examples are the implicit binaryheaps of Williams [25], the leftist heaps of Crane [5] as modi-fied by Knuth [20], and the binomial heaps of Vuillemin [24].

The introduction of Fibonacci heaps [15] by Fredman andTarjan was a breakthrough since they achieved O(1) amor-tized time for all the operations above except for delete anddelete-min, which require O(lg n) amortized time, where nis the number of items in the heap and lg the base-two log-arithm. The drawback of Fibonacci heaps is that they arecomplicated compared to existing solutions and not as ef-ficient in practice as other, theoretically less e!cient solu-tions. Thus, Fibonacci heaps opened the way for further

Strict Fibonacci Heaps

Gerth Stølting BrodalMADALGO!

Dept. of Computer ScienceAarhus University

Åbogade 34, 8200 Aarhus NDenmark

[email protected]

George LagogiannisAgricultural University

of AthensIera Odos 75, 11855 Athens

[email protected]

Robert E. Tarjan†

Dept. of Computer SciencePrinceton University

and HP Labs35 Olden Street, PrincetonNew Jersey 08540, [email protected]

ABSTRACTWe present the first pointer-based heap implementation withtime bounds matching those of Fibonacci heaps in the worstcase. We support make-heap, insert, find-min, meld anddecrease-key in worst-case O(1) time, and delete and delete-min in worst-case O(lg n) time, where n is the size of theheap. The data structure uses linear space.

A previous, very complicated, solution achieving the sametime bounds in the RAM model made essential use of arraysand extensive use of redundant counter schemes to maintainbalance. Our solution uses neither. Our key simplificationis to discard the structure of the smaller heap when doinga meld. We use the pigeonhole principle in place of theredundant counter mechanism.

Categories and Subject DescriptorsE.1 [Data Structures]: Trees; F.2.2 [Theory of Compu-tation]: Analysis of Algorithms and Problem Complexity—Nonnumerical Algorithms and Problems

General TermsAlgorithms

KeywordsData structures, heaps, meld, decrease-key, worst-case com-plexity

!Center for Massive Data Algorithmics, a Center of the Dan-ish National Research Foundation.†Partially supported by NSF grant CCF-0830676, US-IsraelBinational Science Foundation Grant 2006204, and theDistinguished Visitor Program of the Stanford UniversityComputer Science Department. The information containedherein does not necessarily reflect the opinion or policy ofthe federal government and no o!cial endorsement shouldbe inferred.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.STOC.12,May 19–22, 2012, New York, New York, USA.Copyright 2012 ACM 978-1-4503-1245-5/12/05 ...$10.00.

1. INTRODUCTIONWilliams in 1964 introduced binary heaps [25]. Since then

the design and analysis of heaps has been thoroughly inves-tigated. The most common operations supported by theheaps in the literature are those listed below. We assumethat each item stored contains an associated key. No itemcan be in more than one heap at a time.

makeheap() Create a new, empty heap and return a ref-erence to it.

insert(H, i) Insert item i, not currently in a heap, intoheap H , and return a reference to where i is storedin H .

meld(H1,H2) Return a reference to a new heap containingall items in the two heaps H1 and H2 (H1 and H2

cannot be accessed after meld).

find-min(H) Return a reference to where the item withminimum key is stored in the heap H .

delete-min(H) Delete the item with minimum key fromthe heap H .

delete(H,e) Delete an item from the heap H given a ref-erence e to where it is stored.

decrease-key(H,e, k) Decrease the key of the item givenby the reference e in heap H to the new key k.

There are many heap implementations in the literature,with a variety of characteristics. We can divide them intotwo main categories, depending on whether the time boundsare worst case or amortized. Most of the heaps in the lit-erature are based on heap-ordered trees, i.e. tree structureswhere the item stored in a node has a key not smaller thanthe key of the item stored in its parent. Heap-ordered treesgive heap implementations that achieve logarithmic time forall the operations. Early examples are the implicit binaryheaps of Williams [25], the leftist heaps of Crane [5] as modi-fied by Knuth [20], and the binomial heaps of Vuillemin [24].

The introduction of Fibonacci heaps [15] by Fredman andTarjan was a breakthrough since they achieved O(1) amor-tized time for all the operations above except for delete anddelete-min, which require O(lg n) amortized time, where nis the number of items in the heap and lg the base-two log-arithm. The drawback of Fibonacci heaps is that they arecomplicated compared to existing solutions and not as ef-ficient in practice as other, theoretically less e!cient solu-tions. Thus, Fibonacci heaps opened the way for further

Q. Are Fibonacci heaps useful in practice?

A. They are part of LEDA and Boost C++ libraries.

(but other heaps seem to perform better in practice)

79

Fibonacci heaps: practice

80

Pairing heap. A self-adjusting heap-ordered general tree.

Theory. Same amortized running times as Fibonacci heaps for all operations

except DECREASE-KEY.

・O(log n) amortized. [Fredman et al. 1986]

・Ω(log log n) lower bound on amortized cost. [Fredman 1999]

・ amortized. [Pettie 2005]

Pairing heaps

2�

O(log log n)

81

Pairing heap. A self-adjusting heap-ordered general tree.

Practice. As fast as (or faster than) the binary heap on some problems.

Included in GNU C++ library and LEDA.

Pairing heaps

RESEARCH CONlRlWlIONS

Algorithms and Data Structures Pairing Heaps: G. Scott Graham Editor Experiments and Analysis

JOHN T. STASKO and JEFFREY SCOTT VlllER

ABSTRACT: The pairing heap has recently been introduced as a new data structure for priority queues. Pairing heaps are extremely simple to implement and seem to be very efficient in practice, but they are difficult to analyze theoretically, and open problems remain. It has been conjectured that they achieve the same amortized time bounds as Fibonacci heaps, namely, O(log n) time for delete and delete-min and O(1) for all other operations, where n is the size of the priority queue at the time of the operation. We provide empirical evidence that supports this conjecture. The most promising algorithm in our simulations is a new variant of the twopass method, called auxiliary twopass. We prove that, assuming no decrease-key operations are performed, it achieves the same amortized time bounds as Fibonacci heaps.

1. INTRODUCTION A priority queue is an abstract data type for main- taining and manipulating a set of items based on priority [I]. Prio’rity queues derive great theoretical

Support was provided in part by NSF research grant DCR-84-03613, an NSF Presidential Young Investigator Award, an IBM Faculty Development Award, and a Guggenheim Fellowship. Part of this research was performed at Mathematical Sciences Research Insti- tute. Berkeley, Calif., and the Institut National de Recherche en Informatique et en Automatique, Rocquencourt. France.

0 1987 ACM OOOl-0782/87/0300-0234 75a:

and practical importance from their use in solving a wide range of combinatorial problems, including job scheduling, minimal spanning tree, shortest path, and graph traversal.

Priority queues support the operations insert, find-min, and delete-min; additional operations often include decrease-key and delete. The insert(t, v) operation adds item t with key value v to the priority queue. The find-min operation returns the item with minimum key value. The delete-min operation returns the item with minimum key value and removes it from the priority queue. The decrease- key(t, d) operation reduces item t’s key value by d. The delete(t) operation removes item t from the priority queue. The decrease-key and delete operations require that a pointer to the location in the priority queue of item t be supplied explicitly, since priority queues do not support searching for arbitrary items by value. Some priority queues also support the merge operation, which combines two item- disjoint priority queues.

We will concentrate on the insert, delete-min, and decrease-key operations because they are the operations that primarily distinguish priority queues from other set manipulation algorithms and because they are the critical operations as far as the time bounds are concerned.

Communications of the ACM March 1987 Volume 30 Number 3

82


operation linked list binary heap binomial heap

pairing heap †

Fibonacci heap †

Brodal queue

MAKE-HEAP O(1) O(1) O(1) O(1) O(1) O(1)

IS-EMPTY O(1) O(1) O(1) O(1) O(1) O(1)

INSERT O(1) O(log n) O(log n) O(1) O(1) O(1)

EXTRACT-MIN O(n) O(log n) O(log n) O(log n) O(log n) O(log n)

DECREASE-KEY O(1) O(log n) O(log n) O(1) O(1)

DELETE O(1) O(log n) O(log n) O(log n) O(log n) O(log n)

MELD O(1) O(n) O(log n) O(1) O(1) O(1)

FIND-MIN O(n) O(1) O(log n) O(1) O(1) O(1)

† amortized

2�

O(log log n)

Assumption. Keys are integers between 0 and C.

Theorem. [Thorup 2004] There exists a priority queue that supports INSERT,

FIND-MIN, and DECREASE-KEY in constant time and EXTRACT-MIN and DELETE-KEY

in either O(log log n) or O(log log C) time.

83

Priority queues with integer priorities

http://www.elsevier.com/locate/jcssJournal of Computer and System Sciences 69 (2004) 330–353

Integer priority queues with decrease key in constant time andthe single source shortest paths problem

Mikkel Thorup

AT&T Labs Research, Shannon Laboratory, 180 Park Avenue, Florham Park, NJ 07932, USA

Received 22 July 2003; revised 8 March 2004

Available online 20 July 2004

Abstract

We consider Fibonacci heap style integer priority queues supporting find-min, insert, and decrease keyoperations in constant time. We present a deterministic linear space solution that with n integer keyssupports delete in O!log log n" time. If the integers are in the range #0;N"; we can also support delete inO!log logN" time.

Even for the special case of monotone priority queues, where the minimum has to be non-decreasing, the

best previous bounds on delete were O!!log n"1=!3$e"" and O!!logN"1=!4$e"": These previous bounds usedboth randomization and amortization. Our new bounds are deterministic, worst-case, with no restriction tomonotonicity, and exponentially faster.

As a classical application, for a directed graph with n nodes and m edges with non-negative integerweights, we get single source shortest paths in O!m% n log log n" time, or O!m% n log logC" if C is themaximal edge weight. The latter solves an open problem of Ahuja, Mehlhorn, Orlin, and Tarjan from 1990.r 2004 Elsevier Inc. All rights reserved.

Keywords: Integer priority queues; Decrease key; Fibonacci heaps; Single source shortest paths

1. Introduction

In 1984, Fredman and Tarjan introduced Fibonacci heaps [15], which is a priority queue over adynamic ordered set H supporting the following operations:

find-min!H" Returns an element from H with minimum key value in constant time.insert!H; a" Adds the element a to H in constant time.dec-key!H; a; x" Reduces the key value of element a to x in constant time. If the current key

value of a was smaller than x; it is an error.

ARTICLE IN PRESS

E-mail address: [email protected].

0022-0000/$ - see front matter r 2004 Elsevier Inc. All rights reserved.doi:10.1016/j.jcss.2004.04.003

Assumption. Keys are integers between 0 and C.

Theorem. [Thorup 2004] There exists a priority queue that supports INSERT,

FIND-MIN, and DECREASE-KEY in constant time and EXTRACT-MIN and DELETE-KEY

in either O(log log n) or O(log log C) time.

Corollary 1. Can implement Dijkstra's algorithm in either O(m log log n) or

O(m log log C) time.

Corollary 2. Can sort n integers in O(n log log n) time.

Computational model. Word RAM.

84

Priority queues with integer priorities

Goal. Break information-theoretic lower bound by allowing priority queue to

corrupt 10% of the keys (by increasing them).

85

Soft heaps

0

57

11

66

22

77

44

8888 11 22 99 44 33 77 66 0 55

soft heap

elements inserted 55

99

corrupted



Representation.

・Set of binomial trees (with some subtrees missing).

・Each node may store several elements.

・Each node stores a value that is an upper bound on the original keys.

・Binomial trees are heap-ordered with respect to these values.

86

Soft heaps



Theorem. [Chazelle 2000] Starting from an empty soft heap, any sequence

of n INSERT, MIN, EXTRACT-MIN, MELD, and DELETE operations takes O(n) time

and at most 10% of its elements are corrupted at any given time.

87

Soft heaps

The Soft Heap: An Approximate Priority Queue withOptimal Error Rate

BERNARD CHAZELLE

Princeton University, Princeton, New Jersey, and NEC Research Institute

Abstract. A simple variant of a priority queue, called a soft heap, is introduced. The data structuresupports the usual operations: insert, delete, meld, and findmin. Its novelty is to beat the logarithmicbound on the complexity of a heap in a comparison-based model. To break this information-theoreticbarrier, the entropy of the data structure is reduced by artificially raising the values of certain keys.Given any mixed sequence of n operations, a soft heap with error rate ! (for any 0 ! ! " 1/2) ensuresthat, at any time, at most !n of its items have their keys raised. The amortized complexity of eachoperation is constant, except for insert, which takes O(log 1/!) time. The soft heap is optimal for anyvalue of ! in a comparison-based model. The data structure is purely pointer-based. No arrays areused and no numeric assumptions are made on the keys. The main idea behind the soft heap is tomove items across the data structure not individually, as is customary, but in groups, in adata-structuring equivalent of “car pooling.” Keys must be raised as a result, in order to preserve theheap ordering of the data structure. The soft heap can be used to compute exact or approximatemedians and percentiles optimally. It is also useful for approximate sorting and for computingminimum spanning trees of general graphs.

Categories and Subject Descriptors: E.1 [Data Structures]: Nonnumerical Algorithms and Problems

General Terms: Theory

Additional Key Words and Phrases: Amortization, heap, priority queue, soft heap

1. Introduction

We design a simple variant of a priority queue, called a soft heap. The datastructure stores items with keys from a totally ordered universe, and supports theoperations:

A preliminary version of this paper as CHAZELLE, B. 1998. Car-pooling as a data structuring device:The soft heap. In Proceedings of the 6th Annual European Symposium on Algorithms, pp. 35– 42.This work was supported in part by National Science Foundation (NSF) Grants CCR 93-01254 andCCR 96-23768, ARO Grant DAAH04-96-1-0181, and NEC Research Institute.Author’s address: Department of Computer Science, Princeton University, 35 Olden Street, Prince-ton, NJ 08544-2087, e-mail: [email protected] or NEC Research Institute, e-mail: [email protected] to make digital / hard copy of part or all of this work for personal or classroom use isgranted without fee provided that the copies are not made or distributed for profit or commercialadvantage, the copyright notice, the title of the publication, and its date appear, and notice is giventhat copying is by permission of the Association for Computing Machinery (ACM), Inc. To copyotherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permissionand / or a fee.© 2000 ACM 0004-5411/00/1100-1012 $05.00

Journal of the ACM, Vol. 47, No. 6, November 2000, pp. 1012–1027.

88



Q. Brilliant. But how could it possibly be useful?

Ex. Linear-time deterministic selection. To find kth smallest element:

・Insert the n elements into soft heap.

・Extract the minimum element n / 2 times.

・The largest element deleted ≥ 4n / 10 elements and ≤ 6n / 10 elements.

・Can remove ≥ 4n / 10 of elements and recur.

・T(n) ≤ T(3n / 5) + O(n) ⇒ T(n) = O(n). ▪

Soft heaps

89

Theorem. [Chazelle 2000] There exists an O(m α(m, n)) time deterministic

algorithm to compute an MST in a graph with n nodes and m edges.

Algorithm. Borůvka + nongreedy + divide-and-conquer + soft heap + …

Soft heaps

A Minimum Spanning Tree Algorithm with Inverse-Ackermann Type Complexity

BERNARD CHAZELLE

Princeton University, Princeton, New Jersey, and NEC Research Institute

Abstract. A deterministic algorithm for computing a minimum spanning tree of a connected graph ispresented. Its running time is O(m!(m, n)), where ! is the classical functional inverse ofAckermann’s function and n (respectively, m) is the number of vertices (respectively, edges). Thealgorithm is comparison-based: it uses pointers, not arrays, and it makes no numeric assumptions onthe edge costs.

Categories and Subject Descriptors: F.2.2 [Analysis of Algorithms and Problem Complexity]:Nonnumerical Algorithms and Problems.

General Terms: Theory

Additional Key Words and Phrases: Graphs, matroids, minimum spanning trees

1. Introduction

The history of the minimum spanning tree (MST) problem is long and rich, goingas far back as Boruvka’s work in 1926 [Boruvka 1926; Graham and Hell 1985;Nesetril 1997]. In fact, MST is perhaps the oldest open problem in computerscience. According to Nesetril [1997], “this is a cornerstone problem of combina-torial optimization and in a sense its cradle.” Textbook algorithms run in O(mlog n) time, where n and m denote, respectively, the number of vertices andedges in the graph. Improvements to O(m log log n) were given independentlyby Yao [1975] and by Cheriton and Tarjan [1976]. In the mid-eighties, Fredmanand Tarjan [1987] lowered the complexity to O(m"(m, n)), where "(m, n) isthe number of log-iterations necessary to map n to a number less than m/n. Inthe worst case, m ! O(n) and the running time is O(m log* m). Soon after, the

A preliminary version of this paper appeared as CHAZELLE, B. 1997. A faster deterministic algorithmfor minimum spanning trees. In Proceedings of the 38th Annual IEEE Symposium on Foundations ofComputer Science. IEEE Computer Society Press, Los Alamitos, Calif., pp. 22–31.This work was supported in part by the National Science Foundation (NSF) Grants CCR 93-01254and CCR 96-23768, ARO Grant DAAH04-96-1-0181, and NEC Research Institute.Author’s address: Department of Computer Science, Princeton University, 35 Olden Street, Prince-ton, NJ 083-44-2087, e-mail: [email protected] and NEC Research Institute, e-mail:[email protected] to make digital / hard copy of part or all of this work for personal or classroom use isgranted without fee provided that the copies are not made or distributed for profit or commercialadvantage, the copyright notice, the title of the publication, and its date appear, and notice is giventhat copying is by permission of the Association for Computing Machinery (ACM), Inc. To copyotherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permissionand / or a fee.© 2000 ACM 0004-5411/00/1100-1028 $05.00

Journal of the ACM, Vol. 47, No. 6, November 2000, pp. 1028 –1047.

fibonacci heaps

Documents