binary heaps d-ary heaps binomial heaps fibonacci … › ~15750 › notes ›...

52
Lecture slides by Kevin Wayne Copyright © 2005 Pearson-Addison Wesley http://www.cs.princeton.edu/~wayne/kleinberg-tardos Last updated on 7/25/17 11:07 AM P RIORITY Q UEUES binary heaps d-ary heaps binomial heaps Fibonacci heaps

Upload: others

Post on 05-Jul-2020

13 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Lecture slides by Kevin WayneCopyright © 2005 Pearson-Addison Wesley

http://www.cs.princeton.edu/~wayne/kleinberg-tardos

Last updated on 7/25/17 11:07 AM

PRIORITY QUEUES

‣ binary heaps

‣ d-ary heaps

‣ binomial heaps

‣ Fibonacci heaps

Page 2: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Priority queue data type

A min-oriented priority queue supports the following core operations:

・MAKE-HEAP(): create an empty heap.

・INSERT(H, x): insert an element x into the heap.

・EXTRACT-MIN(H): remove and return an element with the smallest key.

・DECREASE-KEY(H, x, k): decrease the key of element x to k. The following operations are also useful:

・IS-EMPTY(H): is the heap empty?

・FIND-MIN(H): return an element with smallest key.

・DELETE(H, x): delete element x from the heap.

・MELD(H1, H2): replace heaps H1 and H2 with their union.

Note. Each element contains a key (duplicate keys are permitted) from a totally-ordered universe.

2

Page 3: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Priority queue applications

Applications.

・A* search.

・Heapsort.

・Online median.

・Huffman encoding.

・Prim’s MST algorithm.

・Discrete event-driven simulation.

・Network bandwidth management.

・Dijkstra’s shortest-paths algorithm.

・…

3http://younginc.site11.com/source/5895/fos0092.html

Page 4: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

ROBERT SEDGEWICK | KEVIN WAYNE

F O U R T H E D I T I O N

Algorithms

SECTION 2.4

PRIORITY QUEUES

‣ binary heaps

‣ d-ary heaps

‣ binomial heaps

‣ Fibonacci heaps

Page 5: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Complete binary tree

Binary tree. Empty or node with links to two disjoint binary trees.

Complete tree. Perfectly balanced, except for bottom level.

Property. Height of complete binary tree with n nodes is ⎣log2 n⎦. Pf. Height increases (by 1) only when n is a power of 2. ▪

5

complete tree with n = 16 nodes (height = 4)

Page 6: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

A complete binary tree in nature

6

Page 7: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binary heap

Binary heap. Heap-ordered complete binary tree.

Heap-ordered tree. For each child, the key in child ≥ key in parent.

7

8

18 11 2512

21 17 19

10

6

parent

child child

Page 8: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Explicit binary heap

Pointer representation. Each node has a pointer to parent and two children.

・Maintain number of elements n.

・Maintain pointer to root node.

・Can find pointer to last node or next node in O(log n) time.

8

8

18 11 2512

21 17 19

10

6

root

last next

Page 9: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Implicit binary heap

Array representation. Indices start at 1.

・Take nodes in level order.

・Parent of node at k is at ⎣k / 2⎦.

・Children of node at k are at 2k and 2k + 1.

9

8

18 11 2512

21 17 19

10

6

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

6 10 8 12 18 11 25 21 17 19

1

2 3

4 5

98 10

6 7

Page 10: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binary heap demo

10

heap ordered

8

18 11 2512

21 17 19

10

6

Page 11: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binary heap: insert

Insert. Add element in new node at end; repeatedly exchange new element

with element in its parent until heap order is restored.

11

8

18 11 2512

21 17 19

10

6

7

8

10 11 2512

21 17 19

7

6

18

add key to heap (violates heap order)

swim up

Page 12: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

exchange with root

element to remove

Binary heap: extract the minimum

Extract min. Exchange element in root node with last node; repeatedly

exchange element in root with its smaller child until heap order is restored.

12

8

10 11 2512

21 17 19

7

6

18

8

10 11 2512

21 17 19

7

18

6 remove from heap

violates heap order

8

18 11 2512

21 17 19

10

7

6

sink down

Page 13: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binary heap: decrease key

Decrease key. Given a handle to node, repeatedly exchange element with

its parent until heap order is restored.

13

8

18 11 2512

21 17 19

10

6

decrease key of node x to 11

x

Page 14: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binary heap: analysis

Theorem. In an implicit binary heap, any sequence of m INSERT, EXTRACT-MIN,

and DECREASE-KEY operations with n INSERT operations takes O(m log n) time.

Pf.

・Each heap op touches nodes only on a path from the root to a leaf; the height of the tree is at most log2 n.

・The total cost of expanding and contracting the arrays is O(n). ▪ Theorem. In an explicit binary heap with n nodes, the operations INSERT,

DECREASE-KEY, and EXTRACT-MIN take O(log n) time in the worst case.

14

Page 15: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binary heap: find-min

Find the minimum. Return element in the root node.

15

10

8 11 2512

21 17 9

7

6

root

Page 16: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binary heap: delete

Delete. Given a handle to a node, exchange element in node with last node;

either swim down or sink up the node until heap order is restored.

16

10

8 11 2512

21 17 9

7

6

delete node x or y

x

last

y

Page 17: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binary heap: meld

Meld. Given two binary heaps H1 and H2, merge into a single binary heap.

Observation. No easy solution: Ω(n) time apparently required.

17

10

8 11 2512

21 17 9

7

H1 H2

Page 18: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binary heap: heapify

Heapify. Given n elements, construct a binary heap containing them.

Observation. Can do in O(n log n) time by inserting each element.

Bottom-up method. For i = n to 1, repeatedly exchange the element in node i with its smaller child until subtree rooted at i is heap-ordered.

18

8 12 9 7 22 3 26 14 11 15 22

1 2 3 4 5 6 7 8 9 10 11

5

10 11

9

22 3 267

14 11 15 22

12

8

8 9

4 76

32

1

Page 19: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binary heap: heapify

Theorem. Given n elements, can construct a binary heap containing those n

elements in O(n) time.

Pf.

・There are at most ⎡n / 2h+1⎤ nodes of height h.

・The amount of work to sink a node is proportional to its height h.

・Thus, the total work is bounded by:

Corollary. Given two binary heaps H1 and H2 containing n elements in total,

can implement MELD in O(n) time.

�log2 n��

h=0

�n / 2h+1� h ��log2 n��

h=0

nh / 2h

� n��

h=1

h / 2h

= 2n

19

�log2 n��

h=0

�n / 2h+1� h ��log2 n��

h=0

nh / 2h

� n��

h=1

h / 2h

= 2n

�log2 n��

h=0

�n / 2h+1� h ��log2 n��

h=0

nh / 2h

� n��

h=1

h / 2h

= 2n

�log2 n��

h=0

�n / 2h+1� h ��log2 n��

h=0

nh / 2h

� n��

h=1

h / 2h

= 2n

k�

i=1

i

2i= 2 � k

2k� 1

2k�1

� 2

Page 20: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Priority queues performance cost summary

20

operation linked list binary heap

MAKE-HEAP O(1) O(1)

ISEMPTY O(1) O(1)

INSERT O(1) O(log n)

EXTRACT-MIN O(n) O(log n)

DECREASE-KEY O(1) O(log n)

DELETE O(1) O(log n)

MELD O(1) O(n)

FIND-MIN O(n) O(1)

Page 21: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Priority queues performance cost summary

Q. Reanalyze so that EXTRACT-MIN and DELETE take O(1) amortized time?

21

operation linked list binary heap binary heap †

MAKE-HEAP O(1) O(1) O(1)

ISEMPTY O(1) O(1) O(1)

INSERT O(1) O(log n) O(log n)

EXTRACT-MIN O(n) O(log n) O(1) †

DECREASE-KEY O(1) O(log n) O(log n)

DELETE O(1) O(log n) O(1) †

MELD O(1) O(n) O(n)

FIND-MIN O(n) O(1) O(1)

† amortized

Page 22: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

CHAPTER 6 (2ND EDITION)

PRIORITY QUEUES

‣ binary heaps

‣ d-ary heaps

‣ binomial heaps

‣ Fibonacci heaps

Page 23: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Priority queues performance cost summary

Goal. O(log n) INSERT, DECREASE-KEY, EXTRACT-MIN, and MELD.

30

operation linked list binary heap d-ary heap

MAKE-HEAP O(1) O(1) O(1)

ISEMPTY O(1) O(1) O(1)

INSERT O(1) O(log n) O(logd n)

EXTRACT-MIN O(n) O(log n) O(d logd n)

DECREASE-KEY O(1) O(log n) O(logd n)

DELETE O(1) O(log n) O(d logd n)

MELD O(1) O(n) O(n)

FIND-MIN O(n) O(1) O(1)

mergeable heap

Page 24: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heaps

31

Programming Techniques

S.L. Graham, R.L. Rivest Editors

A Data Structure for Manipulating Priority Queues J e a n V u i l l e m i n U n i v e r s i t 6 d e P a r i s - S u d

A data structure is described which can be used for representing a collection of priority queues. The primitive operations are insertion, deletion, union, update, and search for an item of earliest priority.

Key Words and Phrases: data structures, implementation of set operations, priority queues, mergeable heaps, binary trees

CR Categories: 4.34, 5.24, 5.25, 5.32, 8.1

I. Introduction

In order to design correct and efficient algorithms for solving a specific problem, it is often helpful to describe our first approach to a solution in a language close to that in which the problem is formulated. One such language is that of set theory, augmented by primitive set manipulation operations. Once the algorithm is out- lined in terms of these set operations, one can then look for data structures most suitable for representing each of the sets involved. This choice depends only upon the collection of primitive operations required for each set. It is thus important to establish a good catalogue of such data structures. A summary of the state of the art on this question can be found in [2]. In this paper, we add to this catalogue a data structure which allows efficient manipulation of priority queues.

General permission to make fair use in teaching or research of all or part of this material is granted to individual readers and to nonprofit libraries acting for them provided that ACM's copyright notice is given and that reference is made to the publication, to its date of issue, and to the fact that reprinting privileges were granted by permission of the Association for Computing Machinery. To otherwise reprint a figure, table, other substantial excerpt, or the entire work requires specific permission as does republication, or systematic or multiple reproduc- tion.

Author's address: Laboratoire de Recherche en Informatique, Brit. 490, Universit6 de Paris-Sud, Centre d'Orsay 91405--Orsay, France. © 1978 ACM 0001-0782/78/0400-0309 $00.75

309

A priority queue is a set; each element of such a set has a name, which is used to uniquely identify the element, and a label or priority drawn from a totally ordered set. Elements of the priority queue can be thought of as awaiting service, where the item with the ~ smallest label is always to be served next. Ordinary stacks and queues are special cases of priority queues.

A variety of applications directly require using prior- ity queues: job scheduling, discrete simulation languages where labels represent the time at which events are to occur, as well as various sorting problems. These are discussed, for example, in [2, 3, 5, 11, 15, 17, 24]. Priority queues also play a central role in several good algorithms, such as optimal code constructions, Chartre's prime number generator, and Brown's power series multipli- cation (see [16] and [17]); applications have also been found in numerical analysis algorithms [10, 17, 19] and in graph algorithms for such problems as finding shortest paths [2, 13] and minimum cost spanning tree [2, 4, 25].

Typical applications require primitive operations among the following five: INSERT, DELETE, MIN, UP- DATE, and UNION. The operation INSERT (name, label, Q) adds an element to queue Q, while DELETE (name) removes the element having that name. Operation MIN (Q) returns the name of the element in Q having the least label, and UPDATE (name, label) changes the label of the element named. Finally, UNION (Q1, Q2, Q3) merges into Qa all elements of Q1 and Q2; the sets Q1 and Q2 become empty. In what follows, we assume that names are handled in a separate dictionary [2, 17] such as a hash- table or a balanced tree. If deletions are restricted to elements extracted by MIN, such an auxiliary symbol table is not needed. 1

The heap, a truly elegant data structure discovered by J. W. Williams and R. W. Floyd, handles a sequence of n primitives INSERT, DELETE, and MIN, and runs in O(nlogn) elementary operations using absolutely mini- mal storage [17]. For applications in which UNION is necessary, more sophisticated data structures have been devised, such as 2-3 trees [2, 17], leftist trees [5, 17], and binary heaps [9].

The data structure we present here handles an arbi- trary sequence of n primitives, each drawn from the five described above, in O(nlogn) machine operations and O(n) memory cells. It also allows for an efficient treat- ment of a large number of updates, which is crucial in connection with spanning tree algorithms: Our data structure provides an implementation (described in [25]) of the Cheriton-Tarjan-Yao [3] minimum cost span- ning tree algorithm which is much more straightforward than the original one.

The proposed data structure uses less storage than leftist, AVL, or 2-3 trees; in addition, when the primitive operations are carefully machine coded from the pro- grams given in Section 4, they yield worst case running times which compare favorably with those of their corn-

We-g'gsume here that indexing through the symbol table is done in constant time.

Communications April 1978 of Volume 21 the ACM Number 4

Page 25: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial tree

Def. A binomial tree of order k is defined recursively:

・Order 0: single node.

・Order k: one binomial tree of order k – 1 linked to another of order k – 1.

32

B0 B1 B2 B3 B4

Bk-1

Bk-1

BkB0

Page 26: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial tree properties

Properties. Given an order k binomial tree Bk,

・Its height is k.

・It has 2k nodes.

・It has nodes at depth i.

・The degree of its root is k.

・Deleting its root yields k binomial trees Bk–1, …, B0.

Pf. [by induction on k]

33

B4

B1

Bk

Bk+1

B2

B0

!ki

"

Page 27: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heap

Def. A binomial heap is a sequence of binomial trees such that:

・Each tree is heap-ordered.

・There is either 0 or 1 binomial tree of order k.

34

55

45 32

30

24

23 22

50

48 31 17

448 29 10

6

37

3 18

B4 B1 B0

Page 28: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heap representation

Binomial trees. Represent trees using left-child, right-sibling pointers.

Roots of trees. Connect with singly-linked list, with degrees decreasing

from left to right.

35

50

48 31 17

4410

6

37

3 18

29

6

37

3 18

48

3150

10

4417

29

leftist power-of-2 heap representationbinomial heap

paren

t

left

right

root

Page 29: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heap properties

Properties. Given a binomial heap with n nodes:

・The node containing the min element is a root of B0, B1, …, or Bk.

・It contains the binomial tree Bi iff bi = 1, where bk⋅ b2 b1 b0 is binary

representation of n.

・It has ≤ ⎣log2 n⎦ + 1 binomial trees.

・Its height ≤ ⎣log2 n⎦.

36

55

45 32

30

24

23 22

50

48 31 17

448 29 10

6

37

3 18

B4 B1 B0

n = 19# trees = 3height = 4

binary = 10011

Page 30: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heap: meld

Meld operation. Given two binomial heaps H1 and H2, (destructively) replace with a binomial heap H that is the union of the two.

Warmup. Easy if H1 and H2 are both binomial trees of order k.

・Connect roots of H1 and H2.

・Choose node with smaller key to be root of H.

37

55

45 32

30

24

23 22

50

48 31 17

448 29 10

6

H1 H2

Page 31: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

55

45 32

30

24

23 22

50

48 31 17

448 29 10

6

37

3 18

41

3328

15

25

7 12

+

Page 32: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

55

45 32

30

24

23 22

50

48 31 17

448 29 10

6

37

3

41

3328

15

25

7

12

18

18

12

+

Page 33: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

55

45 32

30

24

23 22

50

48 31 17

448 29 10

6

37

3

41

3328

15

25

7

12

18

25

377

3

18

12

18

12

+

Page 34: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

55

45 32

30

24

23 22

50

48 31 17

448 29 10

6

37

3

41

3328

15

25

7

12

18

25

377

3

41

28 33 25

3715 7

3

18

12

18

12

+

Page 35: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

28

55

45 32

30

24

23 22

50

48 31 17

448 29 10

6

37

3

41

3328

15

25

7

18

12

41

33 25

3715 7

3

12

18

25

377

3

41

28 33 25

3715 7

3

18

12

+

Page 36: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

55

45 32

30

24

23 22

50

48 31 17

448 29 10

6

37

3

41

3328

15

25

7

18

12

41

28 33 25

3715 7

3

3

55

45 32

30

24

23 22

50

48 31 17

448 29 10

18

12

12

18

25

377

41

28 33 25

3715 7

3

6

+

Page 37: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

19 + 7 = 26

55

45 32

30

24

23 22

50

48 31 17

448 29 10

6

37

3 18

41

3328

15

25

7 12

+

1 1 1

1 0 0 1 1

+ 0 0 1 1 1

1 1 0 1 0

Page 38: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heap: meld

Meld operation. Given two binomial heaps H1 and H2, (destructively) replace with a binomial heap H that is the union of the two.

Solution. Analogous to binary addition.

Running time. O(log n). Pf. Proportional to number of trees in root lists ≤ 2 ( ⎣log2 n⎦ + 1). ▪

45

1 1 1

1 0 0 1 1

+ 0 0 1 1 1

1 1 0 1 0

19 + 7 = 26

Page 39: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heap: extract the minimum

Extract-min. Delete the node with minimum key in binomial heap H.

・Find root x with min key in root list of H, and delete.

46

3

37

6 18

55

45 32

30

24

23 22

50

48 31 17

448 29 10

H

Page 40: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heap: extract the minimum

Extract-min. Delete the node with minimum key in binomial heap H.

・Find root x with min key in root list of H, and delete.

・Hʹ ← broken binomial trees.

・H ← MELD(Hʹ, H). Running time. O(log n).

47

37

6 18

55

45 32

30

24

23 22

50

48 31 17

448 29 10

H

H′

Page 41: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heap: decrease key

Decrease key. Given a handle to an element x in H, decrease its key to k.

・Suppose x is in binomial tree Bk.

・Repeatedly exchange x with its parent until heap order is restored.

Running time. O(log n).

48

3

37

6 18

55

x 32

30

24

23 22

50

48 31 17

448 29 10

H

Page 42: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heap: delete

Delete. Given a handle to an element x in a binomial heap, delete it.

・DECREASE-KEY(H, x, -∞).

・DELETE-MIN(H).

Running time. O(log n).

49

3

37

6 18

55

32

30

24

23 22

50

48 31 17

448 29 10

H

45

Page 43: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heap: insert

Insert. Given a binomial heap H, insert an element x.

・Hʹ ← MAKE-HEAP( ).

・Hʹ ← INSERT(Hʹ, x).

・H ← MELD(Hʹ, H). Running time. O(log n).

50

3

37

6 18

55

45 32

30

24

23 22

50

48 31 17

448 29 10

H

x

H′

Page 44: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heap: sequence of insertions

Insert. How much work to insert a new node x ?

・If n = .......0, then only 1 credit.

・If n = .......01, then only 2 credits.

・If n = .......011, then only 3 credits.

・If n = .......0111, then only 4 credits.

Observation. Inserting one element can take Ω(log n) time.

Theorem. Starting from an empty binomial heap, a sequence of n

consecutive INSERT operations takes O(n) time.

Pf. (n / 2) (1) + (n / 4)(2) + (n / 8)(3) + … ≤ 2 n. ▪

51

50

48 31 17

4429 10

3

37

6 x

if n = 11...111

k�

i=1

i

2i= 2 � k

2k� 1

2k�1

� 2

Page 45: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heap: amortized analysis

Theorem. In a binomial heap, the amortized cost of INSERT is O(1) and the

worst-case cost of EXTRACT-MIN and DECREASE-KEY is O(log n). Pf. Define potential function Φ(Hi) = trees(Hi) = # trees in binomial heap Hi.

・Φ(H0) = 0.

・Φ(Hi) ≥ 0 for each binomial heap Hi.

Case 1. [INSERT]

・Actual cost ci = number of trees merged + 1.

・∆Φ = Φ(Hi) – Φ(Hi–1) = 1 – number of trees merged.

・Amortized cost = ĉi = ci + Φ(Hi) – Φ(Hi–1) = 2.

52

Page 46: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heap: amortized analysis

Theorem. In a binomial heap, the amortized cost of INSERT is O(1) and the

worst-case cost of EXTRACT-MIN and DECREASE-KEY is O(log n). Pf. Define potential function Φ(Hi) = trees(Hi) = # trees in binomial heap Hi.

・Φ(H0) = 0.

・Φ(Hi) ≥ 0 for each binomial heap Hi.

Case 2. [ DECREASE-KEY ]

・Actual cost ci = O(log n).

・∆Φ = Φ(Hi) – Φ(Hi–1) = 0.

・Amortized cost = ĉi = ci = O(log n).

53

Page 47: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Binomial heap: amortized analysis

Theorem. In a binomial heap, the amortized cost of INSERT is O(1) and the

worst-case cost of EXTRACT-MIN and DECREASE-KEY is O(log n). Pf. Define potential function Φ(Hi) = trees(Hi) = # trees in binomial heap Hi.

・Φ(H0) = 0.

・Φ(Hi) ≥ 0 for each binomial heap Hi.

Case 3. [ EXTRACT-MIN or DELETE ]

・Actual cost ci = O(log n).

・∆Φ = Φ(Hi) – Φ(Hi–1) ≤ Φ(Hi) ≤ ⎣log2 n⎦.

・Amortized cost = ĉi = ci + Φ(Hi) – Φ(Hi–1) = O(log n). ▪

54

Page 48: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Priority queues performance cost summary

Hopeless challenge. O(1) INSERT, DECREASE-KEY and EXTRACT-MIN. Why?

Challenge. O(1) INSERT and DECREASE-KEY, O(log n) EXTRACT-MIN.

55

operation linked list binary heap binomial heap binomial heap

MAKE-HEAP O(1) O(1) O(1) O(1)

ISEMPTY O(1) O(1) O(1) O(1)

INSERT O(1) O(log n) O(log n) O(1) †

EXTRACT-MIN O(n) O(log n) O(log n) O(log n)

DECREASE-KEY O(1) O(log n) O(log n) O(log n)

DELETE O(1) O(log n) O(log n) O(log n)

MELD O(1) O(n) O(log n) O(1) †

FIND-MIN O(n) O(1) O(log n) O(1)

† amortized

homework

Page 49: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Lecture slides by Kevin Waynehttp://www.cs.princeton.edu/~wayne/kleinberg-tardos

Last updated on 7/25/17 11:07 AM

FIBONACCI HEAPS

‣ preliminaries

‣ insert

‣ extract the minimum

‣ decrease key

‣ bounding the rank

‣ meld and delete

Page 50: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Priority queues performance cost summary

Ahead. O(1) INSERT and DECREASE-KEY, O(log n) EXTRACT-MIN.

2

operation linked list binary heap binomial heapFibonacci heap

MAKE-HEAP O(1) O(1) O(1) O(1)

IS-EMPTY O(1) O(1) O(1) O(1)

INSERT O(1) O(log n) O(log n) O(1)

EXTRACT-MIN O(n) O(log n) O(log n) O(log n)

DECREASE-KEY O(1) O(log n) O(log n) O(1)

DELETE O(1) O(log n) O(log n) O(log n)

MELD O(1) O(n) O(log n) O(1)

FIND-MIN O(n) O(1) O(log n) O(1)

† amortized

Page 51: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Fibonacci heaps

Theorem. [Fredman-Tarjan 1986] Starting from an empty Fibonacci heap, any sequence of m INSERT, EXTRACT-MIN, and DECREASE-KEY operations involving n INSERT operations takes O(m + n log n) time.

3

Fibonacci Heaps and Their Uses in Improved Network Optimization Algorithms

MICHAEL L. FREDMAN

University of California, San Diego, L.a Jolla, California

AND

ROBERT ENDRE TARJAN

AT&T Bell Laboratories, Murray HilI, New Jersey

Abstract. In this paper we develop a new data structure for implementing heaps (priority queues). Our structure, Fibonacci heaps (abbreviated F-heaps), extends the binomial queues proposed by Vuillemin and studied further by Brown. F-heaps support arbitrary deletion from an n-item heap in qlogn) amortized time and all other standard heap operations in o( 1) amortized time. Using F-heaps we are able to obtain improved running times for several network optimization algorithms. In particular, we obtain the following worst-case bounds, where n is the number of vertices and m the number of edges in the problem graph:

( 1) O(n log n + m) for the single-source shortest path problem with nonnegative edge lengths, improved from O(m logfmh+2)n);

(2) O(n*log n + nm) for the all-pairs shortest path problem, improved from O(nm lo&,,,+2,n); (3) O(n*logn + nm) for the assignment problem (weighted bipartite matching), improved from

O(nm log0dn+2)n); (4) O(mj3(m, n)) for the minimum spanning tree problem, improved from O(mloglo&,,.+2,n), where

j3(m, n) = min {i 1 log% 5 m/n). Note that B(m, n) 5 log*n if m 2 n.

Of these results, the improved bound for minimum spanning trees is the most striking, although all the results give asymptotic improvements for graphs of appropriate densities.

Categories and Subject Descriptors: E.l [Data]: Data Structures--trees; graphs; F.2.2 [Analysis of Algorithms and Problem Complexity]: Nonnumerical Algorithms and Problems-computations on discrete structures; sorting and searching; G.2.2 [Discrete Mathematics]: Graph Theory-graph algo- rithms; network problems; trees General Terms: Algorithms, Theory Additional Key Words and Phrases: Heap, matching, minimum spanning tree, priority queue, shortest Pati

’ A preliminary version of this paper appeared in the Proceedings of the 25th IEEE Symposium on the Foundations of Computer Science (Singer Island, Fla., Oct. 24-26). IEEE, New York, 1984, pp. 338- 346, 0 IEEE. Any portion of this paper that appeared in the preliminary version is reprinted with permission. This research was supported in part by the National Science Foundation under grant MCS 82-0403 1. Authors’ addresses: M. L. Fredman, Electrical Engineering and Computer Science Department, Uni- versity of California, San Diego, La Jolla, CA 92093; R. E. Tarjan, AT&T Bell Laboratories, Murray Hill, NJ 07974. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. 0 1987 ACM 0004-541 l/87/0700-0596 $1.50

Journal ofthe Association for Computing Machinery, Vol. 34, No. 3, July 1987, Pages 596-615.

this statement is a bit weaker than the actual theorem

Page 52: binary heaps d-ary heaps binomial heaps Fibonacci … › ~15750 › notes › binomial-heaps-slides.pdfBinary heap: heapify Theorem. Given n elements, can construct a binary heap

Fibonacci heaps

Theorem. [Fredman–Tarjan 1986] Starting from an empty Fibonacci heap,

any sequence of m INSERT, EXTRACT-MIN, and DECREASE-KEY operations involving n INSERT operations takes O(m + n log n) time.

History.

・Ingenious data structure and application of amortized analysis.

・Original motivation: improve Dijkstra’s shortest path algorithmfrom O(m log n) to O(m + n log n).

・Also improved best-known bounds for all-pairs shortest paths,

assignment problem, minimum spanning trees.

4