Transcript
Page 1: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Thinking Points

How do we find (and remove) the smallest element in a ordered binarytree? What is the complexity?

Consider a modified linked list structure where node had an extra ‘nextnext’ pointer to the node two steps ahead. How would we search in thislist? What would the complexity be?

Did we understand tree rotations?

Giles Reger Lecture 10 December 2019 1 / 20

Page 2: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Lecture 10Advanced Data Structures Part 2

Heaps and Skip Lists

COMP26120

Giles Reger

December 2019

Giles Reger Lecture 10 December 2019 2 / 20

Page 3: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Last Time

Representing trees with array or linked structure

Lots of properties of trees

Breadth-first vs Depth-first

Binary search trees

Self-balancing AVL trees

Giles Reger Lecture 10 December 2019 3 / 20

Page 4: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Priority Queue ADT

Remember this...

A priority queue P supports the following fundamental operations:

insert(k,e): Insert an element e with key k into P

removeMin(): Remove and return from P the element with the smallestkey (may not be unique)

(error if P is empty)

Usually also support size() and isEmpty() operations.

How can we efficiently implement removeMin()?

Giles Reger Lecture 10 December 2019 4 / 20

Page 5: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

(Binary) Heap

A (binary) heap is a complete (binary) tree with the heap-order property.

Definition (Heap-Order Property)

In a heap T , for every node v other than the root, the key stored at v isgreater or equal to the key stored at v ’s parent.

Consequently, keys on any path from the root to an external node are innon-decreasing order.

Note that a complete tree always has a last node, defined as theright-most deepest node. Also, note that the height of a complete tree isalways dlog(n + 1)e, so if operations are O(h) they are logarithmic in n.

Finally, because a heap is complete, it is efficient to use a vectorrepresentation.

Giles Reger Lecture 10 December 2019 5 / 20

Page 6: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Max-Heap vs Min-Heap

By default we assume we are defining a Min-Heap e.g. things got biggeras we go down the heap and we want to remove the minimum element.

The alternative is a Max-Heap. This straightforwardly reverses the ordereverywhere. Without loss of generality we just talk about Min-Heap but itis important to understand the difference.

Giles Reger Lecture 10 December 2019 6 / 20

Page 7: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Implementing insert

We need to preserve the heap-order and completeness properties.

To preserve completeness, we add the node as the last node. In a vectorrepresentation this will be the index immediately after the current lastnode (think about it). This is O(1).

To preserve the heap-order property we then need to ‘bubble’ the newvalue up the heap until it is preserved. The general idea is that werecursively swap parent and child until the parent is smaller than the childagain. This process is called bubble-up or sift-up.

This second step is O(h), which we previously saw was O(log n).

Why do we only compare child and parent and not siblings?

Giles Reger Lecture 10 December 2019 7 / 20

Page 8: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Implementing removeMin()

Identifying the minimum element is now straightforward - it is always theroot of the heap.

We now need to remove it whilst maintaining the heap-order andcompleteness properties.

We first replace the root with the last element. This preservescompleteness.

But this will violate the heap-order property so we need to bubble-down orsift-down. To do this we compare a node to its children and if it is largerthan both we terminate, otherwise we pick one, swap the parent and childand then repeat.

What is the complexity? Does it matter which child we pick?

Giles Reger Lecture 10 December 2019 8 / 20

Page 9: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Examples

Examples

Giles Reger Lecture 10 December 2019 9 / 20

Page 10: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Other Heap Thoughts

How can we can implement other container-like operations such as find

and remove?

What about melding two heaps together?

What would change if we used a linked structure?

Other well-known heap variants include Binomial Heap and FibonacciHeap. These are a bit more complex but give better guarantees for someoperations.

Giles Reger Lecture 10 December 2019 10 / 20

Page 11: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Better Sorted Linked Lists

We cannot perform binary search in a sorted linked list but what if we hadadditional structure?

If we had a pointer into the middle of the list,and then half way between each of those,and then half way between each of those,. . .

This is what a Skip List achieves

However, it is also a probabilistic data structure

Giles Reger Lecture 10 December 2019 11 / 20

Page 12: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Skip List

19.6. Skip Lists 557

19.6 Skip Lists

An interesting data structure for efficiently realizing the ordered set of items isthe skip list. This data structure makes random choices in arranging the itemsin such a way that search and update times are logarithmic on average. A skiplist S for an ordered dictionary, D, of key-value pairs consists of a series of lists{S0, S1, . . . , Sh}. Each list Si stores a subset of the items of D sorted by a nonde-creasing key plus items with two special keys, denoted −∞ and +∞, where −∞is smaller than every possible key that can be inserted in D and +∞ is larger thanevery possible key that can be inserted in D. In addition, the lists in S satisfy thefollowing (see Figure 19.13):

• S0 contains every item in D (plus special items with keys −∞ and +∞).

• For i = 1, . . . , h−1, list Si contains (in addition to −∞ and +∞) a randomlygenerated subset of the items in list Si−1.

• Sh contains only −∞ and +∞.

It is customary to visualize a skip list S with list S0 at the bottom and lists S1, . . . , Sh

above it. Also, we refer to h as the height of skip list S.Intuitively, the lists are set up so that Si+1 contains essentially every other item

in Si. The items in Si+1 are chosen at random from the items in Si by picking eachitem from Si to also be in Si+1 with probability 1/2. That is, in essence, we “flip acoin” for each item in Si and place that item in Si+1 if the coin comes up “heads.”Thus, we expect S1 to have about n/2 items, S2 to have about n/4 items, and, ingeneral, Si to have about n/2i items. In other words, we expect the height h ofS to be about log n. We view a skip list as a two-dimensional collection of nodesarranged horizontally into levels and vertically into towers. Each level is a list Si

and each tower contains nodes storing the same item across consecutive lists.

17 31 38

555012 17 20 25 31 38 39 44

4412

17 5531

17

17 55

55

25

25

25

S5

S4

S3

S2

S1

S0 8-

8-

8-

8-

8-

8- 8+

8+

8+

8+

8+

8+

Figure 19.13: Example of a skip list.

Goodrich, Michael T., and Roberto Tamassia. <i>Algorithm Design and Applications</i>, Wiley, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/manchester/detail.action?docID=5182215.Created from manchester on 2019-12-03 14:59:57.

Cop

yrig

ht ©

201

4. W

iley.

All

right

s re

serv

ed.

Nodes are arranged horizontally into levels and vertically into towers

Giles Reger Lecture 10 December 2019 12 / 20

Page 13: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Structure

A node contains a key-value pair and pointers to surrounding nodes.

Conceptually, each node p supports the operations

after(p) - return the node following p on the same level

before(p) - return the node preceding p on the same level

above(p) - return the node above p in the same tower

below(p) - return the node below p in the same tower

Nodes above/below have the same key-value pair. Keys arenon-decreasing across the level (it is sorted).

In reality, we are more likely to represent towers as a separate datastructure with a height and an array of next nodes.

Giles Reger Lecture 10 December 2019 13 / 20

Page 14: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Search for k

Effectively zig-zag down and right through the structure moving sidewayswhen the next key is bigger than k and down otherwise

Start at ‘top left’ node p

While below(p) is not nullp = below(p)While key(after(p)) ≤ k

p = after(p)

Return p

Finds the bottom-level node with the largest key less-than-or-equal-to k

For contains can just check if key(p) = k

To ensure that the ‘top left’ node and after(p) always exist we alwayshave two sentinels with keys −∞ and ∞.

Giles Reger Lecture 10 December 2019 14 / 20

Page 15: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Insertion and Removal

To insert (k , v) we need to create a new node after search(k) at thebottom level and then decide how many additional levels to include thenode in e.g. the height of the tower.

We do this randomly. Usually with probability 12 , e.g. flipping a coin, for

each extra level.

We typically bound the maximum number of levels (e.g. to 3dlog ne).What is the probability we go to a level higher than log n?

Updating the actual nodes is synonymous to the linked list case.

For removal we simply remove the node p = search(k) if key(p) = kusing the standard operations. Note that we never remove the sentinels.

Giles Reger Lecture 10 December 2019 15 / 20

Page 16: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Worst Case Complexity

As all operations are based on the search method (plus some local work)we just need to look at this.

The worst case is when every tower is of height h − 1. This collapses to alinked list where we also need to traverse down the last tower e.g.O(n + h).

This is very unlikely.

Giles Reger Lecture 10 December 2019 16 / 20

Page 17: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Average Complexity

Firstly, the probability that the height h is greater than 3log n is

n

23log n=

n

n3=

1

n2

e.g. n chances to get 3log n ‘heads’.

We have high probability that h is O(log n).

The number of ‘down’ steps in the search method is h

What about scan-forward? We never return to a level. If we are using 12 to

decide whether to add a level then each key appears in 2 levels on average(expected number of times to flip a coin and get a sequence of ‘heads’).

Therefore, work per level is O(1) and overall complexity isO(h) = O(log n)

Giles Reger Lecture 10 December 2019 17 / 20

Page 18: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Space Complexity

The expected number of items at level i is n2i

meaning that the totalnumber of items stored is

h∑i=0

n

2i= n

h∑i=0

1

2i< 2n

Therefore, a skip list containing n items has a O(n) space requirement.

Giles Reger Lecture 10 December 2019 18 / 20

Page 19: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Examples

Examples

Giles Reger Lecture 10 December 2019 19 / 20

Page 20: Lecture 10 Advanced Data Structures Part 2 Heaps and Skip ...syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/... · 19.6. Skip Lists 557 19.6 Skip Lists An interesting data structure

Summary

Heap

Heap-order property

Complete binary tree (use vector representation)

Bubble-up and bubble-down

Skip List

Probabilistic data structure

Levels of linked lists where each list contains roughly half theelements of the previous level

Search by ‘zig-zagging’ down

Giles Reger Lecture 10 December 2019 20 / 20


Top Related