14 skip lists

211
Analysis of Algorithms Skip Lists Andres Mendez-Vazquez October 18, 2015 1 / 99

Upload: andres-mendez-vazquez

Post on 19-Jul-2015

130 views

Category:

Engineering


2 download

TRANSCRIPT

Page 1: 14 skip lists

Analysis of AlgorithmsSkip Lists

Andres Mendez-Vazquez

October 18, 2015

1 / 99

Page 2: 14 skip lists

Outline1 Dictionaries

DefinitionsDictionary operationsDictionary implementation

2 Skip ListsWhy Skip Lists?The Idea Behind All of It!!!Skip List DefinitionSkip list implementationInsertion for Skip ListsDeletion in Skip ListsPropertiesSearch and Insertion TimesApplicationsSummary

2 / 99

Page 3: 14 skip lists

Outline1 Dictionaries

DefinitionsDictionary operationsDictionary implementation

2 Skip ListsWhy Skip Lists?The Idea Behind All of It!!!Skip List DefinitionSkip list implementationInsertion for Skip ListsDeletion in Skip ListsPropertiesSearch and Insertion TimesApplicationsSummary

3 / 99

Page 4: 14 skip lists

DictionariesDefinitionA dictionary is a collection of elements; each of which has a unique searchkey.

Uniqueness criteria may be relaxed (multi-set).Do not force uniqueness.

PurposeDictionaries keep track of current members, with periodic insertions anddeletions into the set (similar to a database).

ExamplesMembership in a club.Course records.Symbol table (with duplicates).Language dictionary (Webster, RAE, Oxford).

4 / 99

Page 5: 14 skip lists

DictionariesDefinitionA dictionary is a collection of elements; each of which has a unique searchkey.

Uniqueness criteria may be relaxed (multi-set).Do not force uniqueness.

PurposeDictionaries keep track of current members, with periodic insertions anddeletions into the set (similar to a database).

ExamplesMembership in a club.Course records.Symbol table (with duplicates).Language dictionary (Webster, RAE, Oxford).

4 / 99

Page 6: 14 skip lists

DictionariesDefinitionA dictionary is a collection of elements; each of which has a unique searchkey.

Uniqueness criteria may be relaxed (multi-set).Do not force uniqueness.

PurposeDictionaries keep track of current members, with periodic insertions anddeletions into the set (similar to a database).

ExamplesMembership in a club.Course records.Symbol table (with duplicates).Language dictionary (Webster, RAE, Oxford).

4 / 99

Page 7: 14 skip lists

DictionariesDefinitionA dictionary is a collection of elements; each of which has a unique searchkey.

Uniqueness criteria may be relaxed (multi-set).Do not force uniqueness.

PurposeDictionaries keep track of current members, with periodic insertions anddeletions into the set (similar to a database).

ExamplesMembership in a club.Course records.Symbol table (with duplicates).Language dictionary (Webster, RAE, Oxford).

4 / 99

Page 8: 14 skip lists

DictionariesDefinitionA dictionary is a collection of elements; each of which has a unique searchkey.

Uniqueness criteria may be relaxed (multi-set).Do not force uniqueness.

PurposeDictionaries keep track of current members, with periodic insertions anddeletions into the set (similar to a database).

ExamplesMembership in a club.Course records.Symbol table (with duplicates).Language dictionary (Webster, RAE, Oxford).

4 / 99

Page 9: 14 skip lists

DictionariesDefinitionA dictionary is a collection of elements; each of which has a unique searchkey.

Uniqueness criteria may be relaxed (multi-set).Do not force uniqueness.

PurposeDictionaries keep track of current members, with periodic insertions anddeletions into the set (similar to a database).

ExamplesMembership in a club.Course records.Symbol table (with duplicates).Language dictionary (Webster, RAE, Oxford).

4 / 99

Page 10: 14 skip lists

DictionariesDefinitionA dictionary is a collection of elements; each of which has a unique searchkey.

Uniqueness criteria may be relaxed (multi-set).Do not force uniqueness.

PurposeDictionaries keep track of current members, with periodic insertions anddeletions into the set (similar to a database).

ExamplesMembership in a club.Course records.Symbol table (with duplicates).Language dictionary (Webster, RAE, Oxford).

4 / 99

Page 11: 14 skip lists

DictionariesDefinitionA dictionary is a collection of elements; each of which has a unique searchkey.

Uniqueness criteria may be relaxed (multi-set).Do not force uniqueness.

PurposeDictionaries keep track of current members, with periodic insertions anddeletions into the set (similar to a database).

ExamplesMembership in a club.Course records.Symbol table (with duplicates).Language dictionary (Webster, RAE, Oxford).

4 / 99

Page 12: 14 skip lists

Example: Course records

Dictionary with member recordskey ID Student Name HW1123 Stan Smith 49 · · ·125 Sue Margolin 45 · · ·128 Billie King 24 · · ·

...

...190 Roy Miller 36 · · ·

5 / 99

Page 13: 14 skip lists

Outline1 Dictionaries

DefinitionsDictionary operationsDictionary implementation

2 Skip ListsWhy Skip Lists?The Idea Behind All of It!!!Skip List DefinitionSkip list implementationInsertion for Skip ListsDeletion in Skip ListsPropertiesSearch and Insertion TimesApplicationsSummary

6 / 99

Page 14: 14 skip lists

The dictionary ADT operations

Some operations on dictionariessize(): Returns the size of the dictionary.empty(): Returns TRUE if the dictionary is empty.findItem(key): Locates the item with the specified key.findAllItems(key): Locates all items with the specified key.removeItem(key): Removes the item with the specified key.removeAllItems(key): Removes all items with the specified key.insertItem(key,element): Inserts a new key-element pair.

7 / 99

Page 15: 14 skip lists

The dictionary ADT operations

Some operations on dictionariessize(): Returns the size of the dictionary.empty(): Returns TRUE if the dictionary is empty.findItem(key): Locates the item with the specified key.findAllItems(key): Locates all items with the specified key.removeItem(key): Removes the item with the specified key.removeAllItems(key): Removes all items with the specified key.insertItem(key,element): Inserts a new key-element pair.

7 / 99

Page 16: 14 skip lists

The dictionary ADT operations

Some operations on dictionariessize(): Returns the size of the dictionary.empty(): Returns TRUE if the dictionary is empty.findItem(key): Locates the item with the specified key.findAllItems(key): Locates all items with the specified key.removeItem(key): Removes the item with the specified key.removeAllItems(key): Removes all items with the specified key.insertItem(key,element): Inserts a new key-element pair.

7 / 99

Page 17: 14 skip lists

The dictionary ADT operations

Some operations on dictionariessize(): Returns the size of the dictionary.empty(): Returns TRUE if the dictionary is empty.findItem(key): Locates the item with the specified key.findAllItems(key): Locates all items with the specified key.removeItem(key): Removes the item with the specified key.removeAllItems(key): Removes all items with the specified key.insertItem(key,element): Inserts a new key-element pair.

7 / 99

Page 18: 14 skip lists

The dictionary ADT operations

Some operations on dictionariessize(): Returns the size of the dictionary.empty(): Returns TRUE if the dictionary is empty.findItem(key): Locates the item with the specified key.findAllItems(key): Locates all items with the specified key.removeItem(key): Removes the item with the specified key.removeAllItems(key): Removes all items with the specified key.insertItem(key,element): Inserts a new key-element pair.

7 / 99

Page 19: 14 skip lists

The dictionary ADT operations

Some operations on dictionariessize(): Returns the size of the dictionary.empty(): Returns TRUE if the dictionary is empty.findItem(key): Locates the item with the specified key.findAllItems(key): Locates all items with the specified key.removeItem(key): Removes the item with the specified key.removeAllItems(key): Removes all items with the specified key.insertItem(key,element): Inserts a new key-element pair.

7 / 99

Page 20: 14 skip lists

The dictionary ADT operations

Some operations on dictionariessize(): Returns the size of the dictionary.empty(): Returns TRUE if the dictionary is empty.findItem(key): Locates the item with the specified key.findAllItems(key): Locates all items with the specified key.removeItem(key): Removes the item with the specified key.removeAllItems(key): Removes all items with the specified key.insertItem(key,element): Inserts a new key-element pair.

7 / 99

Page 21: 14 skip lists

Example of unordered dictionary

ExampleConsider an empty unordered dictionary, we have then...

Operation Dictionary OutputInsertItem(5, A) {(5, A)}InsertItem(7, B) {(5, A) , (7, B)}findItem(7) {(5, A) , (7, B)} BfindItem(4) {(5, A) , (7, B)} No Such Key

size() {(5, A) , (7, B)} 2removeItem(5) {(7, B)} AfindItem(4) {(7, B)} No Such Key

8 / 99

Page 22: 14 skip lists

Outline1 Dictionaries

DefinitionsDictionary operationsDictionary implementation

2 Skip ListsWhy Skip Lists?The Idea Behind All of It!!!Skip List DefinitionSkip list implementationInsertion for Skip ListsDeletion in Skip ListsPropertiesSearch and Insertion TimesApplicationsSummary

9 / 99

Page 23: 14 skip lists

How to implement a dictionary?

There are many ways of implementing a dictionarySequences / Arrays

I OrderedI Unordered

Binary search treesSkip listsHash tables

10 / 99

Page 24: 14 skip lists

How to implement a dictionary?

There are many ways of implementing a dictionarySequences / Arrays

I OrderedI Unordered

Binary search treesSkip listsHash tables

10 / 99

Page 25: 14 skip lists

How to implement a dictionary?

There are many ways of implementing a dictionarySequences / Arrays

I OrderedI Unordered

Binary search treesSkip listsHash tables

10 / 99

Page 26: 14 skip lists

How to implement a dictionary?

There are many ways of implementing a dictionarySequences / Arrays

I OrderedI Unordered

Binary search treesSkip listsHash tables

10 / 99

Page 27: 14 skip lists

Recall Arrays...

Unordered array

ComplexitySearching and removing takes O(n).Inserting takes O(1).

ApplicationsThis approach is good for log files where insertions are frequent butsearches and removals are rare.

11 / 99

Page 28: 14 skip lists

Recall Arrays...

Unordered array

ComplexitySearching and removing takes O(n).Inserting takes O(1).

ApplicationsThis approach is good for log files where insertions are frequent butsearches and removals are rare.

11 / 99

Page 29: 14 skip lists

Recall Arrays...

Unordered array

ComplexitySearching and removing takes O(n).Inserting takes O(1).

ApplicationsThis approach is good for log files where insertions are frequent butsearches and removals are rare.

11 / 99

Page 30: 14 skip lists

Recall Arrays...

Unordered array

ComplexitySearching and removing takes O(n).Inserting takes O(1).

ApplicationsThis approach is good for log files where insertions are frequent butsearches and removals are rare.

11 / 99

Page 31: 14 skip lists

More Arrays

Ordered array

ComplexitySearching takes O(log n) time (binary search).Insert and removing takes O(n) time.

ApplicationsThis aproach is good for look-up tables where searches are frequent butinsertions and removals are rare.

12 / 99

Page 32: 14 skip lists

More Arrays

Ordered array

ComplexitySearching takes O(log n) time (binary search).Insert and removing takes O(n) time.

ApplicationsThis aproach is good for look-up tables where searches are frequent butinsertions and removals are rare.

12 / 99

Page 33: 14 skip lists

More Arrays

Ordered array

ComplexitySearching takes O(log n) time (binary search).Insert and removing takes O(n) time.

ApplicationsThis aproach is good for look-up tables where searches are frequent butinsertions and removals are rare.

12 / 99

Page 34: 14 skip lists

More Arrays

Ordered array

ComplexitySearching takes O(log n) time (binary search).Insert and removing takes O(n) time.

ApplicationsThis aproach is good for look-up tables where searches are frequent butinsertions and removals are rare.

12 / 99

Page 35: 14 skip lists

Binary searches

FeaturesNarrow down the search range in stages“High-low” game.

13 / 99

Page 36: 14 skip lists

Binary searches

Example find Element(22)2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW

↑MID

↑HIGH

2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW

↑MID

↑HIGH

2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW

↑MID

↑HIGH

2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW=MID=HIGH

14 / 99

Page 37: 14 skip lists

Binary searches

Example find Element(22)2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW

↑MID

↑HIGH

2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW

↑MID

↑HIGH

2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW

↑MID

↑HIGH

2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW=MID=HIGH

14 / 99

Page 38: 14 skip lists

Binary searches

Example find Element(22)2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW

↑MID

↑HIGH

2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW

↑MID

↑HIGH

2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW

↑MID

↑HIGH

2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW=MID=HIGH

14 / 99

Page 39: 14 skip lists

Binary searches

Example find Element(22)2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW

↑MID

↑HIGH

2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW

↑MID

↑HIGH

2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW

↑MID

↑HIGH

2 4 5 7 8 9 12 14 17 19 22 25 27 28 33

↑LOW=MID=HIGH

14 / 99

Page 40: 14 skip lists

Recall binary search trees

Implement a dictionary with a BSTA binary search tree is a binary tree T such that:

Each internal node stores an item (k, e) of a dictionary.Keys stored at nodes in the left subtree of v are less than or equal tok.Keys stored at nodes in the right subtree of v are greater than orequal to k.

15 / 99

Page 41: 14 skip lists

Recall binary search trees

Implement a dictionary with a BSTA binary search tree is a binary tree T such that:

Each internal node stores an item (k, e) of a dictionary.Keys stored at nodes in the left subtree of v are less than or equal tok.Keys stored at nodes in the right subtree of v are greater than orequal to k.

15 / 99

Page 42: 14 skip lists

Recall binary search trees

Implement a dictionary with a BSTA binary search tree is a binary tree T such that:

Each internal node stores an item (k, e) of a dictionary.Keys stored at nodes in the left subtree of v are less than or equal tok.Keys stored at nodes in the right subtree of v are greater than orequal to k.

15 / 99

Page 43: 14 skip lists

Recall binary search trees

Implement a dictionary with a BSTA binary search tree is a binary tree T such that:

Each internal node stores an item (k, e) of a dictionary.Keys stored at nodes in the left subtree of v are less than or equal tok.Keys stored at nodes in the right subtree of v are greater than orequal to k.

15 / 99

Page 44: 14 skip lists

Binary searches Trees

Problem!!! Keeping a Well Balanced Binary Search Tree can bedifficult!!!

44

17 88

65

54 82

76

80

32

28

29

97

16 / 99

Page 45: 14 skip lists

Not only that...

Binary Search TreesThey are not so well suited for parallel environments.

I Unless a heavy modifications are done

In additionWe want to have a

Compact Data Structure.Using as little memory as possible

17 / 99

Page 46: 14 skip lists

Not only that...

Binary Search TreesThey are not so well suited for parallel environments.

I Unless a heavy modifications are done

In additionWe want to have a

Compact Data Structure.Using as little memory as possible

17 / 99

Page 47: 14 skip lists

Not only that...

Binary Search TreesThey are not so well suited for parallel environments.

I Unless a heavy modifications are done

In additionWe want to have a

Compact Data Structure.Using as little memory as possible

17 / 99

Page 48: 14 skip lists

Not only that...

Binary Search TreesThey are not so well suited for parallel environments.

I Unless a heavy modifications are done

In additionWe want to have a

Compact Data Structure.Using as little memory as possible

17 / 99

Page 49: 14 skip lists

Thus, we have the following possibilities

Unordered array complexitiesInsertion: O(1)Search: O(n)

Ordered array complexitiesInsertion: O(n)Search: O(n log n)

Well balanced binary trees complexitiesInsertion: O(log n)Search: O(log n)

Big Drawback - Complex parallel Implementation and waste ofmemory.

18 / 99

Page 50: 14 skip lists

Thus, we have the following possibilities

Unordered array complexitiesInsertion: O(1)Search: O(n)

Ordered array complexitiesInsertion: O(n)Search: O(n log n)

Well balanced binary trees complexitiesInsertion: O(log n)Search: O(log n)

Big Drawback - Complex parallel Implementation and waste ofmemory.

18 / 99

Page 51: 14 skip lists

Thus, we have the following possibilities

Unordered array complexitiesInsertion: O(1)Search: O(n)

Ordered array complexitiesInsertion: O(n)Search: O(n log n)

Well balanced binary trees complexitiesInsertion: O(log n)Search: O(log n)

Big Drawback - Complex parallel Implementation and waste ofmemory.

18 / 99

Page 52: 14 skip lists

We want something better!!!

For thisWe will present a probabilistic data structure known as SkipList!!!

19 / 99

Page 53: 14 skip lists

Outline1 Dictionaries

DefinitionsDictionary operationsDictionary implementation

2 Skip ListsWhy Skip Lists?The Idea Behind All of It!!!Skip List DefinitionSkip list implementationInsertion for Skip ListsDeletion in Skip ListsPropertiesSearch and Insertion TimesApplicationsSummary

20 / 99

Page 54: 14 skip lists

Starting from Scratch

FirstImagine that you only require to have searches.A first approximation to it is the use of a link list for it (Θ(n) searchcomplexity).Then, using this How do we speed up searches?

Something NotableUse two link list, one a subsequence of the other.

Imagine the two lists as a road system1 The Bottom is the normal road system, L2.2 The Top is the high way system, L1.

21 / 99

Page 55: 14 skip lists

Starting from Scratch

FirstImagine that you only require to have searches.A first approximation to it is the use of a link list for it (Θ(n) searchcomplexity).Then, using this How do we speed up searches?

Something NotableUse two link list, one a subsequence of the other.

Imagine the two lists as a road system1 The Bottom is the normal road system, L2.2 The Top is the high way system, L1.

21 / 99

Page 56: 14 skip lists

Starting from Scratch

FirstImagine that you only require to have searches.A first approximation to it is the use of a link list for it (Θ(n) searchcomplexity).Then, using this How do we speed up searches?

Something NotableUse two link list, one a subsequence of the other.

Imagine the two lists as a road system1 The Bottom is the normal road system, L2.2 The Top is the high way system, L1.

21 / 99

Page 57: 14 skip lists

Starting from Scratch

FirstImagine that you only require to have searches.A first approximation to it is the use of a link list for it (Θ(n) searchcomplexity).Then, using this How do we speed up searches?

Something NotableUse two link list, one a subsequence of the other.

Imagine the two lists as a road system1 The Bottom is the normal road system, L2.2 The Top is the high way system, L1.

21 / 99

Page 58: 14 skip lists

Starting from Scratch

FirstImagine that you only require to have searches.A first approximation to it is the use of a link list for it (Θ(n) searchcomplexity).Then, using this How do we speed up searches?

Something NotableUse two link list, one a subsequence of the other.

Imagine the two lists as a road system1 The Bottom is the normal road system, L2.2 The Top is the high way system, L1.

21 / 99

Page 59: 14 skip lists

Starting from Scratch

FirstImagine that you only require to have searches.A first approximation to it is the use of a link list for it (Θ(n) searchcomplexity).Then, using this How do we speed up searches?

Something NotableUse two link list, one a subsequence of the other.

Imagine the two lists as a road system1 The Bottom is the normal road system, L2.2 The Top is the high way system, L1.

21 / 99

Page 60: 14 skip lists

Outline1 Dictionaries

DefinitionsDictionary operationsDictionary implementation

2 Skip ListsWhy Skip Lists?The Idea Behind All of It!!!Skip List DefinitionSkip list implementationInsertion for Skip ListsDeletion in Skip ListsPropertiesSearch and Insertion TimesApplicationsSummary

22 / 99

Page 61: 14 skip lists

Example

High-Bottom Way System14

14 23 34 42 47 63

34 42

23 / 99

Page 62: 14 skip lists

Thus, we have...

The following ruleTo Search first search in the top one (L1) as far as possible, then godown and search in the bottom one (L2).

24 / 99

Page 63: 14 skip lists

We can use a little bit of optimization

We have the following worst costSearch Cost High-Bottom Way System = Cost Searching Top +...

Cost Search Bottom

Or

Search Cost =length (L1) + Cost Search Bottom

The interesting part is “Cost Search Bottom”This can be calculated by the following quotient:

length (L2)length (L1)

25 / 99

Page 64: 14 skip lists

We can use a little bit of optimization

We have the following worst costSearch Cost High-Bottom Way System = Cost Searching Top +...

Cost Search Bottom

Or

Search Cost =length (L1) + Cost Search Bottom

The interesting part is “Cost Search Bottom”This can be calculated by the following quotient:

length (L2)length (L1)

25 / 99

Page 65: 14 skip lists

Why?

If we think we are jumping

Segment 1 Segment 2

14

14 23 34

34

26 30

26

Then cost of searching each of the bottom segments = 2Thus the ratio is a “decent” approximation to the worst case search

length (L2)length (L1) = 5

3 = 1.66

26 / 99

Page 66: 14 skip lists

Why?

If we think we are jumping

Segment 1 Segment 2

14

14 23 34

34

26 30

26

Then cost of searching each of the bottom segments = 2Thus the ratio is a “decent” approximation to the worst case search

length (L2)length (L1) = 5

3 = 1.66

26 / 99

Page 67: 14 skip lists

Thus, we have...

Then, the cost for a search (when length (L2) = n)

Search Cost = length (L1)+ length (L2)length (L1) = length (L1)+ n

length (L1) (1)

Taking the derivative with respect to length (L1) and making theresult equal 0

1− nlength2 (L1) = 0

27 / 99

Page 68: 14 skip lists

Thus, we have...

Then, the cost for a search (when length (L2) = n)

Search Cost = length (L1)+ length (L2)length (L1) = length (L1)+ n

length (L1) (1)

Taking the derivative with respect to length (L1) and making theresult equal 0

1− nlength2 (L1) = 0

27 / 99

Page 69: 14 skip lists

Final Cost

We have that the optimal length for L1

length (L1) =√

n

Plugging back in (Eq. 1)

Search Cost =√

n + n√n =

√n +√

n = 2×√

n

28 / 99

Page 70: 14 skip lists

Final Cost

We have that the optimal length for L1

length (L1) =√

n

Plugging back in (Eq. 1)

Search Cost =√

n + n√n =

√n +√

n = 2×√

n

28 / 99

Page 71: 14 skip lists

Data structure with a Square Root Relation

Something like this

29 / 99

Page 72: 14 skip lists

NowFor a three layer link list data structure

We get a search cost of 3× 3√

n

In general for k layers, we have

k × k√n

Thus, if we make k = log2 n, we get

Search Cost = log2 n × log2 n√n

= log2 n × (n)1/log2 n

= log2 n × (n)logn 2

= log2 n × 2=Θ (log2 n)

30 / 99

Page 73: 14 skip lists

NowFor a three layer link list data structure

We get a search cost of 3× 3√

n

In general for k layers, we have

k × k√n

Thus, if we make k = log2 n, we get

Search Cost = log2 n × log2 n√n

= log2 n × (n)1/log2 n

= log2 n × (n)logn 2

= log2 n × 2=Θ (log2 n)

30 / 99

Page 74: 14 skip lists

NowFor a three layer link list data structure

We get a search cost of 3× 3√

n

In general for k layers, we have

k × k√n

Thus, if we make k = log2 n, we get

Search Cost = log2 n × log2 n√n

= log2 n × (n)1/log2 n

= log2 n × (n)logn 2

= log2 n × 2=Θ (log2 n)

30 / 99

Page 75: 14 skip lists

Thus

Something NotableWe get the advantages of the binary search trees with a simplerarchitecture!!!

31 / 99

Page 76: 14 skip lists

Thus

Binary Search Trees44

17 88

65

54 82

76

80

32

28

29

97

New Architecture

32 / 99

Page 77: 14 skip lists

Thus

Binary Search Trees44

17 88

65

54 82

76

80

32

28

29

97

New Architecture14

14 23 34 42 47 63

34 42

32 / 99

Page 78: 14 skip lists

Now

We are ready to give aDefinition for Skip List

33 / 99

Page 79: 14 skip lists

Outline1 Dictionaries

DefinitionsDictionary operationsDictionary implementation

2 Skip ListsWhy Skip Lists?The Idea Behind All of It!!!Skip List DefinitionSkip list implementationInsertion for Skip ListsDeletion in Skip ListsPropertiesSearch and Insertion TimesApplicationsSummary

34 / 99

Page 80: 14 skip lists

A Little Bit of History

Skip ListThey were invented by William Worthington "Bill" Pugh Jr.!!!

How is him?He is is an American computer scientist who invented the skip list andthe Omega test for deciding Presburger arithmetic.He was the co-author of the static code analysis tool FindBugs.He was highly influential in the development of the current memorymodel of the Java language together with his PhD student JeremyManson.

35 / 99

Page 81: 14 skip lists

A Little Bit of History

Skip ListThey were invented by William Worthington "Bill" Pugh Jr.!!!

How is him?He is is an American computer scientist who invented the skip list andthe Omega test for deciding Presburger arithmetic.He was the co-author of the static code analysis tool FindBugs.He was highly influential in the development of the current memorymodel of the Java language together with his PhD student JeremyManson.

35 / 99

Page 82: 14 skip lists

A Little Bit of History

Skip ListThey were invented by William Worthington "Bill" Pugh Jr.!!!

How is him?He is is an American computer scientist who invented the skip list andthe Omega test for deciding Presburger arithmetic.He was the co-author of the static code analysis tool FindBugs.He was highly influential in the development of the current memorymodel of the Java language together with his PhD student JeremyManson.

35 / 99

Page 83: 14 skip lists

A Little Bit of History

Skip ListThey were invented by William Worthington "Bill" Pugh Jr.!!!

How is him?He is is an American computer scientist who invented the skip list andthe Omega test for deciding Presburger arithmetic.He was the co-author of the static code analysis tool FindBugs.He was highly influential in the development of the current memorymodel of the Java language together with his PhD student JeremyManson.

35 / 99

Page 84: 14 skip lists

Skip List Definition

DefinitionA skip list for a set S of distinct (key,element) items is a series of listsS0, S1, ..., Sh such that:

Each list Si contains the special keys +∞ and −∞List S0 contains the keys of S in nondecreasing orderEach list is a subsequence of the previous one

I S0 ⊇ S1 ⊇ S2 ⊇ ... ⊇ Sh

List Sh contains only the two special keys

36 / 99

Page 85: 14 skip lists

Skip List Definition

DefinitionA skip list for a set S of distinct (key,element) items is a series of listsS0, S1, ..., Sh such that:

Each list Si contains the special keys +∞ and −∞List S0 contains the keys of S in nondecreasing orderEach list is a subsequence of the previous one

I S0 ⊇ S1 ⊇ S2 ⊇ ... ⊇ Sh

List Sh contains only the two special keys

36 / 99

Page 86: 14 skip lists

Skip List Definition

DefinitionA skip list for a set S of distinct (key,element) items is a series of listsS0, S1, ..., Sh such that:

Each list Si contains the special keys +∞ and −∞List S0 contains the keys of S in nondecreasing orderEach list is a subsequence of the previous one

I S0 ⊇ S1 ⊇ S2 ⊇ ... ⊇ Sh

List Sh contains only the two special keys

36 / 99

Page 87: 14 skip lists

Skip List Definition

DefinitionA skip list for a set S of distinct (key,element) items is a series of listsS0, S1, ..., Sh such that:

Each list Si contains the special keys +∞ and −∞List S0 contains the keys of S in nondecreasing orderEach list is a subsequence of the previous one

I S0 ⊇ S1 ⊇ S2 ⊇ ... ⊇ Sh

List Sh contains only the two special keys

36 / 99

Page 88: 14 skip lists

Skip List Definition

DefinitionA skip list for a set S of distinct (key,element) items is a series of listsS0, S1, ..., Sh such that:

Each list Si contains the special keys +∞ and −∞List S0 contains the keys of S in nondecreasing orderEach list is a subsequence of the previous one

I S0 ⊇ S1 ⊇ S2 ⊇ ... ⊇ Sh

List Sh contains only the two special keys

36 / 99

Page 89: 14 skip lists

Skip List Definition

DefinitionA skip list for a set S of distinct (key,element) items is a series of listsS0, S1, ..., Sh such that:

Each list Si contains the special keys +∞ and −∞List S0 contains the keys of S in nondecreasing orderEach list is a subsequence of the previous one

I S0 ⊇ S1 ⊇ S2 ⊇ ... ⊇ Sh

List Sh contains only the two special keys

36 / 99

Page 90: 14 skip lists

Skip List Definition

Example

12

23

23 26

31

31

31 34-

-

-

-

+

+

+

+

34

44 56 64

64

78

37 / 99

Page 91: 14 skip lists

Skip list search

We search for a key x in a skip list as followsWe start at the first position of the top list.At the current position p, we compare x with y == p.next.key

I x == y: we return p.next.elementI x > y: we scan forwardI x < y: we “drop down”

If we try to drop down past the bottom list, we return null.

38 / 99

Page 92: 14 skip lists

Skip list search

We search for a key x in a skip list as followsWe start at the first position of the top list.At the current position p, we compare x with y == p.next.key

I x == y: we return p.next.elementI x > y: we scan forwardI x < y: we “drop down”

If we try to drop down past the bottom list, we return null.

38 / 99

Page 93: 14 skip lists

Skip list search

We search for a key x in a skip list as followsWe start at the first position of the top list.At the current position p, we compare x with y == p.next.key

I x == y: we return p.next.elementI x > y: we scan forwardI x < y: we “drop down”

If we try to drop down past the bottom list, we return null.

38 / 99

Page 94: 14 skip lists

Skip list search

We search for a key x in a skip list as followsWe start at the first position of the top list.At the current position p, we compare x with y == p.next.key

I x == y: we return p.next.elementI x > y: we scan forwardI x < y: we “drop down”

If we try to drop down past the bottom list, we return null.

38 / 99

Page 95: 14 skip lists

Skip list search

We search for a key x in a skip list as followsWe start at the first position of the top list.At the current position p, we compare x with y == p.next.key

I x == y: we return p.next.elementI x > y: we scan forwardI x < y: we “drop down”

If we try to drop down past the bottom list, we return null.

38 / 99

Page 96: 14 skip lists

Skip list search

We search for a key x in a skip list as followsWe start at the first position of the top list.At the current position p, we compare x with y == p.next.key

I x == y: we return p.next.elementI x > y: we scan forwardI x < y: we “drop down”

If we try to drop down past the bottom list, we return null.

38 / 99

Page 97: 14 skip lists

Example search for 78

x < p.next.key: “drop down”

12

23

23 26

31

31

31 34-

-

-

-

+

+

+

+

34

44 56 64

64

78

p

39 / 99

Page 98: 14 skip lists

Example search for 78

x > p.next.key: “scan forward”

12

23

23 26

31

31

31 34-

-

-

-

+

+

+

+

34

44 56 64

64

78

p

40 / 99

Page 99: 14 skip lists

Example search for 78

x < p.next.key: “drop down”

12

23

23 26

31

31

31 34-

-

-

-

+

+

+

+

34

44 56 64

64

78

p

41 / 99

Page 100: 14 skip lists

Example search for 78

x > p.next.key: “scan forward”

12

23

23 26

31

31

31 34-

-

-

-

+

+

+

+

34

44 56 64

64

78

p

42 / 99

Page 101: 14 skip lists

Example search for 78

x > p.next.key: “scan forward”

12

23

23 26

31

31

31 34-

-

-

-

+

+

+

+

34

44 56 64

64

78

p

43 / 99

Page 102: 14 skip lists

Example search for 78

x < p.next.key: “drop down”

12

23

23 26

31

31

31 34-

-

-

-

+

+

+

+

34

44 56 64

64

78

p

44 / 99

Page 103: 14 skip lists

Example search for 78

x == y: we return p.next.element

12

23

23 26

31

31

31 34-

-

-

-

+

+

+

+

34

44 56 64

64

78p

45 / 99

Page 104: 14 skip lists

Outline1 Dictionaries

DefinitionsDictionary operationsDictionary implementation

2 Skip ListsWhy Skip Lists?The Idea Behind All of It!!!Skip List DefinitionSkip list implementationInsertion for Skip ListsDeletion in Skip ListsPropertiesSearch and Insertion TimesApplicationsSummary

46 / 99

Page 105: 14 skip lists

How do we implement this data structure?

We can implement a skip list with quad-nodesA quad-node stores:

Entry Value

Link to the previous node

Link to the next node

Link to the above node

Link to the below nodeAlso we define special keys PLUS_INF and MINUS_INF, and we modify the keycomparator to handle them.

47 / 99

Page 106: 14 skip lists

How do we implement this data structure?

We can implement a skip list with quad-nodesA quad-node stores:

Entry Value

Link to the previous node

Link to the next node

Link to the above node

Link to the below nodeAlso we define special keys PLUS_INF and MINUS_INF, and we modify the keycomparator to handle them.

47 / 99

Page 107: 14 skip lists

How do we implement this data structure?

We can implement a skip list with quad-nodesA quad-node stores:

Entry Value

Link to the previous node

Link to the next node

Link to the above node

Link to the below nodeAlso we define special keys PLUS_INF and MINUS_INF, and we modify the keycomparator to handle them.

47 / 99

Page 108: 14 skip lists

How do we implement this data structure?

We can implement a skip list with quad-nodesA quad-node stores:

Entry Value

Link to the previous node

Link to the next node

Link to the above node

Link to the below nodeAlso we define special keys PLUS_INF and MINUS_INF, and we modify the keycomparator to handle them.

47 / 99

Page 109: 14 skip lists

How do we implement this data structure?

We can implement a skip list with quad-nodesA quad-node stores:

Entry Value

Link to the previous node

Link to the next node

Link to the above node

Link to the below nodeAlso we define special keys PLUS_INF and MINUS_INF, and we modify the keycomparator to handle them.

47 / 99

Page 110: 14 skip lists

How do we implement this data structure?

We can implement a skip list with quad-nodesA quad-node stores:

Entry Value

Link to the previous node

Link to the next node

Link to the above node

Link to the below nodeAlso we define special keys PLUS_INF and MINUS_INF, and we modify the keycomparator to handle them.

47 / 99

Page 111: 14 skip lists

How do we implement this data structure?

We can implement a skip list with quad-nodesA quad-node stores:

Entry Value

Link to the previous node

Link to the next node

Link to the above node

Link to the below nodeAlso we define special keys PLUS_INF and MINUS_INF, and we modify the keycomparator to handle them.

47 / 99

Page 112: 14 skip lists

Example

Quad-Node Example

48 / 99

Page 113: 14 skip lists

Skip lists uses Randomization

Use of randomizationWe use a randomized algorithm to insert items into a skip list.

Running timeWe analyze the expected running time of a randomized algorithm underthe following assumptions:

The coins are unbiased.The coin tosses are independent.

Worst case running timeThe worst case running time of a randomized algorithm is often large buthas very low probability.

e.g. It occurs when all the coin tosses give “heads.”

49 / 99

Page 114: 14 skip lists

Skip lists uses Randomization

Use of randomizationWe use a randomized algorithm to insert items into a skip list.

Running timeWe analyze the expected running time of a randomized algorithm underthe following assumptions:

The coins are unbiased.The coin tosses are independent.

Worst case running timeThe worst case running time of a randomized algorithm is often large buthas very low probability.

e.g. It occurs when all the coin tosses give “heads.”

49 / 99

Page 115: 14 skip lists

Skip lists uses Randomization

Use of randomizationWe use a randomized algorithm to insert items into a skip list.

Running timeWe analyze the expected running time of a randomized algorithm underthe following assumptions:

The coins are unbiased.The coin tosses are independent.

Worst case running timeThe worst case running time of a randomized algorithm is often large buthas very low probability.

e.g. It occurs when all the coin tosses give “heads.”

49 / 99

Page 116: 14 skip lists

Skip lists uses Randomization

Use of randomizationWe use a randomized algorithm to insert items into a skip list.

Running timeWe analyze the expected running time of a randomized algorithm underthe following assumptions:

The coins are unbiased.The coin tosses are independent.

Worst case running timeThe worst case running time of a randomized algorithm is often large buthas very low probability.

e.g. It occurs when all the coin tosses give “heads.”

49 / 99

Page 117: 14 skip lists

Skip lists uses Randomization

Use of randomizationWe use a randomized algorithm to insert items into a skip list.

Running timeWe analyze the expected running time of a randomized algorithm underthe following assumptions:

The coins are unbiased.The coin tosses are independent.

Worst case running timeThe worst case running time of a randomized algorithm is often large buthas very low probability.

e.g. It occurs when all the coin tosses give “heads.”

49 / 99

Page 118: 14 skip lists

Skip lists uses Randomization

Use of randomizationWe use a randomized algorithm to insert items into a skip list.

Running timeWe analyze the expected running time of a randomized algorithm underthe following assumptions:

The coins are unbiased.The coin tosses are independent.

Worst case running timeThe worst case running time of a randomized algorithm is often large buthas very low probability.

e.g. It occurs when all the coin tosses give “heads.”

49 / 99

Page 119: 14 skip lists

Outline1 Dictionaries

DefinitionsDictionary operationsDictionary implementation

2 Skip ListsWhy Skip Lists?The Idea Behind All of It!!!Skip List DefinitionSkip list implementationInsertion for Skip ListsDeletion in Skip ListsPropertiesSearch and Insertion TimesApplicationsSummary

50 / 99

Page 120: 14 skip lists

Insertion

To insertTo insert an entry (key, object) into a skip list, we use a randomized algorithm:

We repeatedly toss a coin until we get tails:I We denote with i the number of times the coin came up heads.

If i ≥ h, we add to the skip list new lists Sh+1, ..., Si+1:I Each containing only the two special keys.

We search for x in the skip list and find the positions p0, p1, ..., pi of the itemswith largest key less than x in each lists S0, S1, ..., Si .

For j ← 0, ..., i, we insert item (key, object) into list Sj after position pj .

51 / 99

Page 121: 14 skip lists

Insertion

To insertTo insert an entry (key, object) into a skip list, we use a randomized algorithm:

We repeatedly toss a coin until we get tails:I We denote with i the number of times the coin came up heads.

If i ≥ h, we add to the skip list new lists Sh+1, ..., Si+1:I Each containing only the two special keys.

We search for x in the skip list and find the positions p0, p1, ..., pi of the itemswith largest key less than x in each lists S0, S1, ..., Si .

For j ← 0, ..., i, we insert item (key, object) into list Sj after position pj .

51 / 99

Page 122: 14 skip lists

Insertion

To insertTo insert an entry (key, object) into a skip list, we use a randomized algorithm:

We repeatedly toss a coin until we get tails:I We denote with i the number of times the coin came up heads.

If i ≥ h, we add to the skip list new lists Sh+1, ..., Si+1:I Each containing only the two special keys.

We search for x in the skip list and find the positions p0, p1, ..., pi of the itemswith largest key less than x in each lists S0, S1, ..., Si .

For j ← 0, ..., i, we insert item (key, object) into list Sj after position pj .

51 / 99

Page 123: 14 skip lists

Insertion

To insertTo insert an entry (key, object) into a skip list, we use a randomized algorithm:

We repeatedly toss a coin until we get tails:I We denote with i the number of times the coin came up heads.

If i ≥ h, we add to the skip list new lists Sh+1, ..., Si+1:I Each containing only the two special keys.

We search for x in the skip list and find the positions p0, p1, ..., pi of the itemswith largest key less than x in each lists S0, S1, ..., Si .

For j ← 0, ..., i, we insert item (key, object) into list Sj after position pj .

51 / 99

Page 124: 14 skip lists

Insertion

To insertTo insert an entry (key, object) into a skip list, we use a randomized algorithm:

We repeatedly toss a coin until we get tails:I We denote with i the number of times the coin came up heads.

If i ≥ h, we add to the skip list new lists Sh+1, ..., Si+1:I Each containing only the two special keys.

We search for x in the skip list and find the positions p0, p1, ..., pi of the itemswith largest key less than x in each lists S0, S1, ..., Si .

For j ← 0, ..., i, we insert item (key, object) into list Sj after position pj .

51 / 99

Page 125: 14 skip lists

Insertion

To insertTo insert an entry (key, object) into a skip list, we use a randomized algorithm:

We repeatedly toss a coin until we get tails:I We denote with i the number of times the coin came up heads.

If i ≥ h, we add to the skip list new lists Sh+1, ..., Si+1:I Each containing only the two special keys.

We search for x in the skip list and find the positions p0, p1, ..., pi of the itemswith largest key less than x in each lists S0, S1, ..., Si .

For j ← 0, ..., i, we insert item (key, object) into list Sj after position pj .

51 / 99

Page 126: 14 skip lists

Insertion

To insertTo insert an entry (key, object) into a skip list, we use a randomized algorithm:

We repeatedly toss a coin until we get tails:I We denote with i the number of times the coin came up heads.

If i ≥ h, we add to the skip list new lists Sh+1, ..., Si+1:I Each containing only the two special keys.

We search for x in the skip list and find the positions p0, p1, ..., pi of the itemswith largest key less than x in each lists S0, S1, ..., Si .

For j ← 0, ..., i, we insert item (key, object) into list Sj after position pj .

51 / 99

Page 127: 14 skip lists

Insertion

To insertTo insert an entry (key, object) into a skip list, we use a randomized algorithm:

We repeatedly toss a coin until we get tails:I We denote with i the number of times the coin came up heads.

If i ≥ h, we add to the skip list new lists Sh+1, ..., Si+1:I Each containing only the two special keys.

We search for x in the skip list and find the positions p0, p1, ..., pi of the itemswith largest key less than x in each lists S0, S1, ..., Si .

For j ← 0, ..., i, we insert item (key, object) into list Sj after position pj .

51 / 99

Page 128: 14 skip lists

Example: Insertion of 15 in the skip list

First, we use i = 2 to insert S3 into the skip list

-

-

-

12

23

23 26 +

+

+

-

-

-

12

23

23 26 +

+

+

- +- +

52 / 99

Page 129: 14 skip lists

Example: Insertion of 15 in the skip list

Clearly, you first search for the predecessor key!!!

-

-

-

12

23

23 26 +

+

+

-

-

-

12

23

23 26 +

+

+

- +- +

53 / 99

Page 130: 14 skip lists

Example: Insertion of 15 in the skip list

Insert the necessary Quad-Nodes and necessary information

-

-

-

12

23

23 26 +

+

+

- +

pred

54 / 99

Page 131: 14 skip lists

Example: Insertion of 15 in the skip list

Insert the necessary Quad-Nodes and necessary information

-

-

-

12

23

23 26 +

+

+

- +

pred15

55 / 99

Page 132: 14 skip lists

Example: Insertion of 15 in the skip list

Insert the necessary Quad-Nodes and necessary information

-

-

-

12

23

23 26 +

+

+

- +

pred15

15

56 / 99

Page 133: 14 skip lists

Example: Insertion of 15 in the skip list

Finally!!!

-

-

-

12

23

23 26 +

+

+

- +

15

15

15

57 / 99

Page 134: 14 skip lists

Outline1 Dictionaries

DefinitionsDictionary operationsDictionary implementation

2 Skip ListsWhy Skip Lists?The Idea Behind All of It!!!Skip List DefinitionSkip list implementationInsertion for Skip ListsDeletion in Skip ListsPropertiesSearch and Insertion TimesApplicationsSummary

58 / 99

Page 135: 14 skip lists

Deletion

To remove an entry with key x from a skip list, we proceed as followsWe search for x in the skip list and find the positions p0, p1, ..., pi ofthe items with key x, where position pj is in list Sj .We remove positions p0, p1, ..., pi from the lists S0, S1, ..., Si .We remove all but one list containing only the two special keys

59 / 99

Page 136: 14 skip lists

Deletion

To remove an entry with key x from a skip list, we proceed as followsWe search for x in the skip list and find the positions p0, p1, ..., pi ofthe items with key x, where position pj is in list Sj .We remove positions p0, p1, ..., pi from the lists S0, S1, ..., Si .We remove all but one list containing only the two special keys

59 / 99

Page 137: 14 skip lists

Deletion

To remove an entry with key x from a skip list, we proceed as followsWe search for x in the skip list and find the positions p0, p1, ..., pi ofthe items with key x, where position pj is in list Sj .We remove positions p0, p1, ..., pi from the lists S0, S1, ..., Si .We remove all but one list containing only the two special keys

59 / 99

Page 138: 14 skip lists

Example: Delete of 34 in the skip list

We search for 34 in the skip list and find the positions p0, p1, ..., p2 ofthe items with key 34

-

-

-

12

-

23

23 45 +

+

+

+

34

34

34

60 / 99

Page 139: 14 skip lists

Example: Delete of 34 in the skip list

We search for 34 in the skip list and find the positions p0, p1, ..., p2 ofthe items with key 34

-

-

-

12

-

23

23 45 +

+

+

+

34

34

34

61 / 99

Page 140: 14 skip lists

Example: Delete of 34 in the skip list

We start doing the deletion!!!

-

-

-

12

-

23

23 45 +

+

+

+

34

34

34

62 / 99

Page 141: 14 skip lists

Example: Delete of 34 in the skip list

One Quad-Node after another

-

-

-

12

-

23

23 45 +

+

+

+

34

34

63 / 99

Page 142: 14 skip lists

Example: Delete of 34 in the skip list

One Quad-Node after another

-

-

-

12

-

23

23 45 +

+

+

+

34

64 / 99

Page 143: 14 skip lists

Example: Delete of 34 in the skip list

One Quad-Node after another

-

-

-

12

-

23

23 45 +

+

+

+

65 / 99

Page 144: 14 skip lists

Example: Delete of 34 in the skip list

Remove One Level

-

-

-

12

23

23 45 +

+

+

66 / 99

Page 145: 14 skip lists

Outline1 Dictionaries

DefinitionsDictionary operationsDictionary implementation

2 Skip ListsWhy Skip Lists?The Idea Behind All of It!!!Skip List DefinitionSkip list implementationInsertion for Skip ListsDeletion in Skip ListsPropertiesSearch and Insertion TimesApplicationsSummary

67 / 99

Page 146: 14 skip lists

Space usage

Space usageThe space used by a skip list depends on the random bits used by eachinvocation of the insertion algorithm.

68 / 99

Page 147: 14 skip lists

Space : O (n)TheoremThe expected space usage of a skip list with n items is O(n).

ProofWe use the following two basic probabilistic facts:

1 Fact 1: The probability of getting i consecutive heads when flipping acoin is 1

2i .2 Fact 2: If each of n entries is present in a set with probability p, the

expected size of the set is np.1 How? Remember X = X1 + X2 + ... + Xn where Xi is an indicator

function for event Ai = the i element is present in the set. Thus:

E [X ] =n∑

i=1E [Xi ] =

n∑i=1

Pr {Ai}︸ ︷︷ ︸Equivalence E[XA] and Pr{A}

=n∑

i=1p = np

69 / 99

Page 148: 14 skip lists

Space : O (n)TheoremThe expected space usage of a skip list with n items is O(n).

ProofWe use the following two basic probabilistic facts:

1 Fact 1: The probability of getting i consecutive heads when flipping acoin is 1

2i .2 Fact 2: If each of n entries is present in a set with probability p, the

expected size of the set is np.1 How? Remember X = X1 + X2 + ... + Xn where Xi is an indicator

function for event Ai = the i element is present in the set. Thus:

E [X ] =n∑

i=1E [Xi ] =

n∑i=1

Pr {Ai}︸ ︷︷ ︸Equivalence E[XA] and Pr{A}

=n∑

i=1p = np

69 / 99

Page 149: 14 skip lists

Space : O (n)TheoremThe expected space usage of a skip list with n items is O(n).

ProofWe use the following two basic probabilistic facts:

1 Fact 1: The probability of getting i consecutive heads when flipping acoin is 1

2i .2 Fact 2: If each of n entries is present in a set with probability p, the

expected size of the set is np.1 How? Remember X = X1 + X2 + ... + Xn where Xi is an indicator

function for event Ai = the i element is present in the set. Thus:

E [X ] =n∑

i=1E [Xi ] =

n∑i=1

Pr {Ai}︸ ︷︷ ︸Equivalence E[XA] and Pr{A}

=n∑

i=1p = np

69 / 99

Page 150: 14 skip lists

Space : O (n)TheoremThe expected space usage of a skip list with n items is O(n).

ProofWe use the following two basic probabilistic facts:

1 Fact 1: The probability of getting i consecutive heads when flipping acoin is 1

2i .2 Fact 2: If each of n entries is present in a set with probability p, the

expected size of the set is np.1 How? Remember X = X1 + X2 + ... + Xn where Xi is an indicator

function for event Ai = the i element is present in the set. Thus:

E [X ] =n∑

i=1E [Xi ] =

n∑i=1

Pr {Ai}︸ ︷︷ ︸Equivalence E[XA] and Pr{A}

=n∑

i=1p = np

69 / 99

Page 151: 14 skip lists

Space : O (n)TheoremThe expected space usage of a skip list with n items is O(n).

ProofWe use the following two basic probabilistic facts:

1 Fact 1: The probability of getting i consecutive heads when flipping acoin is 1

2i .2 Fact 2: If each of n entries is present in a set with probability p, the

expected size of the set is np.1 How? Remember X = X1 + X2 + ... + Xn where Xi is an indicator

function for event Ai = the i element is present in the set. Thus:

E [X ] =n∑

i=1E [Xi ] =

n∑i=1

Pr {Ai}︸ ︷︷ ︸Equivalence E[XA] and Pr{A}

=n∑

i=1p = np

69 / 99

Page 152: 14 skip lists

Space : O (n)TheoremThe expected space usage of a skip list with n items is O(n).

ProofWe use the following two basic probabilistic facts:

1 Fact 1: The probability of getting i consecutive heads when flipping acoin is 1

2i .2 Fact 2: If each of n entries is present in a set with probability p, the

expected size of the set is np.1 How? Remember X = X1 + X2 + ... + Xn where Xi is an indicator

function for event Ai = the i element is present in the set. Thus:

E [X ] =n∑

i=1E [Xi ] =

n∑i=1

Pr {Ai}︸ ︷︷ ︸Equivalence E[XA] and Pr{A}

=n∑

i=1p = np

69 / 99

Page 153: 14 skip lists

Proof

Now consider a skip list with n entriesUsing Fact 1, an element is inserted in list Si with a probability of

12i

Now by Fact 2The expected size of list Si is

n2i

70 / 99

Page 154: 14 skip lists

Proof

Now consider a skip list with n entriesUsing Fact 1, an element is inserted in list Si with a probability of

12i

Now by Fact 2The expected size of list Si is

n2i

70 / 99

Page 155: 14 skip lists

Proof

The expected number of nodes used by the skip list with height h

h∑i=0

n2i = n

h∑i=0

12i

Here, we have a problem!!! What is the value of h?

71 / 99

Page 156: 14 skip lists

Height h

FirstThe running time of the search and insertion algorithms is affected by theheight h of the skip list.

SecondWe show that with high probability, a skip list with n items has heightO(log n).

72 / 99

Page 157: 14 skip lists

Height h

FirstThe running time of the search and insertion algorithms is affected by theheight h of the skip list.

SecondWe show that with high probability, a skip list with n items has heightO(log n).

72 / 99

Page 158: 14 skip lists

For this, we have the following fact!!!

We use the following Fact 3We can view the level l (xi) = max {j|where xi ∈ Sj} of the elements inthe skip list as the following random variable

Xi = l (xi)

for each element xi in the skip list.

And this is a random variable!!!Remember the insertions!!! Using an unbiased coin!!Thus, all Xi have a geometric distribution.

73 / 99

Page 159: 14 skip lists

For this, we have the following fact!!!

We use the following Fact 3We can view the level l (xi) = max {j|where xi ∈ Sj} of the elements inthe skip list as the following random variable

Xi = l (xi)

for each element xi in the skip list.

And this is a random variable!!!Remember the insertions!!! Using an unbiased coin!!Thus, all Xi have a geometric distribution.

73 / 99

Page 160: 14 skip lists

For this, we have the following fact!!!

We use the following Fact 3We can view the level l (xi) = max {j|where xi ∈ Sj} of the elements inthe skip list as the following random variable

Xi = l (xi)

for each element xi in the skip list.

And this is a random variable!!!Remember the insertions!!! Using an unbiased coin!!Thus, all Xi have a geometric distribution.

73 / 99

Page 161: 14 skip lists

Example for l (xi)

We have

-

-

-

12

-

23

23 45 +

+

+

+

34

34

34

1 2 3 1

74 / 99

Page 162: 14 skip lists

BTW What is the geometric distribution?

k failures where

k = {1, 2, 3, ...}

Probability mass function

Pr (X = k) = (1− p)k−1 p

75 / 99

Page 163: 14 skip lists

BTW What is the geometric distribution?

k failures where

k = {1, 2, 3, ...}

Probability mass function

Pr (X = k) = (1− p)k−1 p

75 / 99

Page 164: 14 skip lists

Probability Mass FunctionFor Different Probabilities

76 / 99

Page 165: 14 skip lists

Then

We have the following inequality for the geometric variables

Pr [Xi > t] ≤ (1− p)t ∀i = 1, 2, ..., n

Because if the cdf F (t) = P (X ≤ t) = 1− (1− p)t+1

Then, we have

Pr{

maxi

Xi > t}≤ n(1− p)t

This comes from Fmaxi Xi (t) = (F (t))n

77 / 99

Page 166: 14 skip lists

Then

We have the following inequality for the geometric variables

Pr [Xi > t] ≤ (1− p)t ∀i = 1, 2, ..., n

Because if the cdf F (t) = P (X ≤ t) = 1− (1− p)t+1

Then, we have

Pr{

maxi

Xi > t}≤ n(1− p)t

This comes from Fmaxi Xi (t) = (F (t))n

77 / 99

Page 167: 14 skip lists

Observations

The maxi Xi

It represents the list with the one entry apart from the special keys.

-

-

-

12

-

23

23 45 +

+

+

+

34

34

34

78 / 99

Page 168: 14 skip lists

Observations

REMEMBER!!!We are talking about a fair coin, thus p = 1

2 .

79 / 99

Page 169: 14 skip lists

Height: 3 log2 n with probability at least 1− 1n2

TheoremA skip list with n entries has height at most 3 log2 n with probability atleast 1− 1

n2

80 / 99

Page 170: 14 skip lists

Proof

Consider a skip list with n entiresBy Fact 3, the probability that list St has at least one item is at most n

2t .

P (|St | ≥ 1) = P(

maxi

Xi > t)

= n2t .

By picking t = 3 log nWe have that the probability that S3 log2 n has at least one entry is at most:

n23 log2 n = n

n3 = 1n2 .

81 / 99

Page 171: 14 skip lists

Proof

Consider a skip list with n entiresBy Fact 3, the probability that list St has at least one item is at most n

2t .

P (|St | ≥ 1) = P(

maxi

Xi > t)

= n2t .

By picking t = 3 log nWe have that the probability that S3 log2 n has at least one entry is at most:

n23 log2 n = n

n3 = 1n2 .

81 / 99

Page 172: 14 skip lists

Look at we want to model

We want to modelThe height of the Skip List is at most t = 3 log2 nEquivalent to the negation of having list S3 log2 n

Then, the probability that the height h = 3 log2 n of the skip list is

P (Skip List height 3 log2 n) = 1− 1n2

82 / 99

Page 173: 14 skip lists

Look at we want to model

We want to modelThe height of the Skip List is at most t = 3 log2 nEquivalent to the negation of having list S3 log2 n

Then, the probability that the height h = 3 log2 n of the skip list is

P (Skip List height 3 log2 n) = 1− 1n2

82 / 99

Page 174: 14 skip lists

Look at we want to model

We want to modelThe height of the Skip List is at most t = 3 log2 nEquivalent to the negation of having list S3 log2 n

Then, the probability that the height h = 3 log2 n of the skip list is

P (Skip List height 3 log2 n) = 1− 1n2

82 / 99

Page 175: 14 skip lists

Finally

The expected number of nodes used by the skip list with height h

Given that h = 3 log2 n

3 log2 n∑i=0

n2i = n

3 log2 n∑i=0

12i

Given the geometric sum

Sm =m∑

k=0rk = 1− rm+1

1− r

83 / 99

Page 176: 14 skip lists

Finally

The expected number of nodes used by the skip list with height h

Given that h = 3 log2 n

3 log2 n∑i=0

n2i = n

3 log2 n∑i=0

12i

Given the geometric sum

Sm =m∑

k=0rk = 1− rm+1

1− r

83 / 99

Page 177: 14 skip lists

We have finallyThe Upper Bound on the number of nodes

n3 log2 n∑

i=0

12i =n

(1− (1/2)3 log2 n+1

1− 1/2

)

=n(

1− 1/2 (1/2log2 n)3

1/2

)

We have then1

2log2 n = 1n

Then

n(

1− 1/2 (1/2log2 n)3

1/2

)= n

(1− 1

2n2

1/2

)= n

(2− 1

2n2

)= 2n − 1

2n84 / 99

Page 178: 14 skip lists

We have finallyThe Upper Bound on the number of nodes

n3 log2 n∑

i=0

12i =n

(1− (1/2)3 log2 n+1

1− 1/2

)

=n(

1− 1/2 (1/2log2 n)3

1/2

)

We have then1

2log2 n = 1n

Then

n(

1− 1/2 (1/2log2 n)3

1/2

)= n

(1− 1

2n2

1/2

)= n

(2− 1

2n2

)= 2n − 1

2n84 / 99

Page 179: 14 skip lists

We have finallyThe Upper Bound on the number of nodes

n3 log2 n∑

i=0

12i =n

(1− (1/2)3 log2 n+1

1− 1/2

)

=n(

1− 1/2 (1/2log2 n)3

1/2

)

We have then1

2log2 n = 1n

Then

n(

1− 1/2 (1/2log2 n)3

1/2

)= n

(1− 1

2n2

1/2

)= n

(2− 1

2n2

)= 2n − 1

2n84 / 99

Page 180: 14 skip lists

Finally

The Upper Bound with probability 1− 1n2

2n − 12n ≤ 2n = O (n)

85 / 99

Page 181: 14 skip lists

Outline1 Dictionaries

DefinitionsDictionary operationsDictionary implementation

2 Skip ListsWhy Skip Lists?The Idea Behind All of It!!!Skip List DefinitionSkip list implementationInsertion for Skip ListsDeletion in Skip ListsPropertiesSearch and Insertion TimesApplicationsSummary

86 / 99

Page 182: 14 skip lists

Search and Insertion Times

Something NotableThe expected number of coin tosses required in order to get tails is 2.

We use thisTo prove that a search in a skip list takes O(log n) expected time.

After all insertions require searches!!!

87 / 99

Page 183: 14 skip lists

Search and Insertion Times

Something NotableThe expected number of coin tosses required in order to get tails is 2.

We use thisTo prove that a search in a skip list takes O(log n) expected time.

After all insertions require searches!!!

87 / 99

Page 184: 14 skip lists

Search and Insertions times

Search timeThe search time in skip list is proportional to

the number of drop-down steps + the number of scan-forward steps

Drop-down stepsThe drop-down steps are bounded by the height of the skip list and thusare O (log2 n) with high probability.

TheoremA search in a skip list takes O (log2 n) expected time.

88 / 99

Page 185: 14 skip lists

Search and Insertions times

Search timeThe search time in skip list is proportional to

the number of drop-down steps + the number of scan-forward steps

Drop-down stepsThe drop-down steps are bounded by the height of the skip list and thusare O (log2 n) with high probability.

TheoremA search in a skip list takes O (log2 n) expected time.

88 / 99

Page 186: 14 skip lists

Search and Insertions times

Search timeThe search time in skip list is proportional to

the number of drop-down steps + the number of scan-forward steps

Drop-down stepsThe drop-down steps are bounded by the height of the skip list and thusare O (log2 n) with high probability.

TheoremA search in a skip list takes O (log2 n) expected time.

88 / 99

Page 187: 14 skip lists

Proof

FirstWhen we scan forward in a list, the destination key does not belong to ahigher list.

A scan-forward step is associated with a former coin toss that gavetailsBy Fact 4, in each list the expected number of scan-forward steps is 2.

89 / 99

Page 188: 14 skip lists

Proof

FirstWhen we scan forward in a list, the destination key does not belong to ahigher list.

A scan-forward step is associated with a former coin toss that gavetailsBy Fact 4, in each list the expected number of scan-forward steps is 2.

89 / 99

Page 189: 14 skip lists

Why?

Given the list Si

Then, the scan-forward intervals (Jumps between xi and xi+1) to the rightof Si are

I1 = [x1, x2],I2 = [x2, x3]...Ik = [xk , +∞]

ThenThese interval exist at level i if and only if all x1, x2, ..., xk belong to Si .

90 / 99

Page 190: 14 skip lists

Why?

Given the list Si

Then, the scan-forward intervals (Jumps between xi and xi+1) to the rightof Si are

I1 = [x1, x2],I2 = [x2, x3]...Ik = [xk , +∞]

ThenThese interval exist at level i if and only if all x1, x2, ..., xk belong to Si .

90 / 99

Page 191: 14 skip lists

We introduce the following concept based on theseintervals

Scan-forward siblingsThese are element that you find in the search path before finding anelement in the upper list.

91 / 99

Page 192: 14 skip lists

Now

Given that a search is being done, Si contains l forward siblingsIt must be the case that given x1, ..., xl scan-forward siblings, we have that

x1, ..., xl /∈ Si+1

and xl+1 ∈ Si+1

92 / 99

Page 193: 14 skip lists

Thus

We haveSince each element of Si is independently chosen to be in Si+1 withprobability p = 1

2 .

We haveThe number of scan-forward siblings is bounded by a geometric randomvariable Xi with parameter p = 1

2 .

Thus, we have thatThe expected number of scan-forward siblings is bounded by 2!!!

Expected # Scan-Fordward Siblings at i ≤ E [Xi ] = 11/2︸ ︷︷ ︸

Mean

= 2

93 / 99

Page 194: 14 skip lists

Thus

We haveSince each element of Si is independently chosen to be in Si+1 withprobability p = 1

2 .

We haveThe number of scan-forward siblings is bounded by a geometric randomvariable Xi with parameter p = 1

2 .

Thus, we have thatThe expected number of scan-forward siblings is bounded by 2!!!

Expected # Scan-Fordward Siblings at i ≤ E [Xi ] = 11/2︸ ︷︷ ︸

Mean

= 2

93 / 99

Page 195: 14 skip lists

Thus

We haveSince each element of Si is independently chosen to be in Si+1 withprobability p = 1

2 .

We haveThe number of scan-forward siblings is bounded by a geometric randomvariable Xi with parameter p = 1

2 .

Thus, we have thatThe expected number of scan-forward siblings is bounded by 2!!!

Expected # Scan-Fordward Siblings at i ≤ E [Xi ] = 11/2︸ ︷︷ ︸

Mean

= 2

93 / 99

Page 196: 14 skip lists

Then

In the worst case scenarioA search is bounded by 2 log2 n = O (log2 n)

An given that a insertion is a (search) + (deletion bounded bythe height)Thus, an insertion is bounded by 2 log2 n + 3 logn n = O (log2 n)

94 / 99

Page 197: 14 skip lists

Then

In the worst case scenarioA search is bounded by 2 log2 n = O (log2 n)

An given that a insertion is a (search) + (deletion bounded bythe height)Thus, an insertion is bounded by 2 log2 n + 3 logn n = O (log2 n)

94 / 99

Page 198: 14 skip lists

Outline1 Dictionaries

DefinitionsDictionary operationsDictionary implementation

2 Skip ListsWhy Skip Lists?The Idea Behind All of It!!!Skip List DefinitionSkip list implementationInsertion for Skip ListsDeletion in Skip ListsPropertiesSearch and Insertion TimesApplicationsSummary

95 / 99

Page 199: 14 skip lists

Applications

We haveCyrus IMAP servers offer a "skiplist" backend Data Baseimplementation.Lucene uses skip lists to search delta-encoded posting lists inlogarithmic time.Redis, an ANSI-C open-source persistent key/value store for Posixsystems, uses skip lists in its implementation of ordered sets.leveldb, a fast key-value storage library written at Google thatprovides an ordered mapping from string keys to string values.Skip lists are used for efficient statistical computations of runningmedians.

96 / 99

Page 200: 14 skip lists

Applications

We haveCyrus IMAP servers offer a "skiplist" backend Data Baseimplementation.Lucene uses skip lists to search delta-encoded posting lists inlogarithmic time.Redis, an ANSI-C open-source persistent key/value store for Posixsystems, uses skip lists in its implementation of ordered sets.leveldb, a fast key-value storage library written at Google thatprovides an ordered mapping from string keys to string values.Skip lists are used for efficient statistical computations of runningmedians.

96 / 99

Page 201: 14 skip lists

Applications

We haveCyrus IMAP servers offer a "skiplist" backend Data Baseimplementation.Lucene uses skip lists to search delta-encoded posting lists inlogarithmic time.Redis, an ANSI-C open-source persistent key/value store for Posixsystems, uses skip lists in its implementation of ordered sets.leveldb, a fast key-value storage library written at Google thatprovides an ordered mapping from string keys to string values.Skip lists are used for efficient statistical computations of runningmedians.

96 / 99

Page 202: 14 skip lists

Applications

We haveCyrus IMAP servers offer a "skiplist" backend Data Baseimplementation.Lucene uses skip lists to search delta-encoded posting lists inlogarithmic time.Redis, an ANSI-C open-source persistent key/value store for Posixsystems, uses skip lists in its implementation of ordered sets.leveldb, a fast key-value storage library written at Google thatprovides an ordered mapping from string keys to string values.Skip lists are used for efficient statistical computations of runningmedians.

96 / 99

Page 203: 14 skip lists

Applications

We haveCyrus IMAP servers offer a "skiplist" backend Data Baseimplementation.Lucene uses skip lists to search delta-encoded posting lists inlogarithmic time.Redis, an ANSI-C open-source persistent key/value store for Posixsystems, uses skip lists in its implementation of ordered sets.leveldb, a fast key-value storage library written at Google thatprovides an ordered mapping from string keys to string values.Skip lists are used for efficient statistical computations of runningmedians.

96 / 99

Page 204: 14 skip lists

Outline1 Dictionaries

DefinitionsDictionary operationsDictionary implementation

2 Skip ListsWhy Skip Lists?The Idea Behind All of It!!!Skip List DefinitionSkip list implementationInsertion for Skip ListsDeletion in Skip ListsPropertiesSearch and Insertion TimesApplicationsSummary

97 / 99

Page 205: 14 skip lists

Summary

SummaryA skip list is a data structure for dictionaries that uses a randomizedinsertion algorithm.In a skip list with n entries:

I The expected space used is O(n)I The expected search, insertion and deletion time is O(log n)

Skip list are fast and simple to implement in practice.

98 / 99

Page 206: 14 skip lists

Summary

SummaryA skip list is a data structure for dictionaries that uses a randomizedinsertion algorithm.In a skip list with n entries:

I The expected space used is O(n)I The expected search, insertion and deletion time is O(log n)

Skip list are fast and simple to implement in practice.

98 / 99

Page 207: 14 skip lists

Summary

SummaryA skip list is a data structure for dictionaries that uses a randomizedinsertion algorithm.In a skip list with n entries:

I The expected space used is O(n)I The expected search, insertion and deletion time is O(log n)

Skip list are fast and simple to implement in practice.

98 / 99

Page 208: 14 skip lists

Summary

SummaryA skip list is a data structure for dictionaries that uses a randomizedinsertion algorithm.In a skip list with n entries:

I The expected space used is O(n)I The expected search, insertion and deletion time is O(log n)

Skip list are fast and simple to implement in practice.

98 / 99

Page 209: 14 skip lists

Summary

SummaryA skip list is a data structure for dictionaries that uses a randomizedinsertion algorithm.In a skip list with n entries:

I The expected space used is O(n)I The expected search, insertion and deletion time is O(log n)

Skip list are fast and simple to implement in practice.

98 / 99

Page 210: 14 skip lists

Summary

SummaryA skip list is a data structure for dictionaries that uses a randomizedinsertion algorithm.In a skip list with n entries:

I The expected space used is O(n)I The expected search, insertion and deletion time is O(log n)

Skip list are fast and simple to implement in practice.

98 / 99

Page 211: 14 skip lists

Thanks

Questions?

99 / 99