skip lists - cs.bgu.ac.ilds202/wiki.files/07-skip-list.pdf · expectation | example consider a...

Skip lists

Skip lists

Expectation — example

Consider a lottery with 1000 tickets and the following prizes.

1 ticket wins 100$.

9 ticket win 10$ each.

90 ticket win 5$ each.

The remaining tickets do not have prizes.

The cost of a ticket is 1$. Is it worthwhile to participate?

The sum of the prizes is

100 · 1 + 10 · 9 + 5 · 90 + 0 · 900 = 640 < 1000

The average win is

100 · 1 + 10 · 9 + 5 · 90 + 0 · 9001000

= 0.64 < 1

Skip lists


The previous computation can be done using probabilitytheory.

The probability to win 100$ is 1/1000.




The expected win is

100 · 11000

+ 10 · 91000

+ 5 · 901000

+ 0 · 9001000

= 0.64

Skip lists

Probability space

A probability space is a pair (Ω,P) whereΩ is a set of outcomes. Ω is called sample space.P : Ω→ [0, 1] is a function that satisfies∑

x∈Ω P(x) = 1. P is called probability measure.

An event is a subset of Ω.For an event A ⊆ Ω, Pr[A] =

∑x∈A P(x).

Example

The lottery example can be modeled by the followingprobability space.

Ω = {1, 2, . . . , 1000}.P(x) = 1/1000 for all x ∈ Ω.

A1 = {1} (the event of winning 100$)A2 = {2, 3, . . . , 10} (the event of winning 10$), etc.Pr[A1] = 1/1000, Pr[A2] = 9/1000.

Skip lists

Union bound

For every two events A, B ,

Pr[A ∪ B] = Pr[A] + Pr[B]− Pr[A ∩ B] ≤ Pr[A] + Pr[B]

Therefore, for every events A1, . . . ,Ak ,

Pr [A1 ∪ A2 ∪ · · · ∪ Ak ] ≤ Pr[A1] + Pr[A2] + · · ·+ Pr[Ak ]

Skip lists

Expectation

A random variable is a function X : Ω→ R .The expectation of a random variable X is

E [X ] =∑k

k · Pr[X = k].

Example

X (y) =

100 if y = 1

10 if 2 ≤ y ≤ 105 if 11 ≤ y ≤ 1000 otherwise

E [X ] = 100 · 11000

+ 10 · 91000

+ 5 · 901000

+ 0 · 9001000

= 0.64

Skip lists


Suppose we toss a coin 3 times. What is the expected numberof times it lands on “head”?

Ω = {HHH,HHT,HTH,HTT,THH,THT,TTH,TTT}P(x) = 1/8 for every x ∈ Ω.Let X (y) = number of ‘H’s in y .For example, X (HHH) = 3, X (HHT) = 2.

E [X ] = 0 · 18

+ 1 · 38

+ 2 · 38

+ 3 · 18

= 32.

Skip lists

Indicator random variable

An indicator random variable for an event A is a randomvariable X such that

X (y) =

{1 if y ∈ A0 otherwise

If X is an indicator random variable for an event A,

E [X ] = 0 · Pr[X = 0] + 1 · Pr[X = 1] = Pr[A]

Skip lists

Linearity of expectation

Theorem

For random variables X1, . . . ,Xk , E [∑k

i=1 Xi ] =∑k

i=1 E [Xi ].

Example

Suppose we toss a coin 3 times. What is the expected numberof times it lands on “head”?

Let X = number of times the coin lands on “head”.

Let Xi be an indicator random variable for the event thatthe coin lands on “head” in the i -th toss.

X = X1 + X2 + X3, so

E [X ] = E [X1] + E [X2] + E [X3] =1

2+

1

2+

1

2=

3

2

Skip lists

Binomial distribution

A random variable X with binomial distribution withparameters n, p is the number of times a coin lands on “head”when the coin is tossed n times, and the probability to land on“head” is p.

Ω = all H/T strings of length n.

P(y) = pk(1− p)n−k where k in the number of ‘H’s in y .X (y) = number of ‘H’s in y .

Theorem

E [X ] = np.

Skip lists

Geometric distribution

A random variable X with geometric distribution is the numberof coin tosses when a coin is tossed until the first “head”.

Ω = {H,TH,TTH, . . .}.P(H) = 1

2, P(TH) = 1

4, P(TTH) = 1

8, . . .

X (y) = length of y .

Theorem

E [X ] = 2.

E [X ] = 1 · 12

+ 2 · 14

+ 3 · 18

+ · · · = 12

+1

4+

1

8+ · · ·} 1

+1

4+

1

8+ · · ·} 1

2

+1

8+ · · ·} 1

4...

Skip lists

Dynamic set ADT

A dynamic set ADT is a structure that stores a set ofelements. Each element has a (unique) key and satellite data.The structure supports the following operations.

Search(S , k) Return the element whose key is k (return NULLif no element has key k).

Insert(S , x) Add x to S .

Delete(S , x) Remove x from S (the operation receives apointer to x).

Minimum(S) Return the element in S with smallest key.

Maximum(S) Return the element in S with largest key.

Successor(S , x) Return the element in S with smallest keythat is larger than x .key.

Predecessor(S , x) Return the element in S with largest keythat is smaller than x .key.

Skip lists

Skip list

Skip list is an implementation of dynamic set with thefollowing complexities:

Search,Insert: Θ(log n) expected.Delete: Θ(1) expected.Minimum,Maximum,Successor,Predecessor: Θ(1) (worstcase).

Skip lists

Skip list

A skip list for a set S consists of sorted linked lists S0, S1, . . . , Shsuch that:

Each list Si contains dummy elements −∞ and ∞.S0 contains all the elements of S .

Si is a sublist of Si−1.

Sh contains only the elements −∞ and ∞.

12-∞

-∞

-∞

-∞

-∞

-∞

17 20 25 31 38 39 44 50 55 ∞

12 17 25 31 38 44 55 ∞

17 25 55 ∞

17 25 55 ∞

17 ∞

∞

20

20

20

Skip lists

Skip list

Each node stores 4 pointers: next, prev, below, above.

The structure stores a pointer S .topleft to the firstelement in Sh.

For efficient Minimum/Maximum, the structure storespointers to the first/last element in S0.

12-∞

-∞

-∞

-∞

-∞

-∞

17 20 25 31 38 39 44 50 55 ∞

12 17 25 31 38 44 55 ∞

17 25 55 ∞

17 25 55 ∞

17 ∞

∞

20

20

20

Skip lists

Find

Find(S , k)

(1) p ← S .topleft(2) while p.below 6= NULL(3) p ← p.below(4) while p.next.key ≤ k(5) p ← p.next(6) return p // p.key is largest key that is ≤ k

12-∞

-∞

-∞

-∞

-∞

-∞

17 20 25 31 38 39 44 50 55 ∞

12 17 25 31 38 44 55 ∞

17 25 55 ∞

17 25 55 ∞

17 ∞

∞

20

20

20

Skip lists

Find

Find(S , k)

(1) p ← S .topleft(2) while p.below 6= NULL(3) p ← p.below(4) while p.next.key ≤ k(5) p ← p.next(6) return p // p.key is largest key that is ≤ k

12-∞

-∞

-∞

-∞

-∞

-∞

17 20 25 31 38 39 44 50 55 ∞

12 17 25 31 38 44 55 ∞

17 25 55 ∞

17 25 55 ∞

17 ∞

∞

20

20

20

Find(S,50)

Skip lists

Search

Search(S , k)

(1) p ← Find(S , k)(2) if p.key = k(3) return p(4) else(5) return NULL

Skip lists

InsertionInsert(S , x)

(1) p ← Find(S , x .key)(2) Insert x after p(3) while random() < 1/2(4) while p.above = NULL(5) p ← p.prev(6) p ← p.above(7) Insert a copy y of x after p(8) if p = S .topleft crate a new top list

12-∞

-∞

-∞

-∞

-∞

-∞

17 20 25 31 38 39 44 50 55 ∞

12 17 25 31 38 44 55 ∞

17 25 55 ∞

17 25 55 ∞

17 ∞

∞

20

20

20

Skip lists



12-∞

-∞

-∞

-∞

-∞

-∞

17 20 25 31 38 39 44 50 55 ∞

12 17 25 31 38 44 55 ∞

17 25 55 ∞

17 25 55 ∞

17 ∞

∞

20

20

20

40

Skip lists



12-∞

-∞

-∞

-∞

-∞

-∞

17 20 25 31 38 39 44 50 55 ∞

12 17 25 31 38 44 55 ∞

17 25 55 ∞

17 25 55 ∞

17 ∞

∞

20

20

20

40

40

Skip lists



12-∞

-∞

-∞

-∞

-∞

-∞

17 20 25 31 38 39 44 50 55 ∞

12 17 25 31 38 44 55 ∞

17 25 55 ∞

17 25 55 ∞

17 ∞

∞

20

20

20

40

40

40

Skip lists

Deletion

Delete(S , x)

(1) while x 6= NULL(2) Remove x from its list(3) if the list becomes empty, delete the list(4) x ← x .above

12-∞

-∞

-∞

-∞

-∞

-∞

17 20 25 38 39 44 50 55 ∞

12 17 25 38 44 55 ∞

17 25 55 ∞

17 25 55 ∞

17 ∞

∞

20

20

20

31x31

Skip lists

Deletion

Delete(S , x)


12-∞

-∞

-∞

-∞

-∞

-∞

17 20 25 38 39 44 50 55 ∞

12 17 25 38 44 55 ∞

17 25 55 ∞

17 25 55 ∞

17 ∞

∞

20

20

20

31

Skip lists

Deletion

Delete(S , x)


12-∞

-∞

-∞

-∞

-∞

-∞

17 20 25 38 39 44 50 55 ∞

12 17 25 38 44 55 ∞

17 25 55 ∞

17 25 55 ∞

17 ∞

∞

20

20

20

Skip lists

The height of skip list

Let S be a skip list with n elements.

The probability that an element x is in the list Si is 1/2i .

Let Ai be the event that Si has at least one element(excluding ±∞), and Axi be the event that x ∈ Si .Ai =

⋃x∈S A

xi .

Pr[Ai ] = Pr[⋃

x∈S Axi ] ≤

∑x∈S Pr[A

xi ] ≤ n · 1/2i .

Pr[A3 log n] ≤ n/23 log n = n/n3 = 1/n2.The probability that S3 log n doesn’t have elements is atleast 1− 1/n2.In other words, with high probability, the height of theskip list is O(log n).

Skip lists




Let Ai be the event that Si has at least one element(excluding ±∞), and Axi be the event that x ∈ Si .

Ai =⋃

x∈S Axi .

Pr[Ai ] = Pr[⋃

x∈S Axi ] ≤

∑x∈S Pr[A

xi ] ≤ n · 1/2i .


Skip lists





⋃x∈S A

xi .

Pr[Ai ] = Pr[⋃

x∈S Axi ] ≤

∑x∈S Pr[A

xi ] ≤ n · 1/2i .


Skip lists





⋃x∈S A

xi .

Pr[Ai ] = Pr[⋃

x∈S Axi ] ≤

∑x∈S Pr[A

xi ] ≤ n · 1/2i .

Pr[A3 log n] ≤ n/23 log n = n/n3 = 1/n2.

The probability that S3 log n doesn’t have elements is atleast 1− 1/n2.In other words, with high probability, the height of theskip list is O(log n).

Skip lists





⋃x∈S A

xi .

Pr[Ai ] = Pr[⋃

x∈S Axi ] ≤

∑x∈S Pr[A

xi ] ≤ n · 1/2i .

Pr[A3 log n] ≤ n/23 log n = n/n3 = 1/n2.The probability that S3 log n doesn’t have elements is atleast 1− 1/n2.

In other words, with high probability, the height of theskip list is O(log n).

Skip lists





⋃x∈S A

xi .

Pr[Ai ] = Pr[⋃

x∈S Axi ] ≤

∑x∈S Pr[A

xi ] ≤ n · 1/2i .


Skip lists

Space complexity

The expected space for the ±∞ elements is O(log n).


The size of Si (excluding ±∞) has binomial distributionwith parameters n and 1/2i .

Therefore, E [|Si |] = 2 + n/2i .The expected size of the skip list is

O(log n)∑i=0

(2 +

n

2i

)≤ O(log n) + n

∞∑i=0

1

2i= O(log n) + 2n

The expected space of a skip list is Θ(n).

Skip lists

Space complexity

The expected space for the ±∞ elements is O(log n).The probability that an element x is in the list Si is 1/2

i .



O(log n)∑i=0

(2 +

n

2i

)≤ O(log n) + n

∞∑i=0

1

2i= O(log n) + 2n


Skip lists

Space complexity


i .


Therefore, E [|Si |] = 2 + n/2i .

The expected size of the skip list is

O(log n)∑i=0

(2 +

n

2i

)≤ O(log n) + n

∞∑i=0

1

2i= O(log n) + 2n


Skip lists

Space complexity


i .



O(log n)∑i=0

(2 +

n

2i

)≤ O(log n) + n

∞∑i=0

1

2i= O(log n) + 2n


Skip lists

Time complexity – search/insertion

Find(S , k)

(1) p ← S .topleft(2) while p.below 6= NULL(3) p ← p.below(4) while p.next.key ≤ k(5) p ← p.next(6) return p

17 25

17

20TTH

40

40

Find(S,30)The expected number of iterations of the while loop inline 2 is O(log n).

What is the number of iterations of while loop in line 4on a list Si?

Consider the reverse search path for a node with key k .

Skip lists


Reverse search path

Start with the node in S0.

Go up when it is possible, if it is not possible go left.

This path is the same as the find path for a node with key k .

12-∞

-∞

-∞

-∞

-∞

-∞

17 20 25 31 38 39 44 50 55 ∞

12 17 25 31 38 44 55 ∞

17 25 55 ∞

17 25 55 ∞

17 ∞

∞

20

20

20

Skip lists


Reverse search path

Start with the node in S0.

Go up when it is possible, if it is not possible go left.

This path is the same as the find path for a node with key k .

12-∞

-∞

-∞

-∞

-∞

-∞

17 20 25 31 38 39 44 50 55 ∞

12 17 25 31 38 44 55 ∞

17 25 55 ∞

17 25 55 ∞

17 ∞

∞

20

20

20

Find(S,50)

Skip lists


The expected number of nodes the path visits at level i isat most 2.

Let Ri denote the number of nodes the path visits at Si

E [Ri ] ≤ E [|Si |] =n

2i

Let R be the length of the search path for some node

E [R] = E

[∞∑i=0

Ri

]

Skip lists


E [R] = E

[∞∑i=0

Ri

]

=∞∑i=0

E [Ri ]

=

blog nc∑i=0

E [Ri ] +∞∑

i=blog nc+1

E [Ri ]

≤blog nc∑i=0

2 +∞∑

i=blog nc+1

n

2i

≤ 2(log n + 1) +∞∑i=0

1

2i= 2 log n + O(1)

Skip lists

Time complexity — deletion

The time of deletion is linear in the height of the tower ofthe deleted element.

The height of a tower has geometric distribution.

The expected height of a tower is 2.

The expected time of Delete is Θ(1).

Skip lists

The expected height of a skip list

For each j ∈ {1, 2, 3, . . . ,∞}, define the indicator randomvariable

Xj =

{0 if Sj is empty1 if Sj is non-empty

The height, h, of the skiplist is then given by

h =∞∑j=1

Xj

Note that Xj ≤ |Sj |, thus

E [Xj ] ≤ E [|Sj |] = n/2j

Poof is from the book “Open Data Structure: An introduction”by Pat Morin

Skip lists

The expected height of a skip list

E [h] = E

[∞∑j=1

Xj

]

=∞∑j=1

E [Xj ]

=

blog nc∑j=1

E [Xj ] +∞∑

r=blog nc+1

E [Xj ]

≤blog nc∑j=1

1 +∞∑

j=blog nc+1

n/2j

≤ log n +∞∑j=0

1/2j = log n + 2

Skip lists

skip lists - cs.bgu.ac.ilds202/wiki.files/07-skip-list.pdf · expectation | example consider a...

Documents