amortized analysis and union-find - wordpress.com · 2011-02-28 · amortized complexity of...

Amortized Analysis and Union-Find

02283, Inge Li Gørtz

1mandag den 28. feb 2011

Today

• Amortized analysis

• 3 different methods

• 2 examples

• Union-Find data structures

• Worst-case complexity

• Amortized complexity


Amortized Analysis

• Amortized analysis.

• Average running time per operation over a worst-case sequence of operations.

• Time required to perform a sequence of data operations is averaged over all the operations performed.

• Motivation: traditional worst-case-per-operation analysis can give too pessimistic bound if the only way of having an expensive operation is to have a lot of cheap ones before it.

• Different from average case analysis: average over time, not input.


Amortized Analysis

• Methods.

• Aggregate method

• Accounting method

• Potential method


Aggregate method

• Aggregate.

• Determine total cost.

• Amortized cost = total cost/#operations.


Dynamic Tables

• Doubling strategy.

• Start with empty array of size 1.

• Insert: If array is full create a new array of double the size and reinsert all elements.

• Analysis: n insert operations. Assume n is a power of 2.

• Number of insertions 1 + 2 + 4 + ... + 2log n = O(n).

• Total cost: O(n).

• Amortized cost per insert: O(1).


Accounting method

• Accounting.

• Some types of operations are overcharged.

• Credit allocated with elements in the data structure used to pay for subsequent operations

• Total credit non-negative at all times -> total amortized cost an upper bound on the actual cost.


Dynamic Tables

• Amortized costs:

• Amortized cost of insertion: 3

• 1 for own insertion

• 1 for its first reinsertion.

• 1 to pay for reinsertion of one of the items that have already been reinserted once.


Dynamic Tables

• Analysis: keep 2 credits on each element in the array that is beyond the middle.

• table not full: insert costs 1, and we have 2 credits to save.

• table full, i.e., doubling: half of the elements have 2 credits each. Use these to pay for reinsertion of all in the new array.

• Amortized cost per operation: 3.2 2 2

x x x x x x x x x x x

2 2 2 2 2 2 2 2

x x x x x x x x x x x x x x x x

x x x x x x x x x x x x x x x x


Example: Stack with MultiPop

• Stack with MultiPop.

• Push(e): push element e onto stack.

• MultiPop(k): pop top k elements from the stack

• Worst case: Implement via linked list or array.

• Push: O(1).

• MultiPop: O(k).

• Amortized cost per operation: 2.


Stack: Aggregate Analysis

• Amortized analysis. Sequence of n Push and MultiPop operations.

• Each object popped at most once for each time it is pushed.

• #pops on non-empty stack ≤ #Push operations ≤ n.

• Total time O(n).

• Amortized cost per operation: 2n/n = 2.


Stack: Accounting Method

• Amortized analysis. Sequence of n Push and MultiPop operations.

• Pay 2 credits for each Push.

• Keep 1 credit on each element on the stack.

• Amortized cost per operation:

• Push: 2

• MultiPop: 1 (to pay for pop on empty stack).


Potential method

• Potential functions.

• Prepaid credit (potential) associated with the data structure (money in the bank).

• Can be used to pay for future operations.

• Ensure there is always enough “money in the bank”.

• Amortized cost of an operation: potential cost plus increase in potential due to the operation.

• Di: data structure after i operations

• Potential function Φ(Di) maps Di onto a real value.

• amortized cost = actual cost + Δ(Di) = actual cost + Φ(Di) - Φ(Di-1).


Potential Functions

• Amortized cost:


• Stack.

•Φ(Di) = #elements on the stack.

• amortized cost of Push = 1 + Δ(Di) = 2.

• amortized cost of MultiPop(k): If k’=min(k,|S|) elements are popped.

• if S ≠ ∅: amortized cost = k‘+ Φ(Di) -Φ(Di-1) = k’ - k’ = 0.

• if S = ∅: amortized cost = 1 + Δ(Di) = 1.


Potential Functions

• Amortized cost:


• Dynamic tables

•Φ(Di) =

• L = current array size, k = number of elements in array.

• amortized cost of insertion:

• Array not full: amortized cost = 1 + 2 = 3

• Array full (doubling): Actual cost = L + 1, Φ(Di-1) = L, Φ(Di)=2: amortized cost = L + 1 + (2 - L) = 3.

!2(k ! L/2) if k " L/20 otherwise


Amortized Cost vs Actual Cost

• Total cost:

• ∑ amortized cost = ∑(actual cost + Δ(Di)) =∑ actual cost + Φ(Dn) - Φ(D0).

• ∑ actual cost = ∑ amortized cost + Φ(D0) - Φ(Dn).

• If potential always nonnegative and Φ(D0) = 0 then

∑ actual cost ≤ ∑ amortized cost.


Potential Method

• Summary:

1. Pick a potential function, Φ, that will work (art).

2. Use potential function to bound the amortized cost of the operations you're interested in.

3. Bound Φ(D0) - Φ(Dfinal)

• Techniques to find potential functions: if the actual cost of an operation is high, then decrease in potential due to this operation must be large, to keep the amortized cost low.


Union-Find Data Structures


Union-Find Data Structure

• Union-Find data structure:

• Makeset(x): Create a singleton set containing x and return its identifier.

• Union(A,B): Combine the sets identified by A and B into a new set, destroying the old sets. Return the identifier of the new set.

• Find(x): Return the identifier of the set containing x.

• Only requirement for identifier: find(x) = find(y) iff x and y are in the same set.

• Applications: Connectivity, Kruskal’s algorithm for MST, ...


A Simple Union-Find Data Structure

• Quick-Union:

• Each set represented by a tree. Elements are represented by nodes. Root is also identifier.

• Make-Set(x): Create a new node x. Set p(x) = x.

• Find(x): Follow parent pointers to the root. Return the root.

• Union(A,B): Make root(B) a child of root(A).



• Quick-Union:


1 2 3 4 5 6 7 8 9

1 2 3 4

5

6 7 8 9

1

2 3 4

5

6 7 8 9

1

2 3 4

5

6 7

8

9

1

2 3 4

5

6

7

8

9

Union(3,1)

Union(7,5)

Union(7,8)

Union(3,7)



• Quick-Union:

• Each set represented by a tree. Elements are represented by nodes. Root is also identifier.

• Make-Set(x): Create a new node x. Set p(x) = x.



• Analysis:

• Make-Set(x) and Union(A,B): O(1)

• Find(x): O(h), where h is the height of the tree containing x. Worst-case O(n).



• Quick Find:

• Each set represented by a tree of height at most one. Elements are represented by nodes. Root is also identifier.

• Make-Set(x): Create a new node x. Set p(x) = x and size(x) = 1.

• Find(x): Follow parent pointer to root. Return root.

• Union(A,B): Move all elements from smallest set to larger set (change parent pointers). I.e., set p(B) = A and size(A) = size(A) + size(B).



• Quick Find:

• Union(A,B): Move all elements from smallest set to larger set (change parent pointers).

1 2 3 4 5 6 7 8 9

1 2 3 4

5

6 7 8 9

1

2 3 4

5

6 7 8 9

1

2 3 4

5

6 7

8

9

Union(3,1)

Union(7,5)

Union(7,8)

Union(3,7)

1

2

3

4

5

6 7

8

9



• Quick Find:

• Each set represented by a tree of height at most one. Elements are represented by nodes. Root is also identifier.

• Make-Set(x): Create a new node x. Set p(x) = x and size(x) = 1.

• Find(x): Follow parent pointer to root. Return root.

• Union(A,B): Move all elements from smallest set to larger set (change parent pointers). I.e., set p(B) = A and size(A) = size(A) + size(B).

• Analysis:

• Make-Set(x) and Find(x): O(1)

• Union(A,B): O(n)


Amortized Complexity of Quick-Find

• Amortized analysis: Consider a sequence of k Unions.

• Observation 1: How many elements can be touched by the k Unions?

• Consider an element x:

• What can we say about the size of the set containing x before and after a union that changes x’s parent pointer?

• How large can the set containing x be after the k Unions?

• How many times can x’s parent pointer be changed?


Amortized Complexity of Quick-Find

• Amortized analysis:

• Each time x’s parent pointer changes the size of the set containing it at least doubles.

• At most 2k elements can be touched by k unions.

• Size of set containing x after k unions at most 2k.

• x’s parent pointer is updated at most lg(2k) times.

• In total O(k log k) parent pointers updated in a sequence of k unions.

• Amortized time per union: O(log k).

• Lemma. Using the Quick-Find data structure a Find operation takes worst case time O(1), a Make-Set operation time O(1), and a sequence of n Union operations takes time O(n log n).


A Better Union-Find Data Structure

• Union-by-Weight or Union-by-Rank.

• Union-by-Weight. Make the root of the smallest tree a child of the root of the bigger tree.

• Union-by-Rank. Each node x has an integer rank(x) associated.

• Make-Set(x): Create a new node x. Set p(x) = x and rank(x) = 0.


• Union(A,B): 3 cases:

• rank(A) > rank(B). Make B a child of A.

• rank(A) < rank(B). Make A a child of B.

• rank(A) = rank(B). Make B a child of A and set rank(A) = rank(A)+1.


Analysis of Union-by-Rank

• Increasing ranks.

• rank(x) < rank(p(x)).

• A root of set containing x: Find(x) takes O(rank(A)+1) time.

• rank(A) ≤ lg n: Show |A| ≥ 2rank(A) by induction.

• A=Makeset(x): rank(A)=0 and |A| = 20 = 1.

• A=Union(B,C): 2 cases

• rank(A)=rank(B) or rank(A)=rank(C): ok, since set only got larger.

• rank(B)=rank(C)=k and rank(A)=k+1.

|A| = |B| + |C| ≥ 2k + 2k = 2k+1.

• Lemma. Using the Union-by-Rank data structure a Find operation takes worst case time O(log n), and a Make-Set or Union operation takes time O(1).


Path Compression

• Path compression. After each Find(x) operation, for all nodes y on the path from x to the root, set p(y) = x.

• Example. Find(6).

1

2

3

4

5

6

7

8

9

1

2

3

4

56

7

8

9


Path Compression

• Path compression. After each Find(x) operation, for all nodes y on the path from x to the root, set p(y) = x.

• Tarjan: Union-by-Rank and path compression. Starting from an empty data structure the total time of n Makes-Set operations, at most n Union operations, and m Find operations takes time O(n + m α(n)).

• α(n). Extremely slowly growing inverse Ackermann’s function.

• Analysis complicated, but algorithm simple.

• 2 one-pass variants path halving and path splitting. Same asymptotic running time.


• Ackermann’s function

• Inverse Ackermann:

• Grows extremely slowly.

Ackermann’s function

Ak(j) =

!j + 1 if k = 0,

A(j+1)k!1 (j) if k ! 1.

!(n) = min{k : Ak(1) ! n}.


• Potential function

• Potential of node x after i operations:

• Potential of forest after i operations:

• Auxiliary functions:

•

•

• Properties:

(1)

(2)

Path Compression: Analysis

!i(x) =!

x

!i(x).

!i(x).

level(x) = max{k : rank(p(x)) ! Ak(rank(x))}.

1 ! iter(x) ! rank(x).

0 ! level(x) ! !(x).

iter(x) = max{j : rank(p(x)) ! A(j)level(x)(rank(x))}



•


• Properties:

(1)

(2)

(3)

(4) If x not root and rank(x) > 0,


!i(x) =!

x

!i(x).

1 ! iter(x) ! rank(x).

0 ! level(x) ! !(x).

!i(x) =

!"(n) · rank(x) if x root or rank(x) = 0("(n)! level(x)) · rank(x)! iter(x) if x not root and rank(x) " 1.

0 ! !i(x) ! "(n) · rank(x).

!i(x) < "(n) · rank(x).



•


• Makeset(x): O(1)

• Union(A,B): O(α(n))

• Find(x): O(α(n))

• Lemma (used in analysis of Union and Find). Suppose x not root, and ith operation Union or Find. Then

If also rank(x) > 0 and either level(x) or iter(x) changes, then


!i(x) =!

x

!i(x).

!i(x) =

!"(n) · rank(x) if x root or rank(x) = 0("(n)! level(x)) · rank(x)! iter(x) if x not root and rank(x) " 1.

!i(x) ! !i!1(x).

!i(x) ! !i!1(x)" 1.35mandag den 28. feb 2011

Summary

• Amortized analysis.

• 2 Examples: Dynamic tables, stack with multipop.

• Union-Find Data Structure.

• Union-by-Rank + path compression: worst case + amortized bounds.


amortized analysis and union-find - wordpress.com · 2011-02-28 · amortized complexity of...

Documents