graph algorithms
DESCRIPTION
Graph Algorithms. Minimum Spanning Trees (MST) Union - Find Dana Shapira. Spanning tree. A spanning tree of G is a subset T E of edges, such that the sub-graph G '=( V,T ) is connected and acyclic. Minimum Spanning Tree. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/1.jpg)
1
Graph Algorithms
Minimum Spanning Trees (MST)Union - Find
Dana Shapira
![Page 2: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/2.jpg)
2
A spanning tree of G is a subset T E of edges, such that the sub-graph G'=(V,T) is connected and acyclic.
Spanning tree
![Page 3: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/3.jpg)
3
Minimum Spanning Tree
Given a graph G = (V, E) and an assignment of weights w(e) to the edges of G, a minimum spanning tree T of G is a spanning tree with minimum total edge weight
5
4
3
3
6
1
6
9
8
3
71
2
7
![Page 4: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/4.jpg)
4
How To Build A Minimum Spanning Tree
General strategy:1. Maintain a set of edges A such that (V, A) is a spanning
forest of G and such that there exists a MST (V, F) of G such that AF.
2. As long as (V, A) is not a tree, find an edge that can be added to A while maintaining the above property.
Generic-MST(G=(V,E))1. A= ;2. while (A is not a spanning tree of G) do3. choose a safe edge e=(u,v)E4. A=A{e}5. return A
![Page 5: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/5.jpg)
5
Cuts
A cut (X, Y) of a graph G = (V, E) is a partition of the vertex set V into two sets X and Y = V \ X.
An edge (v, w) is said to cross the cut (X, Y) if v X and w Y.
A cut (X, Y) respects a set A of edges if no edge in A crosses the cut.
![Page 6: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/6.jpg)
6
A Cut Theorem
Theorem: Let A be a subset of the edges of some minimum spanning tree of G; let (X, Y) be a cut that respects A; and let e be a minimum weight edge that crosses(X, Y). Then A {e} is also a subset of the edges of a minimum spanning tree of G; edge e is safe.
1
49
3
2
4
![Page 7: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/7.jpg)
7
Theorem: Let A be a subset of the edges of some minimum spanning tree of G; let (X, Y) be a cut that respects A; and let e be a minimum weight edge that crosses(X, Y). Then A {e} is also a subset of the edges of a minimum spanning tree of G; edge e is safe.
A Cut Theorem
1
49
3
2
4
A Cut Theorem
![Page 8: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/8.jpg)
8
49
12
Theorem: Let A be a subset of the edges of some minimum spanning tree of G; let (X, Y) be a cut that respects A; and let e be a minimum weight edge that crosses(X, Y). Then A {e} is also a subset of the edges of a minimum spanning tree of G; edge e is safe.
A Cut Theorem
![Page 9: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/9.jpg)
9
e
A Cut Theorem
e
f
T w(e) ≤ w(f)w(e) ≤ w(f)
w(T') ≤ w(T)
u
v
![Page 10: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/10.jpg)
10
Let T be a MST such that AT. If e = (u,v) T, add e to T. The edge e = (u,v) forms a cycle with edges on the path p from u
to v in T. Since u and v are on opposite sides of the cut, there is at least one edge f = (x,y) in T on the path p that also crosses the cut.
f A since the cut respects A. Since f is on the unique path from u to v in T, removing it breaks T into two components.
w(e) ≤ w(f) (why?) Let T ' = T – {f} {e} w(T ') ≤ w(T).
Proof:
![Page 11: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/11.jpg)
11
Proof: The cut (VC, V–VC) respects A, and e is a light edge for this cut. Therefore, e is safe.
Corollary: Let G=(V,E) be a connected undirected graph and A a subset of E included in a minimum spanning tree T for G, and let C=(VC,EC) be a tree in the forest GA=(V,A). If e is a light edge connecting C to some other component in GA, then e is safe for A.
A Cut Theorem
![Page 12: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/12.jpg)
12
Kruskal’s Algorithm
Kruskal(G)1 A ← ∅2 for every edge e = (v, w) of G, sorted by weight3 do if v and w belong to different connected components of (V, A)4 then add edge e to A
(a, d):1 (h, i):1 (c, e):1 (f, h):2 (g, h):2(b, c):3 (b, f):3 (b, e):4 (c, d):5 (f, g):5(e, i):6 (d, g):8 (a, b):9 (c, f):12
2
1
112 8
5
3
9
4
3
25
1
6
a
bc
d
e
f
gi
h2
1
112 8
5
3
9
4
3
25
1
6
a
bc
d
e
f
gi
h
![Page 13: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/13.jpg)
13
Correctness Proof
ei
ei
Sorted edge sequence: e1, e
2, e
3, e
4, e
5, e
6, …, e
i, e
i + 1, e
i + 2, e
i + 3, …, e
n
Every edge ej that cross the cut have a weight w(e
j) ≥ w(e
i).
Hence, edge ei is safe.
![Page 14: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/14.jpg)
14
Union-Find Data Structures
Given a set S of n elements, maintain a partition of S into subsets S
1, S
2, …, S
k
Support the following operations:Union(x, y): Replace sets S
i and S
j such that x S
i and y S
j
with Si S
j in the current partition
Find(x): Returns a member r(Si) of the set S
i that contains x
In particular, Find(x) and Find(y) return the same element if and only if x and y belong to the same set.It is possible to create a data structure that supports the above operations in O(α(n)) amortized time, where α is the inverse Ackermann function.
![Page 15: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/15.jpg)
15
Kruskal’s Algorithm Using Union-Find Data Structure
Kruskal(G,w)A for each vertex vV do
Make-Set(v)sort the edges in E in non-decreasing weight order wfor each edge (u,v)E do
if Find-Set(u) ≠ Find-Set(v) then A A {(u,v)}
Union(u,v) return A
![Page 16: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/16.jpg)
16
Kruskal’s Algorithm Using Union-Find Data Structure
Analysis:O(|E| log |E|) time for everything except the operations on SCost of operations on S:
O(α(|E|,|V|)) amortized time per operation on S|V| – 1 Union operations|E| Find operationsTotal: O((|V| + |E|)α(|E|,|V|)) running time
Total running time: O(|E| lg |E|).
![Page 17: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/17.jpg)
17
Prim’s AlgorithmPrim(G)1 for every vertex v of G2 do label v as unexplored3 for every edge e of G4 do label e as unexplored and non tree edge5 s ← some vertex of G6 Mark s as visited7 Q ← Adj(s)8 while Q is not empty9 do (u, w) ← DeleteMin(Q)10 if (u, w) is unexplored11 then if w is unexplored12 then mark edge (u, w) as tree edge13 mark vertex w as visited14 Insert(Q, Adj(w))
2
1
112 8
5
3
9
4
3
25
1
6
a
bc
d
e
f
gi
h
![Page 18: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/18.jpg)
18
Correctness Proof
Observation: At all times during the algorithm, the set of tree edges defines a tree that contains all visited vertices; priority queue Q contains all unexplored edges incident to these vertices.
Corollary: Prim’s algorithm constructs a minimum spanning tree of G.
![Page 19: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/19.jpg)
19
Union/Find
Assumptions: The Sets are disjoint. Each set is identified by a representative of the set.
Initial state: A union/find structure begins with n elements, each considered
to be a one element set. Functions:
Make-Set(x): Creates a new set with element x in it. Union(x,y): Make one set out of the sets containing x and y. Find-Set(x): Returns a pointer to the representative of the set
containing x.
![Page 20: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/20.jpg)
20
Basic Notation
The elements in the structure will be numbered 0 to n-1 Each set will be referred to by the number of one of the
element it contains Initially we have sets S0,S1,…,Sn-1
If we were to call Union(S2,S4), these sets would be removed from the list, and the new set would now be called either S2 or S4
Notations: n Make-Set operations m total operations nm
![Page 21: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/21.jpg)
21
First Attempt
Represent the Union/Find structure as an array arr of n elements
arr[i] contains the set number of element i Initially, arr[i]=i (Make-Set(i))
Find-Set(i) just returns the value of arr[i] To perform Union(Si,Sj):
For every k such that arr[k]=j, set arr[k]=i
![Page 22: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/22.jpg)
22
Analysis
The worst-case analysis: Find(i) takes O(1) time Union(Si,Sj) takes (n) time
A sequence of n Unions will take (n2) time
![Page 23: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/23.jpg)
23
Second Attempt
Represent the Union/Find structure using linked lists. Each element points to another element of the set. The representative is the first element of the set. Each element points to the representative. How do we perform Union(Si,Sj)?
![Page 24: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/24.jpg)
24
Analysis
The worst-case analysis: Find(i) takes O(1) time Make-Set(i) takes O(1) time Union(Si,Sj) takes (n) time (Why?)
A sequence of n Unions-Find will take (n2) time (Example?)
![Page 25: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/25.jpg)
25
Up-Trees A simple data structure for implementing disjoint sets is the up-
tree. We visualize each element as a node A set will be visualized as a directed tree
Arrows will point from child to parent The set will be referred to by its root
A
H
W
H, A and W belong to the same set. H is the representative
B
X
R
F
X, B, R and F are in the same set. X is the representative
![Page 26: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/26.jpg)
26
Operations in Up-Trees
Follow pointer to representative element.
find(x) {
if (x≠p(x)) // not the representative
then p(x)find(p(x));return p(x);
}
![Page 27: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/27.jpg)
27
Union
Union is more complicated.
Make one representative element point to the other, but which way?Does it matter?
![Page 28: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/28.jpg)
28
Union(H, X)
A
H
W B
X
R
F
A
H
W B
X
R
F
X points to H
B, R and F are now deeper
H points to X
A and W are now deeper
![Page 29: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/29.jpg)
29
A worst case for Union
Union can be done in O(1), but may cause find to become O(n)
A B C D E
Consider the result of the following sequence of operations:
Union (A, B)Union (C, A)Union (D, C)Union (E, D)
![Page 30: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/30.jpg)
30
Array Representation of Up-tree
Assume each element is associated with an integer i=0…n-1. From now on, we deal only with i.
Create an integer array, A[n] An array entry is the element’s parent A -1 entry signifies that element i is the representative element.
![Page 31: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/31.jpg)
31
Array Representation of Up-tree
Now the union algorithm might be:Union(x,y) {
A[y] = x; // attaches y to x}
The find algorithm would befind(x) {
if (A[x] < 0)return(x);
elsereturn(find(A[x]));
}Performance: ???
![Page 32: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/32.jpg)
32
Analysis
Worst case: Union(Si,Sj) take O(1) time Find(i) takes O(n) time
Can we do better in an amortized analysis? What is the maximum amount of time n operations could
take us? Suppose we perform n/2 unions followed by n/2 finds
The n/2 unions could give us one tree of height n/2-1 Thus the total time would be n/2 + (n/2)(n/2) = O(n2)
This strategy doesn’t really help…
![Page 33: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/33.jpg)
33
Array Representation of Up-tree
There are two heuristics that improve the performance of union-find. Union by weight Path compression on find
![Page 34: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/34.jpg)
34
Union by Weight HeuristicMake-Set(x) {
p(x)x;rank(x)=0;
}Always attach smaller tree to larger.
union(x,y) {LINK(FIND-Set(x),Find-Set(y))
}LINK(x,y){
if (rank(x) > rank(y)) { p(y)x;else p(x)y;if(rank(x)=rank(y)){
rank(x) = rank(y)+1;}
}
![Page 35: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/35.jpg)
35
Union by Weight HeuristicLet’s change the weight from rank to number of nodes: union(x,y) {
rep_x = find(x);rep_y = find(y);if (weight[rep_x] < weight[rep_y]) {
A[rep_x] = rep_y;weight[rep_y] += weight[rep_x];
}else {
A[rep_y] = rep_x;weight[rep_x] += weight[rep_y];
}}
![Page 36: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/36.jpg)
36
Implementation
Still represent this with an array of n nodes If element i is a “root node”, then arr[i]= -s, where s is
the size of that set. Otherwise, arr[i] is the index of i’s “parent”
If arr[i] < arr[j], then set arr[i] to arr[i]+arr[j] and set arr[j]to i
Else, set arr[j] to arr[i]+arr[j] and set arr[j] to i
![Page 37: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/37.jpg)
37
Implementation
8
4 2 7
56 9
-2-1802448-77
0
3
1
0123456789
![Page 38: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/38.jpg)
38
Performance w/ Union by Weight If unions are done by weight, the depth of any element is never
greater than lg N. Initially, every element is at depth zero. When its depth increases as a result of a union
operation (it’s in the smaller tree), it is placed in a tree that becomes at least twice as large as before (union of two equal size trees).
How often can each union be done? -- lg n times, because after at most lg n unions, the tree will contain all n elements.
Therefore, find becomes O(lg n) when union by weight is used.
![Page 39: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/39.jpg)
39
New Bound on h
Theorem:Assume we start with a Union/Find structure where each set has 1 node, and perform a sequence of Weighted Unions. Then any tree T of m nodes has a height no greater than log2 m.
![Page 40: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/40.jpg)
40
Proof
Base case: If m=1, then this is clearly true Assumption: Assume it is true for all trees of size m-1 or less Proof: Let T be a tree of m nodes created by a sequence of
Weighted Unions. Consider the last union: Union(Sj,Sk). Assume Sj is the smaller tree. If Sj has a nodes, then Sk has m-a nodes, and 1 a m/2.
![Page 41: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/41.jpg)
41
Proof (continued)
The height of T is either:
The height of Tk
One more than the height of Tj
Since a m-a m-1, the assumptions applies to both Tk and Tj
If T has the height of Tk, then
h log2(m-a) log2m If T is one greater than the height of Tj:
h log2a+1 log2m/2)+1 log2m
![Page 42: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/42.jpg)
42
Example
0 1 2 3 4 5 6 7
Is the bound tight?Yes: “pair them off”
Union(S0,S1), Union(S2,S3), Union(S4,S5), Union(S6,S7), Union(S0,S2), Union(S4,S6), Union(S0,S4)
![Page 43: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/43.jpg)
43
Example
0
1
2
3
4
5
6
7
![Page 44: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/44.jpg)
44
Example
0
1 2
3
4
5 6
7
![Page 45: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/45.jpg)
45
Example
0
1 2
3
4
5 6
7
![Page 46: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/46.jpg)
46
Analysis
Worst case: Union is still O(1) Find is now O(log n)
Amortized case: A “worst amortized case” can be achieved if we perform n/2
unions and n/2 finds Take O(n log n) time
Conclusion: This is better, but we can improve it further
![Page 47: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/47.jpg)
47
Path Compression Each time we do a find on an element x, we make all elements on path
from root to x be immediate children of root by making each element’s parent be the representative.find(x) {
if (A[x]<0)return(x);
A[x] = find(A[x]);return (A[x]);
} When path compression is done, a sequence of m operations takes
O(m lg n) time. Amortized time is O(lg n) per operation.
![Page 48: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/48.jpg)
48
Find(7)
0
1 2
3
4
5 6
7
0
1 2
3 5
4 6 7
![Page 49: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/49.jpg)
49
Analysis
The worst case analysis does not change In fact, we are going to have to increase the worst-case
time of Find by a constant factor The amortized analysis does get better
we need to define Ackerman’s function
![Page 50: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/50.jpg)
50
Performance with Both Optimizations When both optimizations are performed, for a sequence
of m operations (m n) (unions and finds), it takes no more than O(m lg* n) time. lg*n is the iterated (base 2) logarithm of n. The
number of times you take lg n before n becomes 1. Example:
lg*16=3 lg*65536=4 lg*265536=5
Union-find is essentially O(m) for a sequence of m operations (Amortized O(1)).
222
*log 2 1n
n
![Page 51: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/51.jpg)
51
Ackerman’s Function
Ackerman’s A(i,j) is defined as follows:
A(1,j) = 2j for j 1
A(i,1) = A(i-1,2) for i 2
A(i,j) = A(i-1,A(i,j-1)) for i,j 2 Some values:
A(2,1) = 4 A(2,2) = 16 A(2,3) = 65536 A(2,4) = > 10^100 (probably much greater than this) A(4,4) = 2^(2^(2^65536)) - 3
![Page 52: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/52.jpg)
52
(p,q)
The function (m,n) is related to the inverse of Ackerman function
For m n 1:
(m,n) = min{z 1 | A(z, m/n) > log2n} This is a very slow growing function:
(m,n) 3 if n < 216
n has to be huge to make (m,n) = 4
![Page 53: Graph Algorithms](https://reader036.vdocument.in/reader036/viewer/2022062519/56814f05550346895dbc9930/html5/thumbnails/53.jpg)
53
Analysis
Lemma (Tarjan and Van Leeuwen): If we start with a Union/Find structure with one element per set (n elements), and let T(f,u) be the time to process any sequence of f finds and u unions. If u n/2 then
T(f,u) = (n+(f+n,n)) Translation: This is basically linear
In practice, you will never have n large enough to give (f+n,n) a value of even 4