introduction to algorithms l ecture 14 (chap. 22 & 23) greedy algorithms i 22.1 graph...
TRANSCRIPT
![Page 1: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/1.jpg)
Introduction to Algorithms
LECTURE 14 (Chap. 22 & 23)
Greedy Algorithms I• 22.1 Graph representation• 23.1 Minimum spanning trees• 23.1 Optimal substructure• 23.1 Greedy choice• 23.2 Prim’s greedy MST algorithm
![Page 2: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/2.jpg)
Graphs (review)Definition. A directed graph (digraph)G = (V, E) is an ordered pair consisting of• a set V of vertices (singular: vertex),• a set E V × V of edges.
In an undirected graph G = (V, E), the edgeset E consists of unordered pairs of vertices.
In either case, we have |E| =O(V 2) . Moreover,if G is connected, then |E| ≥ |V| – 1, whichimplies that lg |E| = (lgV).
(Review CLRS, Appendix B.)
![Page 3: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/3.jpg)
Adjacency-matrixrepresentation
The adjacency matrix of a graph G = (V, E), whereV = {1, 2, …, n}, is the matrix A[1 . . n, 1 . . n]given by
A 1 2 3 4
1234
0 1 1 0 0 0 1 0 0 0 0 0 0 0 1 0
(V 2) storage denserepresentation.
A[i, j] =1 if (i, j) E, 0 if (i, j) E.
![Page 4: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/4.jpg)
Adjacency-list representationAn adjacency list of a vertex v V is the list Adj[v]of vertices adjacent to v.
Adj[1] = {2, 3}Adj[2] = {3}Adj[3] = {}Adj[4] = {3}For undirected graphs, |Adj[v]| = degree(v).
For digraphs, |Adj[v]| = out-degree(v).
Handshaking Lemma: ∑v∈V = 2 |E| for undirectedgraphs adjacency lists use (V + E) storage —a sparse representation (for either type of graph).
![Page 5: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/5.jpg)
Minimum spanning trees
Input: A connected, undirected graph G = (V, E)with weight function w : E → .• For simplicity, assume that all edge weights are distinct. (CLRS covers the general case.)
Output: A spanning tree T —a tree that connects all vertices — of minimum weight:
![Page 6: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/6.jpg)
Example of MST
![Page 7: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/7.jpg)
Optimal substructure
MST T:(Other edges of Gare not shown.)
Remove any edge (u, v) T. Then, T is partitionedinto two subtrees T1 and T2.
Theorem. The subtree T1 is an MST of G1 = (V1, E1),the subgraph of G induced by the vertices of T1: V1 = vertices of T1, E1 = { (x, y) ∈ E : x, y ∈ V1 }.Similarly for T2.
![Page 8: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/8.jpg)
Proof of optimal substructureProof. Cut and paste: w(T) = w(u, v) + w(T1) + w(T2).If T1 were a lower-weight spanning tree than T1 forG1, then T = {(u, v)} T1 T2 would be alower-weight spanning tree than T for G.
Do we also have overlapping subproblems?• Yes.Great, then dynamic programming may work!• Yes, but MST exhibits another powerful property which leads to an even more efficient algorithm.
![Page 9: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/9.jpg)
Hallmark for “greedy”algorithms
Theorem. Let T be the MST of G = (V, E),and let A V. Suppose that (u, v) ∈ E is theleast-weight edge connecting A to V – A.Then, (u, v) ∈ T.
Greedy-choice propertyA locally optimal choice
is globally optimal.
![Page 10: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/10.jpg)
Proof of theoremProof. Suppose (u, v) T. Cut and paste.
T:
(u, v) = least-weight edgeconnecting A to V – A
A V – A
![Page 11: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/11.jpg)
Proof of theoremProof. Suppose (u, v) T. Cut and paste.
T:
(u, v) = least-weight edgeconnecting A to V – A
A V – A
Consider the unique simple path from u to v in T.
![Page 12: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/12.jpg)
Proof of theoremProof. Suppose (u, v) T. Cut and paste.
T:
(u, v) = least-weight edgeconnecting A to V – A
A V – A
Consider the unique simple path from u to v in T.
Swap (u, v) with the first edge on this path thatconnects a vertex in A to a vertex in V – A.
![Page 13: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/13.jpg)
Proof of theoremProof. Suppose (u, v) T. Cut and paste.
T:
(u, v) = least-weight edgeconnecting A to V – A
A V – A
Consider the unique simple path from u to v in T.
Swap (u, v) with the first edge on this path thatconnects a vertex in A to a vertex in V – A.A lighter-weight spanning tree than T results.
![Page 14: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/14.jpg)
Prim’s algorithmIDEA: Maintain V – A as a priority queue Q. Keyeach vertex in Q with the weight of the leastweightedge connecting it to a vertex in A.Q ← Vkey[v] ← for all v Vkey[s] ← 0 for some arbitrary s Vwhile Q ≠ do u ← EXTRACT-MIN(Q) for each v Adj[u] do if v Q and w(u, v) < key[v] then key[v] ← w(u, v) ⊳ DECREASE-KEY π[v] ← uAt the end, {(v, π[v])} forms the MST.
![Page 15: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/15.jpg)
Example of Prim’s algorithm
∈ A∈ V – A
![Page 16: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/16.jpg)
Example of Prim’s algorithm
∈ A∈ V – A
![Page 17: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/17.jpg)
Example of Prim’s algorithm
∈ A∈ V – A
![Page 18: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/18.jpg)
Example of Prim’s algorithm
∈ A∈ V – A
![Page 19: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/19.jpg)
Example of Prim’s algorithm
∈ A∈ V – A
![Page 20: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/20.jpg)
Example of Prim’s algorithm
∈ A∈ V – A
![Page 21: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/21.jpg)
Example of Prim’s algorithm
∈ A∈ V – A
![Page 22: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/22.jpg)
Example of Prim’s algorithm
∈ A∈ V – A
![Page 23: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/23.jpg)
Example of Prim’s algorithm
∈ A∈ V – A
![Page 24: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/24.jpg)
Example of Prim’s algorithm
∈ A∈ V – A
![Page 25: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/25.jpg)
Example of Prim’s algorithm
∈ A∈ V – A
![Page 26: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/26.jpg)
Example of Prim’s algorithm
∈ A∈ V – A
![Page 27: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/27.jpg)
Example of Prim’s algorithm
∈ A∈ V – A
![Page 28: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/28.jpg)
Analysis of Prim
(V)total
Q ← Vkey[v] ← ∞ for all v ∈ Vkey[s] ← 0 for some arbitrary s ∈ V
while Q ≠
|V|times
degree(u)times
do u ← EXTRACT-MIN(Q)
for each v ∈ Adj[u] do if v ∈ Q and w(u, v) < key[v]
then key[v] ← w(u, v)π[v] ← u
Handshaking Lemma (E) implicit DECREASE-KEY’s.Time = (V)·TEXTRACT-MIN + (E)·TDECREASE-KEY
![Page 29: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/29.jpg)
Analysis of Prim (continued)
Time = (V)·TEXTRACT-MIN + (E)·TDECREASE-KEY
Q TEXTRACT-MIN TDECREASE-KEY Total
array O(V) O(1) O(V2)
binaryheap
O(lg V) O(lg V) O(E lg V)
Fibonacciheap
O(lg V)amortized
O(1)amortized
O(E + V lg V)worst case
![Page 30: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/30.jpg)
MST algorithms
Kruskal’s algorithm (see CLRS):• Uses the disjoint-set data structure (Chapter 21).• Running time = O(E lg V).
Best to date:• Karger, Klein, and Tarjan [1993].• Randomized algorithm.• O(V + E) expected time.
![Page 31: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/31.jpg)
Kruskal Algorithm
MST_KRUSKAL(G, w)
1 A
2 for each vertex v V G [ ]
3 do MAKE-SET(v)
4 sort the edge of E by nondecreasing weight w
5 for each edge ( , )u v E , in order by
nondecreasing weight
6 do if FIND_SET(u)FIND_SET(v)
7 then A A u v {( , )}
8 UNION(u, v)
9 return A Complexity O(E log E)
![Page 32: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/32.jpg)
a
c
i
g f
e
d
h
b
8
4
8
11
7
1
14
10
92
6
7
2
4a
c
i
g f
e
d
h
b
8
4
8
11
7
1
14
10
92
6
7
2
4
a
c
i
g f
e
d
h
b
8
4
8
11
7
1
14
10
92
6
7
2
4a
c
i
g f
e
d
h
b
8
4
8
11
7
1
14
10
92
6
7
2
4
Example of Kruskal Algorithm
![Page 33: Introduction to Algorithms L ECTURE 14 (Chap. 22 & 23) Greedy Algorithms I 22.1 Graph representation 23.1 Minimum spanning trees 23.1 Optimal substructure](https://reader036.vdocument.in/reader036/viewer/2022062304/56649ea45503460f94ba8d1a/html5/thumbnails/33.jpg)
a
c
i
g f
e
d
h
b
8
4
8
11
7
1
14
10
92
6
7
2
4 a
c
i
g f
e
d
h
b
8
4
8
11
7
1
14
10
92
6
7
2
4
a
c
i
g f
e
d
h
b
8
4
8
11
7
1
14
10
92
6
7
2
4 a
c
i
g f
e
d
h
b
8
4
8
11
7
1
14
10
92
6
7
2
4
Continue ……
Example of Kruskal Algorithm