daniel a. spielman mit fast, randomized algorithms for partitioning, sparsification, and the...
TRANSCRIPT
![Page 1: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/1.jpg)
Daniel A. Spielman
MIT
Fast, Randomized Algorithms forPartitioning, Sparsification, andthe Solution of Linear Systems
Joint work with Shang-Hua Teng (Boston University)
![Page 2: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/2.jpg)
Nearly-Linear Time Algorithms forGraph Partitioning, Graph Sparsification, and Solving Linear Systems S-Teng ‘04
Papers
The Mixing Rate of Markov Chains, An Isoperimetric Inequality, and Computing The Volume Lovasz-Simonovits ‘93
The Eigenvalues of Random Symmetric Matrices Füredi-Komlós ‘81
![Page 3: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/3.jpg)
Preconditioning to solve linear system
Overview
Find B that approximates A, such that solving is easy
Three techniques: Augmented Spanning Trees [Vaidya ’90]
Sparsification: Approximate graph by sparse graph
Partitioning: Runtime proportional to #nodes removed
![Page 4: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/4.jpg)
I. Linear system solvers
II. Sparsification using partitioning and random sampling
III. Graph partitioning by truncated random walks
Outline
Eigenvalues of random graphs
Combinatorial and Spectral Graph Theory
![Page 5: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/5.jpg)
Diagonally Dominant Matrices
Just solve for
Solve wherediags of A at least sum of abs of others in row
1 2 3
Ex: Laplacian matrices of weighted graphs = negative weight from i to j diagonal = weighted degree
![Page 6: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/6.jpg)
Complexity of Solving Ax = bA positive semi-definite
General Direct Methods:
Gaussian Elimination (Cholesky)
Conjugate Gradient
Fast matrix inversion
n is dimensionm is number non-zeros
![Page 7: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/7.jpg)
Complexity of Solving Ax = bA positive semi-definite, structured
Planar: Nested dissection [Lipton-Rose-Tarjan ’79]
Tree: like path, work up from leaves
Path: forward and backward elimination
Express A = L LT
![Page 8: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/8.jpg)
Preconditioned Conjugate Gradient
Omit for rest of talk
Iterative Methods
Find easy B that approximates A
Solve in time
Qualityof approximation
Time to solveBy = c
![Page 9: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/9.jpg)
Iteratively solve in time
Main Result
For symmetric, diagonally dominant A
General:
Planar:
![Page 10: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/10.jpg)
Vaidya’s Subgraph Preconditioners
Precondition A by the Laplacian of a subgraph, B
1 2
34
1 2
34
A B
![Page 11: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/11.jpg)
Vaidya ’90 Relate to graph embedding cong/dil
use MSTaugmented MST
History
Gremban, Miller ‘96 Steiner vertices, many details
Bern, Boman, Chen, Gilbert, Hendrickson, Nguyen, Toledo All the details, algebraic approach
Joshi ’97, Reif ‘98 Recursive, multi-level approach
planar:
planar:
![Page 12: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/12.jpg)
Maggs, Miller, Parekh, Ravi, Woo ‘02after preprocessing
History
Boman, Hendrickson ‘01 Low-stretch spanning trees
Clustering, Partitioning, Sparsification, Recursion Lower-stretch spanning trees
S-Teng ‘03 Augmented low-stretch trees
![Page 13: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/13.jpg)
if A-B is positive semi-definite,that is,
if A is positive semi-definite
For A Laplacian of graph with edges E
The relative condition number:
![Page 14: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/14.jpg)
Bounding
For B subgraph of A,
and so
![Page 15: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/15.jpg)
Fundamental Inequality
18
1
2 3
8 7
45
6
![Page 16: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/16.jpg)
18
1
2 3
8 7
45
6
Fundamental Inequality
![Page 17: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/17.jpg)
Application to eigenvalues of graphs
Example: For complete graph on n nodes, all non-zero eigs = n
= 0 for Laplacian matrices, and
-7 -5 -3 -1 1 3 5 7
For path,
![Page 18: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/18.jpg)
Lower bound on
[Gauttery-Leighton-Miller ‘97]
![Page 19: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/19.jpg)
Lower bound on
So
And
![Page 20: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/20.jpg)
Preconditioning with a Spanning Tree B
A B
Every edge of A not in B has unique path in B
![Page 21: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/21.jpg)
When B is a Tree
![Page 22: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/22.jpg)
Low-Stretch Spanning Trees
where
Theorem (Alon-Karp-Peleg-West ’91)
Theorem (Elkin-Emek-S-Teng ’04)
Theorem (Boman-Hendrickson ’01)
![Page 23: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/23.jpg)
B a spanning tree plus s edges, in total
Vaidya’s Augmented Spanning Trees
![Page 24: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/24.jpg)
Adding Edges to a Tree
![Page 25: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/25.jpg)
Adding Edges to a Tree
Partition tree into t sub-trees, balancing stretch
![Page 26: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/26.jpg)
Adding Edges to a Tree
Partition tree into t sub-trees, balancing stretchFor sub-trees connected in A, add one such bridge edge, carefully
(and in blue)
![Page 27: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/27.jpg)
Adding Edges to a Tree
Theorem:
in general
if planar
t sub-trees
improve to
![Page 28: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/28.jpg)
All graphs can be well-approximated by a sparse graph
SparsificationFeder-Motwani ’91, Benczur-Karger ’96
D. Eppstein, Z. Galil, G.F. Italiano, and T. Spencer. ‘93 D. Eppstein, Z. Galil, G.F. Italiano, and A. Nissenzweig ’97
![Page 29: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/29.jpg)
SparsificationBenczur-Karger ’96: Can find sparse subgraph H s.t.
We need
(edges are subset, but different weights)
H has edges
![Page 30: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/30.jpg)
Example: Complete Graph
If A is Laplacian of Kn, all non-zero eigenvalues are n
If B is Laplacian of Ramanujan expander all non-zero eigenvalues satisfy
And so
![Page 31: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/31.jpg)
Example: Dumbell
If B does not contain middle edge,
Kn Kn
A
![Page 32: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/32.jpg)
Example: Grid plus edge
1111
• Random sampling not sufficient.• Cut approximation not sufficient
= (m-1)2 + k(m-1)
(m-1)2
![Page 33: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/33.jpg)
Conductance
Cut = partition of vertices
Conductance of S
S
Conductance of G
![Page 34: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/34.jpg)
Conductance
S S
S
![Page 35: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/35.jpg)
Conductance and Sparsification
If conductance high (expander) can precondition by random sampling
If conductance low can partition graph by removing few edges
Decomposition: Partition of vertex set remove few edges graph on each partition has high conductance
![Page 36: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/36.jpg)
Graph Decomposition
Lemma: Exists partition of vertices
Each Vi has large At most half the edges cross partition
sample these edges
recurse on these edges
Alg:
![Page 37: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/37.jpg)
Graph Decomposition Exists
Each Vi has At most half edges cross
Lemma: Exists partition of vertices
Proof: Let S be largest set s.t.
If
then
![Page 38: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/38.jpg)
Graph Decomposition Exists
Proof: Let S be largest set s.t.
If
then
S
![Page 39: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/39.jpg)
Graph Decomposition Exists
Proof: Let S be largest set s.t.
If
then
S
If S smallonly recurse in S
If S big, V-S not too big
S V - S
Bounded recursion depth
![Page 40: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/40.jpg)
Sparsification from Graph Decomposition
sample these edges
recurse on these edges
Alg:
Theorem: Exists B with #edges <
Need to find the partition in nearly-linear time
![Page 41: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/41.jpg)
Sparsification
Thm: For all A, find B edges
Thm: Given G, find subgraph H s.t.
in time
#edges(H) <
General solve time:
![Page 42: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/42.jpg)
Cheeger’s Inequality[Sinclair-Jerrum ‘89]:
For Laplacian L, and D: diagonal matrix with = degree of node i
![Page 43: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/43.jpg)
Useful if is big
Random Sampling
Given a graph will randomly sample to get
So that is small,
where is diagonal matrix of degrees in
![Page 44: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/44.jpg)
Useful if is big
If
and
Then, for all x orthogonal to (mutual) nullspace
![Page 45: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/45.jpg)
But, don’t want D in there
D has no impact:
So
implies
![Page 46: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/46.jpg)
Work with adjacency matrix
No difference, because
= weight of edge from i to j, 0 if i = j
![Page 47: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/47.jpg)
Random Sampling Rule
Choose param governing sparsity of
Keep edge (i,j) with prob
If keep edge, raise weight by
![Page 48: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/48.jpg)
Random Sampling Rule
guarantees
And,
guarantees expect at most edges in
![Page 49: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/49.jpg)
Random Sampling Theorem
Theorem
Useful for
From now on, all edges have weight 1
Will prove in unweighted case.
![Page 50: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/50.jpg)
Analysis by Trace
Trace = sum of diagonal entries = sum of eigenvalues
For even k
![Page 51: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/51.jpg)
Analysis by Trace
Main Lemma:
Proof of Theorem:
Markov:
k-th root
![Page 52: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/52.jpg)
Expected Trace
21
3
4
![Page 53: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/53.jpg)
Expected Trace
Most terms zero because
So, sum is zero unless each edge appears at least twice.
Will code such walks to count them.
![Page 54: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/54.jpg)
Coding walks
For sequence where
S = {i : edge not used before, either way}
the index of the time edge was used before, either way
![Page 55: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/55.jpg)
Coding walks: Example
2
1
3 4
step: 0, 1, 2, 3, 4, 5, 6, 7, 8vert: 1, 2, 3, 4, 2, 3, 4, 2, 1
S = {1, 2, 3, 4}
1 2 2 3 3 4 4 2
: 5 2 6 3 7 4 8 1
Now, do a few slides pointing out what it meas
![Page 56: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/56.jpg)
Coding walks: Example
2
1
3 4
step: 0, 1, 2, 3, 4, 5, 6, 7, 8vert: 1, 2, 3, 4, 2, 3, 4, 2, 1
S = {1, 2, 3, 4}
1 2 2 3 3 4 4 2
: 5 2 6 3 7 4 8 1
Now, do a few slides pointing out what it meas
![Page 57: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/57.jpg)
S = {1, 2, 3, 4}
1 2 2 3 3 4 4 2
Coding walks: Example
2
1
3 4
step: 0, 1, 2, 3, 4, 5, 6, 7, 8vert: 1, 2, 3, 4, 2, 3, 4, 2, 1
: 5 2 6 3 7 4 8 1
Now, do a few slides pointing out what it meas
![Page 58: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/58.jpg)
S = {1, 2, 3, 4}
1 2 2 3 3 4 4 2
Coding walks: Example
2
1
3 4
step: 0, 1, 2, 3, 4, 5, 6, 7, 8vert: 1, 2, 3, 4, 2, 3, 4, 2, 1
: 5 2 6 3 7 4 8 1
Now, do a few slides pointing out what it meas
![Page 59: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/59.jpg)
: 5 2 6 3 7 4 8 1
S = {1, 2, 3, 4}
1 2 2 3 3 4 4 2
Coding walks: Example
2
1
3 4
step: 0, 1, 2, 3, 4, 5, 6, 7, 8vert: 1, 2, 3, 4, 2, 3, 4, 2, 1
Now, do a few slides pointing out what it meas
![Page 60: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/60.jpg)
Valid s
For each i in S is a neighbor of with a probability of being chosen < 1
That is, can be non-zero
For each i not in S can take edge indicated by
That is
![Page 61: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/61.jpg)
Expected Trace by from Code
![Page 62: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/62.jpg)
Expected Trace from Code
Are ways to choose given
![Page 63: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/63.jpg)
Expected Trace from Code
Are ways to choose given
![Page 64: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/64.jpg)
Random Sampling Theorem
Theorem
Useful for
Random sampling sparsifies graphs of high conductance
![Page 65: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/65.jpg)
Graph Partitioning Algorithms
SDP/LP: too slow
Spectral: one cut quickly, but can be unbalanced many runs
Multilevel (Chaco/Metis): can’t analyze, miss small sparse cuts
New Alg: Based on truncated random walks. Approximates optimal balance,
Can be use to decompose
Run time
![Page 66: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/66.jpg)
Lazy Random Walk
At each step: stay put with prob ½ otherwise, move to neighbor according to weight
Diffusing probability mass: keep ½ for self, distribute rest among neighbors
![Page 67: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/67.jpg)
Lazy Random WalkDiffusing probability mass: keep ½ for self, distribute rest among neighbors
1 0
0
0
Or, with self-loops, distribute evenly over edges
1
3
2
2
![Page 68: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/68.jpg)
Lazy Random WalkDiffusing probability mass: keep ½ for self, distribute rest among neighbors
1/2 1/2
0
0
1
3
2
2
Or, with self-loops, distribute evenly over edges
![Page 69: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/69.jpg)
Lazy Random WalkDiffusing probability mass: keep ½ for self, distribute rest among neighbors
1/3 1/2
1/12
1/12
1
3
2
2
Or, with self-loops, distribute evenly over edges
![Page 70: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/70.jpg)
Lazy Random WalkDiffusing probability mass: keep ½ for self, distribute rest among neighbors
1/4 11/24
7/48
7/48
1
3
2
2
Or, with self-loops, distribute evenly over edges
![Page 71: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/71.jpg)
Lazy Random WalkDiffusing probability mass: keep ½ for self, distribute rest among neighbors
1/8 3/8
2/8
2/8
1
3
2
2
In limit, prob mass ~ degree
![Page 72: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/72.jpg)
Why lazy?
Otherwise, might be no limit
1
1
0
0
![Page 73: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/73.jpg)
Why so lazy to keep ½ mass?
Diagonally dominant matrices
weight of edge from i to j
degree of node i, if i = j
prob go from i to j
![Page 74: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/74.jpg)
Rate of convergence
Cheeger’s inequality: related to conductance
If low conductance, slow convergence
If start uniform on S, prob leave at each step =
After steps, at least ¾ of mass still in S
![Page 75: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/75.jpg)
Lovasz-Simonovits Theorem
If slow convergence, then low conductance
And, can find the cut from highest probability nodes
.137
.135
.134
.129 .112
.094
![Page 76: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/76.jpg)
Lovasz-Simonovits Theorem
From now on, every node has same degree, d
For all vertex sets S, and all t · T
Where
k verts with most prob at step t
![Page 77: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/77.jpg)
Lovasz-Simonovits Theorem
If start from node in set S with small conductance will output a set of small conductance
Extension: mostly contained in S
k verts with most prob at step t
![Page 78: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/78.jpg)
Speed of Lovasz-Simonovits
Want run-time proportional to #vertices removedLocal clustering on massive graph
If want cut of conductancecan run for steps.
Problem: most vertices can have non-zero mass when cut is found
![Page 79: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/79.jpg)
Speeding up Lovasz-Simonovits
Round all small entries of to zero
Algorithm Nibble input: start vertex v, target set size k start: step: all values < map to zero
Theorem: if v in conductance < size k set output a set with conductance < mostly overlapping
![Page 80: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/80.jpg)
The Potential Function
For integer k
Linear in between these points
![Page 81: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/81.jpg)
Concave: slopes decrease
For
![Page 82: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/82.jpg)
Easy Inequality
![Page 83: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/83.jpg)
Fancy Inequality
![Page 84: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/84.jpg)
Fancy Inequality
![Page 85: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/85.jpg)
Chords with little progress
![Page 86: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/86.jpg)
Dominating curve, makes progress
![Page 87: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/87.jpg)
Dominating curve, makes progress
![Page 88: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/88.jpg)
Proof of Easy Inequality
time t-1
time t
Order vertices by probability mass
If top k at time t-1 only connect to top k at time t get equality
![Page 89: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/89.jpg)
Proof of Easy Inequality
time t-1
time t
Order vertices by probability mass
If top k at time t-1 only connect to top k at time t get equality
Otherwise, some mass leaves, and get inequality
![Page 90: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/90.jpg)
External edges from Self Loops
Lemma: For every set S, and every set R of same size At least frac of edges from S don’t hit R
3
3
3
3
3
3
S) = 1/2
S R
Tight example:
![Page 91: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/91.jpg)
External edges from Self Loops
Lemma: For every set S, and every set R of same size At least S) frac of edges from S don’t hit R
Proof:
S R
When R=S, is by definition.Each edge in R-S can absorb d edges from S.But each vertex of S-R has d self-loops that do not go to R.
![Page 92: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/92.jpg)
Proof of Fancy Inequality
time t-1
time t
topin out
It(k) = top + in · top + (in + out)/2 = top/2 + (top + in + out)/2
![Page 93: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/93.jpg)
Local Clustering
Theorem: If S is set of conductance < v is random vertex of S Then output set of conductance < mostly in S, in time proportional to size of output.
![Page 94: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/94.jpg)
Local Clustering
Can it be done for all conductances???
Theorem: If S is set of conductance < v is random vertex of S Then output set of conductance < mostly in S, in time proportional to size of output.
![Page 95: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/95.jpg)
Experimental code: ClusTree
1. Cluster, crudely
2. Make trees in clusters
3. Add edges between trees, optimally
No recursion on reduced matrices
![Page 96: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/96.jpg)
ImplementationClusTree: in java
timing not included could increase total time by 20%
PCGCholeskydroptol Inc CholVaidya
TAUCS Chen, Rotkin, Toledo
Orderings: amd, genmmd, Metis, RCM
Intel Xeon 3.06GHz, 512k L2 Cache, 1M L3 Cache
![Page 97: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/97.jpg)
2D grid, Neumann bdry
run to residual error 10-8
0 0.5 1 1.5 2
x 106
0
20
40
60
80
100
120
num nodes
seco
nd
s
2d Grid, Neumann
pcg/ic/genmmddirect/genmmdVaidyaclusTree
![Page 98: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/98.jpg)
0 0.5 1 1.5 2
x 106
0
20
40
60
80
100
120
num nodes
seco
nd
s
2d Grid, Neumann
pcg/ic/genmmddirect/genmmdVaidyaclusTree
0 0.5 1 1.5 2
x 106
0
20
40
60
80
100
num nodes
seco
nd
s
2d Grid, Dirichlet
direct/genmmdVaidyapcg/mic/rcmclusTree
2d Grid Dirichlet 2D grid, Neumann
Impact of boundary conditions
![Page 99: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/99.jpg)
0 5 10 15
x 105
0
20
40
60
80
100
120
num nodes
seco
nd
s
2d Unstructured Delaunay Mesh, Neumann
Vaidyadirect/Metispcg/ic/genmmdclusTree
2D Unstructured Delaunay Mesh
0 5 10 15
x 105
0
20
40
60
80
100
120
num nodes
seco
nd
s
2d Unstructured Delaunay Mesh, Dirichlet
Vaidyadirect/Metispcg/mic/genmmdclusTree
Dirichlet Neumann
![Page 100: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/100.jpg)
Future Work
Practical Local Clustering
More Sparsification
Other physical problems:
Cheeger’s inequalty
Implications for combinatorics,and Spectral Hypergraph Theory
![Page 101: Daniel A. Spielman MIT Fast, Randomized Algorithms for Partitioning, Sparsification, and the Solution of Linear Systems Joint work with Shang-Hua Teng](https://reader034.vdocument.in/reader034/viewer/2022051401/56649e315503460f94b220f6/html5/thumbnails/101.jpg)
“Nearly-Linear Time Algorithms for Graph Paritioning, Sparsification, and Solving Linear Systems” (STOC ’04, Arxiv)
To learn more
Will be split into two papers:numerical and combinatorial
My lecture notes forSpectral Graph Theory and its Applications