laplacian matrices of...
Post on 07-Jun-2020
2 Views
Preview:
TRANSCRIPT
Laplacian Matrices of Graphs
Daniel A. Spielman Dept. of Computer Science
Program in Applied Mathematics Yale Institute for Network Science
MTNS, July 7, 2014
Outline
Examples of graphs and networks Physical metaphors for graphs and the Laplacian Some applications Algorithms and theorems
Graphs and Networks V: a set of vertices (nodes) E: a set of edges an edge is a pair of vertices
Dan
Donna Allan
Gary Maria
Nikhil
Shang-Hua
A small social network
Graphs and Networks
A big social network
Theorem: There is no nice picture of this network
from: http://nrc.uchsc.edu/STATES/united-states-map.jpg
from: http://nrc.uchsc.edu/STATES/united-states-map.jpg
Examples of Graphs
Nearest neighbor graphs of points
Nearest neighbor graphs of points
Nearest neighbor graphs from data
Nearest neighbor graphs from data
Cartoon Graphs
Path
Grid
Examples of Graphs
8
5
1
7 3 2
9 4 6
10
8
5
1
7 3 2
9 4 6
10
Dan
Amy Allan
Gary Maria
Nikhil
Shang-‐Hua
Examples of Graphs
8
5
1
7 3 2
9 4 6
10
Dan
Amy Allan
Gary Maria
Nikhil
Shang-‐Hua
Examples of Graphs
How to understand large-scale structure
Use physical metaphors Edges as rubber bands Edges as resistors
Examine processes
Diffusion of gas Spilling paint
Identify structures
Communities Hierarchy
Nail down some verEces, let rest seIle
View edges as rubber bands or ideal linear springs spring constant 1 (for now)
Graphs as Spring Networks
Nail down some verEces, let rest seIle
View edges as rubber bands or ideal linear springs spring constant 1 (for now)
Graphs as Spring Networks
When stretched to length potenEal energy is
`
�2/2
Nail down some verEces, let rest seIle.
Physics: posiEon minimizes total potenEal energy
subject to boundary constraints (nails)
x(a)a
1
2
X
(a,b)2E
(x(a)� x(b))2
Graphs as Spring Networks
Nail down some verEces, let rest seIle
Energy minimized when free verEces are averages of neighbors
x(a)a
�x(a) =1
da
X
(a,b)2E
�x(b)
Graphs as Spring Networks
is degree of , the number of attached edges da a
If nail down a face of a planar 3-‐connected graph, get a planar embedding!
Tutte’s Theorem ‘63
3-‐connected: cannot break graph by cuYng 2 edges
Tutte’s Theorem ‘63
3-‐connected: cannot break graph by cuYng 2 edges
Tutte’s Theorem ‘63
Tutte’s Theorem ‘63 3-‐connected: cannot break graph by cuYng 2 edges
Tutte’s Theorem ‘63 3-‐connected: cannot break graph by cuYng 2 edges
Graphs as Resistor Networks
View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow.
1V
0V
Graphs as Resistor Networks
View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow. Induced voltages minimize
X
(a,b)2E
(v(a)� v(b))2
Subject to fixed voltages (by battery)
Graphs as Resistor Networks
View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow.
1V
0V
0.5V
0.5V
0.625V 0.375V
Graphs as Resistor Networks
View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow. Current flow measures strength of connection between endpoints. More short disjoint paths lead to higher flow.
Learning on Graphs [Zhu-‐Ghahramani-‐Lafferty ’03]
Infer values of a function at all vertices from known values at a few vertices.
Minimize
Subject to known values
X
(a,b)2E
(x(a)� x(b))2
0
1
Learning on Graphs [Zhu-‐Ghahramani-‐Lafferty ’03]
Infer values of a function at all vertices from known values at a few vertices.
Minimize
Subject to known values
X
(a,b)2E
(x(a)� x(b))2
0
1 0.5
0.5
0.625 0.375
The Laplacian quadratic form X
(a,b)2E
(x(a)� x(b))2
x
TLx =
X
(a,b)2E
(x(a)� x(b))2
The Laplacian matrix of a graph
There is a symmetric matrix L so that
Weighted Graphs
Edge assigned a non-negative real weight measuring strength of connection spring constant 1/resistance
wa,b 2 R
x
TLx =
X
(a,b)2E
wa,b(x(a)� x(b))2
(a, b)
The Laplacian matrix of a graph
x
TLx =
X
(a,b)2E
wa,b(x(a)� x(b))2
1! 2!
3!4!
The Laplacian matrix of a graph
To minimize subject to boundary constraints, set derivative to zero. Solve equation of form
Lx = b
x
TLx =
X
(a,b)2E
wa,b(x(a)� x(b))2
Can solve Laplacian equations absurdly quickly
Lx = b
Asymptotically best algorithm (today) takes time
O(mplog n log(1/✏))
Where m is number of non-zeros and n is dimension (Cohen-Kyng-Pachocki-Peng-Rao ’14)
Can solve Laplacian equations absurdly quickly
Lx = b
Asymptotically best algorithm (today) takes time
O(mplog n log(1/✏))
Where m is number of non-zeros and n is dimension (Cohen-Kyng-Pachocki-Peng-Rao ’14)
Good code: LAMG (lean algebraic multigrid) – Livne-Brandt CMG (combinatorial multigrid) – Koutis
Measuring boundaries of sets
Boundary: edges leaving a set
S S
Measuring boundaries of sets
Boundary: edges leaving a set
S S
Measuring boundaries of sets
Boundary: edges leaving a set
S
0 0
0 0
0 0
1
1 0
1 1
1 1 1
0
0 0
1
S
Characteristic Vector of S:
x(a) =
(1 a in S
0 a not in S
Measuring boundaries of sets
Boundary: edges leaving a set
S
0 0
0 0
0 0
1
1 0
1 1
1 1 1
0
0 0
1
S
Characteristic Vector of S:
x(a) =
(1 a in S
0 a not in S
x
TLx =
X
(a,b)2E
(x(a)� x(b))
2= |boundary(S)|
Eigenvalues and Eigenvectors A n-by-n symmetric matrix has n real eigenvalues and eigenvectors such that
�1 � �2 · · · � �n
v1, ..., vn
Lvi = �ivi
Spectral Graph Theory A n-by-n symmetric matrix has n real eigenvalues and eigenvectors such that
�1 � �2 · · · � �n
v1, ..., vn
These eigenvalues and eigenvectors tell us a lot about a graph!
Lvi = �ivi
Spectral Graph Drawing [Hall ‘70]
31 2
4
56 7
8 9
Arbitrary Drawing
Spectral Graph Drawing Plot vertex at draw edges as straight lines
[Hall ‘70]
(v2(a), v3(a))a
31 2
4
56 7
8 9
12
4
5
6
9
3
8
7Arbitrary Drawing
Spectral Drawing
A Graph
Drawing of the graph using v2, v3
Plot vertex at a (v2(a), v3(a))
The Airfoil Graph, original coordinates
The Airfoil Graph, spectral coordinates
The Airfoil Graph, spectral coordinates
Spectral drawing of Streets in Rome
Spectral drawing of Erdos graph: edge between co-authors of papers
Dodecahedron
Best embedded by first three eigenvectors
Graph Partitioning
Spectral Graph Partitioning
for some
[Donath-Hoffman ‘72, Barnes ‘82, Hagen-Kahng ‘92]
S = {a : v2(a) � t} t
Spectral Graph Partitioning
Performance guarantees come from Cheeger’s Inequality
[Donath-Hoffman ‘72, Barnes ‘82, Hagen-Kahng ‘92]
Spectral Image Segmentation (Shi-Malik ‘00)
200 400 600 800 1000 1200
100
200
300
400
500
600
700
800
900
Spectral Image Segmentation (Shi-Malik ‘00)
2 4 6 8 10 12 14 16
2
4
6
8
10
12
Spectral Image Segmentation (Shi-Malik ‘00)
2 4 6 8 10 12 14 16
2
4
6
8
10
12
Spectral Image Segmentation (Shi-Malik ‘00)
2 4 6 8 10 12 14 16
2
4
6
8
10
12
Spectral Image Segmentation (Shi-Malik ‘00)
2 4 6 8 10 12 14 16
2
4
6
8
10
12
edge weight
The second eigenvector
50 100 150 200 250 300
50
100
150
200
Second eigenvector cut
50 100 150 200 250 300
50
100
150
200
Third Eigenvector
50 100 150 200 250 300
50
100
150
200
50 100 150 200 250 300
50
100
150
200
Fourth Eigenvector
50 100 150 200 250 300
50
100
150
200
50 100 150 200 250 300
50
100
150
200
Laplacian Linear Systems Lx = b
Algorithms
Solve in Eme where = number of non-‐zeros entries of L
O(m log
c m)
m
(S-‐Teng ’04, ‘14)
Laplacian Linear Systems Lx = b
Algorithms
Solve in Eme where = number of non-‐zeros entries of L Emes for -‐approximate soluEon.
O(m log
c m)
m
log(1/�) ✏
��x� L�1b��L �
��L�1b��L
(S-‐Teng ’04, ‘14)
Laplacian Linear Systems Lx = b
Algorithms
Solve in Eme where = number of non-‐zeros entries of L Emes for -‐approximate soluEon.
O(m log
c m)
m
log(1/�) ✏
��x� L�1b��L �
��L�1b��L
kxkL =px
TLx
(S-‐Teng ’04, ‘14)
Algorithms
Two main ingredients: Graph Sparsifiers
Low Stretch Spanning Trees
(S-‐Teng ’04, ‘14)
Approximating Graphs
for all
A graph H is an -approximation of G if ✏
1
1 + � xTLHx
xTLGx 1 + �
x
Approximating Graphs
for all x
A very strong notion of approximation Preserves all electrical and spectral properties Preserves boundaries of every set
A graph H is an -approximation of G if ✏
1
1 + � xTLHx
xTLGx 1 + �
Sparsifying Graphs
for all 1
1 + � xTLHx
xTLGx 1 + �
x
(Batson-S-Srivastava ’09) Every graph G has an -approximation H with edges
A graph H is an -approximation of G if
n(2 + ✏)2/✏2
✏
✏
for all 1
1 + � xTLHx
xTLGx 1 + �
x
(Batson-S-Srivastava ’09) Every graph G has an -approximation H with edges (S-Srivastava ‘08) Can quickly find one with edges by sampling according to effective resistances
A graph H is an -approximation of G if
n(2 + ✏)2/✏2
Sparsifying Graphs ✏
✏
Cn log n/✏2
If , kAk < 1 (I �A)�1 = I +A+A2 +A3 + · · ·
=Y
i�0
(I +A2i)
Solving Equations (S-‐Peng ‘14)
If , kAk < 1 (I �A)�1 = I +A+A2 +A3 + · · ·
=Y
i�0
(I +A2i)
If , kAk < 1 A2i ! 0
⇡kY
i=0
(I +A2i)
Sparsify
Solving Equations (S-‐Peng ‘14)
I +A2i
Spanning trees of graphs
Connected No cycles Have a unique path between every two verEces
Connected No cycles Have a unique path between every two verEces
Spanning trees of graphs
The stretch of a spanning tree
stT (G) =X
(a,b)2E
path-lengthT (a, b)
(Alon-‐Karp-‐Peleg-‐West ’91)
path-‐len 5
stT (G) =X
(a,b)2E
path-lengthT (a, b)
The stretch of a spanning tree
path-‐len 3
stT (G) =X
(a,b)2E
path-lengthT (a, b)
The stretch of a spanning tree
path-‐len 1
stT (G) =X
(a,b)2E
path-lengthT (a, b)
The stretch of a spanning tree
stT (G) =X
(a,b)2E
path-lengthT (a, b)
For every G there is a T with stT (G) O(m log n log log n)
The stretch of a spanning tree
(Elkin-‐Emek-‐S-‐Teng ‘04, Abraham-‐Bartal-‐Neiman ’08, Abraham-‐Neiman ‘12)
stT (G) =X
(a,b)2E
path-lengthT (a, b)
For every G there is a T with stT (G) O(m log n log log n)
And, we can compute it in Eme O(m log n log log n)
(Elkin-‐Emek-‐S-‐Teng ‘04, Abraham-‐Bartal-‐Neiman ’08, Abraham-‐Neiman ‘12)
The stretch of a spanning tree
Kelner-‐Orecchia-‐Sidford-‐Zhu ‘13
Want the flow saEsfying demands that minimizes
Kirchoff: is the flow saEsfying demands such that
for all cycles C in the graph
X
(a,b)2C
fa,bra,b = 0,
X
(a,b)2E
f2a,bra,b
Kelner-‐Orecchia-‐Sidford-‐Zhu ‘13
Push flow around a cycle unEl it saEsfies KCL. This decreases the energy. To decide which cycles, choose a low-‐stretch tree. Each non-‐tree edge determines a cycle.
X
(a,b)2C
fa,bra,b = 0,
Kelner-‐Orecchia-‐Sidford-‐Zhu ‘13
Choose off-‐tree edge with prob
Push flow around that cycle to minimize energy and saEsfy Kirchoff.
With right data structure, each operaEon takes logarithmic Eme.
Solve in Eme O(m log
2 m log logm log ✏�1)
pa,b ⇠ path-lengthT (a, b)
1.0
1.0
1.0
1.0 1.0
1.0 1.0
1.0
0 0
Kelner-‐Orecchia-‐Sidford-‐Zhu ‘13
3.0 3.0
Start with a simple flow
1.0
1.0
1.0
1.0 1.0
1.0 1.0
1.0
0 0
Kelner-‐Orecchia-‐Sidford-‐Zhu ‘13
3.0 3.0
Minimize energy on a cycle
0.8
0.8
0.8
1.2 1.2
1.0 1.0
1.0
0 0
Kelner-‐Orecchia-‐Sidford-‐Zhu ‘13
3.0 3.0
Minimize energy on a cycle and saEsfy Kirchoff
0.8
0.8
0.8
1.2 1.13
1.0 1.07
1.0
0 0.07
Kelner-‐Orecchia-‐Sidford-‐Zhu ‘13
3.0 3.0
Minimize energy on another cycle
0.8
0.8
0.8
1.13 1.13
1.07 1.07
1.0
0.07 0.07
Kelner-‐Orecchia-‐Sidford-‐Zhu ‘13
3.0 3.0
Minimize energy on another cycle
0.8
0.8
0.8
1.13 1.13
1.07 1.07
0.71
0.36 0.36
Kelner-‐Orecchia-‐Sidford-‐Zhu ‘13
3.0 3.0
Minimize energy on another cycle
0.83
0.83
0.83
1.24 1.24
0.93 0.93
0.62
0.31 0.31
Kelner-‐Orecchia-‐Sidford-‐Zhu ‘13
3.0 3.0
Number of iteraEons is proporEonal to the stretch
Fast Laplacian Solvers
Asymptotically best algorithm (today) takes time
O(mplog n log(1/✏))
(Cohen-Kyng-Pachocki-Peng-Rao ’14)
Good code: LAMG (lean algebraic multigrid) – Livne-Brandt CMG (combinatorial multigrid) – Koutis
A powerful computational primitive!
My web page on: Laplacian linear equations, sparsification, local graph clustering, low-stretch spanning trees, and so on.
My class notes from “Graphs and Networks” and “Spectral Graph Theory”
Lx = b, by Nisheeth Vishnoi
To learn more
top related