![Page 1: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/1.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Spectral Graph Theory,Linear Solvers, and
Applications
Gary Miller
Carnegie Mellon Universityjoiny work with Yiannis Koutis and David Tolliver
Theory and Practice of Computational LearningJune 9, 2009
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 2: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/2.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Linear Systems
3x 2y −z = 32x −5y 4z = 7−x 1/2y 2z = 2
Fundamental ConstraintSystem
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 3: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/3.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Matrix Form
3 2 −12 −5 4
−1 1/2 2
xyz
=
372
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 4: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/4.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Solving the General Case
• Dense Case: O(n3) or O(n2.81) Strassen.
• Sparse Case: Still O(n2.81).
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 5: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/5.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Solving the General Case
• Dense Case: O(n3) or O(n2.81) Strassen.• Sparse Case: Still O(n2.81).
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 6: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/6.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
An Easy Case
• Upper and Lower Triangular Systems
• O(m) time where m = number of nonzerosentries.
• Goal: Find more easy cases.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 7: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/7.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
An Easy Case
• Upper and Lower Triangular Systems• O(m) time where m = number of nonzeros
entries.
• Goal: Find more easy cases.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 8: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/8.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
An Easy Case
• Upper and Lower Triangular Systems• O(m) time where m = number of nonzeros
entries.• Goal: Find more easy cases.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 9: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/9.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Symmetric Matrices
• Assume A is symmetric, A = AT .
• Assume A is positive definite, xTAx > 0 forx 6= 0
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 10: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/10.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Symmetric Matrices
• Assume A is symmetric, A = AT .• Assume A is positive definite, xTAx > 0 forx 6= 0
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 11: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/11.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Direct MethodsGaussian Elimination Matrices
• Goal: algorithms that minimize work andspace.
• Trick: View nonzero entries as an undirectedgraph and view pivoting as a graphoperation.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 12: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/12.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Direct MethodsGaussian Elimination Matrices
• Goal: algorithms that minimize work andspace.
• Trick: View nonzero entries as an undirectedgraph and view pivoting as a graphoperation.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 13: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/13.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Pivoting
Viewing pivoting as a graph operation.
• Let G = (V, E) be a graph and v a vertex.
• PIV OT (v) :• Make a clique out of neighbors of v.• Remove v.
• Fill: New edges formed.• Work: All edges “touched”.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 14: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/14.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Pivoting
Viewing pivoting as a graph operation.
• Let G = (V, E) be a graph and v a vertex.• PIV OT (v) :
• Make a clique out of neighbors of v.• Remove v.
• Fill: New edges formed.• Work: All edges “touched”.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 15: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/15.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Pivoting
Viewing pivoting as a graph operation.
• Let G = (V, E) be a graph and v a vertex.• PIV OT (v) :
• Make a clique out of neighbors of v.• Remove v.
• Fill: New edges formed.
• Work: All edges “touched”.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 16: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/16.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Pivoting
Viewing pivoting as a graph operation.
• Let G = (V, E) be a graph and v a vertex.• PIV OT (v) :
• Make a clique out of neighbors of v.• Remove v.
• Fill: New edges formed.• Work: All edges “touched”.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 17: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/17.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Good Pivot Strategies
1970s and 1980s• Planar systems: O(n3/2) work and O(n log n)
fill/space.
• 3D Systems: O(n2) work and O(n3/2)fill/space.
• O(n3/2) space is a problem for ML sizeproblems.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 18: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/18.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Good Pivot Strategies
1970s and 1980s• Planar systems: O(n3/2) work and O(n log n)
fill/space.• 3D Systems: O(n2) work and O(n3/2)
fill/space.
• O(n3/2) space is a problem for ML sizeproblems.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 19: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/19.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Good Pivot Strategies
1970s and 1980s• Planar systems: O(n3/2) work and O(n log n)
fill/space.• 3D Systems: O(n2) work and O(n3/2)
fill/space.• O(n3/2) space is a problem for ML size
problems.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 20: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/20.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Pure Iterative Methods
Solving Ax = b.• Basic method: x(i+1) = (I − A)x(i) + b
• Convergence/Rate is determined by ||I −A||.• Accelerated Methods: Chebyshev Iteration,
Conjugate Gradient.• CG is still too slow.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 21: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/21.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Pure Iterative Methods
Solving Ax = b.• Basic method: x(i+1) = (I − A)x(i) + b
• Convergence/Rate is determined by ||I −A||.
• Accelerated Methods: Chebyshev Iteration,Conjugate Gradient.
• CG is still too slow.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 22: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/22.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Pure Iterative Methods
Solving Ax = b.• Basic method: x(i+1) = (I − A)x(i) + b
• Convergence/Rate is determined by ||I −A||.• Accelerated Methods: Chebyshev Iteration,
Conjugate Gradient.
• CG is still too slow.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 23: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/23.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Pure Iterative Methods
Solving Ax = b.• Basic method: x(i+1) = (I − A)x(i) + b
• Convergence/Rate is determined by ||I −A||.• Accelerated Methods: Chebyshev Iteration,
Conjugate Gradient.• CG is still too slow.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 24: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/24.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Preconditioned Iterative Methods
Solving B−1Ax = B−1b = b′.• Basic method: x(i+1) = (I −B−1A)x(i) + b′
• Computing the term z = B−1Ax(i)
• y = Ax(i) Forward Multiply• Bz = y Solve the preconditioner system
• Goal: Minimize the number of iteration whileminimizing the cost of the solve.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 25: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/25.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Preconditioned Iterative Methods
Solving B−1Ax = B−1b = b′.• Basic method: x(i+1) = (I −B−1A)x(i) + b′
• Computing the term z = B−1Ax(i)
• y = Ax(i) Forward Multiply• Bz = y Solve the preconditioner system
• Goal: Minimize the number of iteration whileminimizing the cost of the solve.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 26: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/26.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Preconditioned Iterative Methods
Solving B−1Ax = B−1b = b′.• Basic method: x(i+1) = (I −B−1A)x(i) + b′
• Computing the term z = B−1Ax(i)
• y = Ax(i) Forward Multiply• Bz = y Solve the preconditioner system
• Goal: Minimize the number of iteration whileminimizing the cost of the solve.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 27: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/27.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Classic Preconditioners
• Jacobi: B = Diagonal(A).
• Gauss-Seidel: B = UpperTriangular(A).• SSOR: B = (L + 1
ωD) 1ωD(L + 1
ωD)
• Still too slow and unreliable.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 28: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/28.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Classic Preconditioners
• Jacobi: B = Diagonal(A).• Gauss-Seidel: B = UpperTriangular(A).
• SSOR: B = (L + 1ωD) 1
ωD(L + 1ωD)
• Still too slow and unreliable.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 29: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/29.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Classic Preconditioners
• Jacobi: B = Diagonal(A).• Gauss-Seidel: B = UpperTriangular(A).• SSOR: B = (L + 1
ωD) 1ωD(L + 1
ωD)
• Still too slow and unreliable.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 30: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/30.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Classic Preconditioners
• Jacobi: B = Diagonal(A).• Gauss-Seidel: B = UpperTriangular(A).• SSOR: B = (L + 1
ωD) 1ωD(L + 1
ωD)
• Still too slow and unreliable.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 31: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/31.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Graph Laplacian• G = (V,E, w) weighted undirected graph, wij > 0.
• Weighted incidence matrix:
Aij ={
wij if eij ∈ E0 otherwise
• Degree of vi: di =∑
j wij
•
D =
d1 0. . .
0 dn
• Laplacian: L = D −A
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 32: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/32.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Graph Laplacian• G = (V,E, w) weighted undirected graph, wij > 0.• Weighted incidence matrix:
Aij ={
wij if eij ∈ E0 otherwise
• Degree of vi: di =∑
j wij
•
D =
d1 0. . .
0 dn
• Laplacian: L = D −A
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 33: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/33.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Graph Laplacian• G = (V,E, w) weighted undirected graph, wij > 0.• Weighted incidence matrix:
Aij ={
wij if eij ∈ E0 otherwise
• Degree of vi: di =∑
j wij
•
D =
d1 0. . .
0 dn
• Laplacian: L = D −A
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 34: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/34.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Graph Laplacian• G = (V,E, w) weighted undirected graph, wij > 0.• Weighted incidence matrix:
Aij ={
wij if eij ∈ E0 otherwise
• Degree of vi: di =∑
j wij
•
D =
d1 0. . .
0 dn
• Laplacian: L = D −A
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 35: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/35.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Graph Laplacian• G = (V,E, w) weighted undirected graph, wij > 0.• Weighted incidence matrix:
Aij ={
wij if eij ∈ E0 otherwise
• Degree of vi: di =∑
j wij
•
D =
d1 0. . .
0 dn
• Laplacian: L = D −A
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 36: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/36.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Classic Applications of the Laplacian
• View each edge a conductor with conductance wij .
• Let V be a column vector of voltages• If LV = c the c residual current needed to maintain the
given voltages.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 37: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/37.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Classic Applications of the Laplacian
• View each edge a conductor with conductance wij .• Let V be a column vector of voltages
• If LV = c the c residual current needed to maintain thegiven voltages.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 38: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/38.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Classic Applications of the Laplacian
• View each edge a conductor with conductance wij .• Let V be a column vector of voltages• If LV = c the c residual current needed to maintain the
given voltages.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 39: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/39.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Graph Laplacian’s and the HeatEquations
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 40: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/40.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Graph Laplacian’s and RandomWalks
Transition Matrix: D−1LG
Fundamental Eigenvectors: O(n + m) (Spielman Teng)Trick: Inverse Powering only requires O(log n) iterations.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 41: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/41.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Laplacian’s and Spring Mass Systems
• G = (V, E, w) weighted graph and wij isviewed a spring constant.
• M is a diagonal matrix of mass constants• Fact: Modes of vibration of Spring-Mass
system G, M are:Eigen-pairs of LGx = λMx.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 42: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/42.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Laplacian’s and Spring Mass Systems
• G = (V, E, w) weighted graph and wij isviewed a spring constant.
• M is a diagonal matrix of mass constants
• Fact: Modes of vibration of Spring-Masssystem G, M are:Eigen-pairs of LGx = λMx.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 43: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/43.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Laplacian’s and Spring Mass Systems
• G = (V, E, w) weighted graph and wij isviewed a spring constant.
• M is a diagonal matrix of mass constants• Fact: Modes of vibration of Spring-Mass
system G, M are:Eigen-pairs of LGx = λMx.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 44: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/44.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Spring Mass System
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 45: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/45.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Graph Laplacian’s and MaximumFlow
Graph Maximum Flow: O((m + n)3/2) (Daitch Spielman)
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 46: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/46.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Graph LaplaciansApplications
Graph Laplacian’s and ConvexProgramming
Nonuniform TV Denoising: O((m + n)3/2) (Koutis M SinopTolliver)
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 47: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/47.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Spanning Tree Preconditioners• Vaidya ’93: Use Maximum Weight Spanning
Tree (MST) plus a few edges.
b
c
a
d
e f g
h
j
i
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 48: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/48.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Spanning Tree Preconditioners• Advantages: Easy to find and Easy to solve
their systems.
b
c
a
d
e f g
h
j
i
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 49: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/49.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Spanning Tree Preconditioners• Problem: Small edge weights differences
can make MST bad.
b
c
a
d
e f g
h
j
i
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 50: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/50.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Low Stretch Spanning Trees• EEST ’05: Use low stretch spanning trees
plus a few edges.
b
c
a
d
e f g
h
j
i
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 51: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/51.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Low Stretch Spanning Trees• Advantages: Better condition number.
b
c
a
d
e f g
h
j
i
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 52: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/52.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Low Stretch Spanning Trees• Problem: Super linear time to find.
b
c
a
d
e f g
h
j
i
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 53: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/53.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Low Stretch Spanning Trees• Richter-M ’04: There are no good spanning
trees even for square mesh.
b
c
a
d
e f g
h
j
i
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 54: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/54.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Steiner Tree Preconditioners
• Gremban-M ’94: Use Steiner trees.
b
c
a
d
e f g
h
j
i
g
a b
c d
h i je
f
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 55: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/55.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Steiner Tree Preconditioners
• Advantages: Better condition number forgraphs like square mesh.
b
c
a
d
e f g
h
j
i
g
a b
c d
h i je
f
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 56: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/56.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Steiner Tree Preconditioners
• Advantages: Good experimental results.
b
c
a
d
e f g
h
j
i
g
a b
c d
h i je
f
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 57: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/57.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Steiner Tree Preconditioners
• Problem: Hard to construct in general andanalyze.
b
c
a
d
e f g
h
j
i
g
a b
c d
h i je
f
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 58: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/58.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Steiner Forest Preconditioners
• Koutis-M ’07: Use Steiner Forest.
b
c
a
d
e f g
h
j
i
g
a b
c d
h i je
f
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 59: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/59.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Steiner Forest Preconditioners
• Advantages: Easy to find and works wellwith recursive solvers.
b
c
a
d
e f g
h
j
i
g
a b
c d
h i je
f
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 60: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/60.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Steiner Forest Preconditioners
• Advantages: Good experimental results.
b
c
a
d
e f g
h
j
i
g
a b
c d
h i je
f
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 61: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/61.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Steiner Forest Preconditioners
• Problem: Analysis only for planar systems.
b
c
a
d
e f g
h
j
i
g
a b
c d
h i je
f
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 62: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/62.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Recursive Methods
• Vaidya ’93: Idea: Tree have a lot of degree1-2 degree nodes. Pivot on these nodes andthen find a preconditioner for this graph.
d h i jf ga b ec
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 63: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/63.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Recursive Methods
• Vaidya ’93: Idea: Tree have a lot of degree1-2 degree nodes. Pivot on these nodes andthen find a preconditioner for this graph.
The reduced graph.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 64: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/64.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
The Error
• Recurrence: u(i+1) = Gu(i) + b, for usG = I − A.
• Error: e(i) = u(i) − u, where u = Gu + b
• Fact: e(i) = Gie(0)
• We need: limi→∞ Gi = 0
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 65: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/65.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
The Error
• Recurrence: u(i+1) = Gu(i) + b, for usG = I − A.
• Error: e(i) = u(i) − u, where u = Gu + b
• Fact: e(i) = Gie(0)
• We need: limi→∞ Gi = 0
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 66: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/66.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
The Error
• Recurrence: u(i+1) = Gu(i) + b, for usG = I − A.
• Error: e(i) = u(i) − u, where u = Gu + b
• Fact: e(i) = Gie(0)
• We need: limi→∞ Gi = 0
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 67: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/67.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
The Error
• Recurrence: u(i+1) = Gu(i) + b, for usG = I − A.
• Error: e(i) = u(i) − u, where u = Gu + b
• Fact: e(i) = Gie(0)
• We need: limi→∞ Gi = 0
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 68: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/68.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Condition Number
• Ax = λx
• DEF: λ Eigenvalue and x Eigenvector.• Λ(A) = {0 ≤ λ1 ≤ · · ·λn}.• Condition Number: κ(A) = λn/λ1
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 69: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/69.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Condition Number
• Ax = λx
• DEF: λ Eigenvalue and x Eigenvector.
• Λ(A) = {0 ≤ λ1 ≤ · · ·λn}.• Condition Number: κ(A) = λn/λ1
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 70: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/70.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Condition Number
• Ax = λx
• DEF: λ Eigenvalue and x Eigenvector.• Λ(A) = {0 ≤ λ1 ≤ · · ·λn}.
• Condition Number: κ(A) = λn/λ1
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 71: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/71.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Condition Number
• Ax = λx
• DEF: λ Eigenvalue and x Eigenvector.• Λ(A) = {0 ≤ λ1 ≤ · · ·λn}.• Condition Number: κ(A) = λn/λ1
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 72: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/72.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Convergence Rates
• Basic Method: Convergence Rate= O(1/κ(A))
• Conjugate Gradient: O(1/√
κ(A))
• Conjugate Gradient: ≈ 1/diameter(A).
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 73: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/73.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Convergence Rates
• Basic Method: Convergence Rate= O(1/κ(A))
• Conjugate Gradient: O(1/√
κ(A))
• Conjugate Gradient: ≈ 1/diameter(A).
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 74: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/74.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Convergence Rates
• Basic Method: Convergence Rate= O(1/κ(A))
• Conjugate Gradient: O(1/√
κ(A))
• Conjugate Gradient: ≈ 1/diameter(A).
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 75: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/75.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Generalized Condition Number
• Goal: Bound condition number of B−1A
• Note: B−1Ax = λx iff Ax = λBx
• DEF: λ is a generalized eigenvalue.• Condition Number: κ(B−1A) = λn/λ1
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 76: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/76.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Generalized Condition Number
• Goal: Bound condition number of B−1A
• Note: B−1Ax = λx iff Ax = λBx
• DEF: λ is a generalized eigenvalue.• Condition Number: κ(B−1A) = λn/λ1
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 77: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/77.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Generalized Condition Number
• Goal: Bound condition number of B−1A
• Note: B−1Ax = λx iff Ax = λBx
• DEF: λ is a generalized eigenvalue.
• Condition Number: κ(B−1A) = λn/λ1
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 78: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/78.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Generalized Condition Number
• Goal: Bound condition number of B−1A
• Note: B−1Ax = λx iff Ax = λBx
• DEF: λ is a generalized eigenvalue.• Condition Number: κ(B−1A) = λn/λ1
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 79: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/79.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
The Support
• Positive Semi-Definite: ∀x xTAx ≥ 0
• Support of A by B:σ(A/B) = min{τ : τB − A is PSD}
• Fact: κ(A, B) = σ(A, B) · σ(B, A).
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 80: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/80.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
The Support
• Positive Semi-Definite: ∀x xTAx ≥ 0
• Support of A by B:σ(A/B) = min{τ : τB − A is PSD}
• Fact: κ(A, B) = σ(A, B) · σ(B, A).
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 81: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/81.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
The Support
• Positive Semi-Definite: ∀x xTAx ≥ 0
• Support of A by B:σ(A/B) = min{τ : τB − A is PSD}
• Fact: κ(A, B) = σ(A, B) · σ(B, A).
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 82: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/82.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Estimating Support
• G and H are graphs and V (G) = V (H)
• Path Embedding: φ : E(G) then paths(H)s.t. φ(eij) = Vi · · ·Vj.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 83: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/83.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
Estimating Support ExampleEG: G ≡ K4 and H ≡ 4-cycle
Congestion ≡ 3 and Dilation ≡ 2Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 84: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/84.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
History of Planar Solvers
• 1950’s O(n2) (Conjugate Gradient)
• 1970’s O(n1.5) (Nested Dissection) (LRT)• 1990’s O(n1.2) (Combinatorial
Preconditioners) (Vaidya)• 2000’s O(n log2 n) (Low stretch spanning
trees) (ST)• 2006’s O(n) (separator based
preconditioners) (KM)
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 85: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/85.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
History of Planar Solvers
• 1950’s O(n2) (Conjugate Gradient)• 1970’s O(n1.5) (Nested Dissection) (LRT)
• 1990’s O(n1.2) (CombinatorialPreconditioners) (Vaidya)
• 2000’s O(n log2 n) (Low stretch spanningtrees) (ST)
• 2006’s O(n) (separator basedpreconditioners) (KM)
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 86: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/86.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
History of Planar Solvers
• 1950’s O(n2) (Conjugate Gradient)• 1970’s O(n1.5) (Nested Dissection) (LRT)• 1990’s O(n1.2) (Combinatorial
Preconditioners) (Vaidya)
• 2000’s O(n log2 n) (Low stretch spanningtrees) (ST)
• 2006’s O(n) (separator basedpreconditioners) (KM)
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 87: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/87.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
History of Planar Solvers
• 1950’s O(n2) (Conjugate Gradient)• 1970’s O(n1.5) (Nested Dissection) (LRT)• 1990’s O(n1.2) (Combinatorial
Preconditioners) (Vaidya)• 2000’s O(n log2 n) (Low stretch spanning
trees) (ST)
• 2006’s O(n) (separator basedpreconditioners) (KM)
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 88: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/88.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
History of Planar Solvers
• 1950’s O(n2) (Conjugate Gradient)• 1970’s O(n1.5) (Nested Dissection) (LRT)• 1990’s O(n1.2) (Combinatorial
Preconditioners) (Vaidya)• 2000’s O(n log2 n) (Low stretch spanning
trees) (ST)• 2006’s O(n) (separator based
preconditioners) (KM)
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 89: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/89.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
General Laplacian Solver
O(n + m) (Spielman Teng)
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 90: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/90.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.80
2
4
6
8
10
12
14
16
18
20
Number of Pixels (in millions)
Ru
nn
ing
Tim
e (s
ecs)
Two dimensional images
Our solverMATLAB’s direct solver
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 91: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/91.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Combinatorial PreconditionersRecursive Preconditioned MethodsAnalysisRun Times
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90
2
4
6
8
10
12
Number of Pixels (in millions)
Ru
nn
ing
Tim
e (s
eco
nd
s)Three dimensional images
Our solverdirect solver
Out of memory
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 92: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/92.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Symmetric Diagonally DominateSystems
• Goal: Show how to solve SDD using regularLaplacians.
• Let G = (V, E, w) such that ij ∈ E:wij 6= 0• Weighted incidence matrix: A.• Degree of vi: di =
∑j |wij|
• Generalized Laplacian: L = D − A• Note: Every SDD is a Generalized
Laplacian.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 93: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/93.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Symmetric Diagonally DominateSystems
• Goal: Show how to solve SDD using regularLaplacians.
• Let G = (V, E, w) such that ij ∈ E:wij 6= 0
• Weighted incidence matrix: A.• Degree of vi: di =
∑j |wij|
• Generalized Laplacian: L = D − A• Note: Every SDD is a Generalized
Laplacian.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 94: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/94.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Symmetric Diagonally DominateSystems
• Goal: Show how to solve SDD using regularLaplacians.
• Let G = (V, E, w) such that ij ∈ E:wij 6= 0• Weighted incidence matrix: A.
• Degree of vi: di =∑
j |wij|• Generalized Laplacian: L = D − A• Note: Every SDD is a Generalized
Laplacian.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 95: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/95.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Symmetric Diagonally DominateSystems
• Goal: Show how to solve SDD using regularLaplacians.
• Let G = (V, E, w) such that ij ∈ E:wij 6= 0• Weighted incidence matrix: A.• Degree of vi: di =
∑j |wij|
• Generalized Laplacian: L = D − A• Note: Every SDD is a Generalized
Laplacian.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 96: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/96.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Symmetric Diagonally DominateSystems
• Goal: Show how to solve SDD using regularLaplacians.
• Let G = (V, E, w) such that ij ∈ E:wij 6= 0• Weighted incidence matrix: A.• Degree of vi: di =
∑j |wij|
• Generalized Laplacian: L = D − A
• Note: Every SDD is a GeneralizedLaplacian.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 97: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/97.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Symmetric Diagonally DominateSystems
• Goal: Show how to solve SDD using regularLaplacians.
• Let G = (V, E, w) such that ij ∈ E:wij 6= 0• Weighted incidence matrix: A.• Degree of vi: di =
∑j |wij|
• Generalized Laplacian: L = D − A• Note: Every SDD is a Generalized
Laplacian.Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 98: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/98.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Basic Properties: GeneralizedLaplacian
• xTLx =∑wij>0 wij(xi − xj)
2 −∑
wij<0 wij(xi + xj)2
• Thus L is positive semidefinite.• Claim: Rank = n− 1 if G is connected.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 99: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/99.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Basic Properties: GeneralizedLaplacian
• xTLx =∑wij>0 wij(xi − xj)
2 −∑
wij<0 wij(xi + xj)2
• Thus L is positive semidefinite.
• Claim: Rank = n− 1 if G is connected.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 100: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/100.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Basic Properties: GeneralizedLaplacian
• xTLx =∑wij>0 wij(xi − xj)
2 −∑
wij<0 wij(xi + xj)2
• Thus L is positive semidefinite.• Claim: Rank = n− 1 if G is connected.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 101: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/101.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Solving Generalized Laplacian byChange of Variables
• First Idea: Find a change of variables it get aregular Laplacian.
• Note: multiplying the ith column and row by−1 preserves Laplacian.
• This is just Flipping xi and bi to −xi and −bi.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 102: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/102.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Solving Generalized Laplacian byChange of Variables
• First Idea: Find a change of variables it get aregular Laplacian.
• Note: multiplying the ith column and row by−1 preserves Laplacian.
• This is just Flipping xi and bi to −xi and −bi.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 103: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/103.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Solving Generalized Laplacian byChange of Variables
• First Idea: Find a change of variables it get aregular Laplacian.
• Note: multiplying the ith column and row by−1 preserves Laplacian.
• This is just Flipping xi and bi to −xi and −bi.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 104: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/104.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Generalized Laplacians Example
+1
V1
−1
V2−1V3 2 +1 −1
+1 2 +1−1 +1 2
x1
x2
x3
=
b1
b2
b3
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 105: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/105.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Change of Variables for Laplacians
+1
V1
−1
V2−1V3
+1−1
2 −1 +1−1 2 +1+1 +1 2
−x1
x2
x3
=
−b1
b2
b3
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 106: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/106.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Orientable Generalized Laplacians
• DEF: G = (V, E, w) is orientable is ∃sequence of flips s.t. w > 0.
• DEF: LG is orientable if G is.• Note: Orientability is linear testable, greedy.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 107: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/107.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Orientable Generalized Laplacians
• DEF: G = (V, E, w) is orientable is ∃sequence of flips s.t. w > 0.
• DEF: LG is orientable if G is.
• Note: Orientability is linear testable, greedy.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 108: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/108.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Orientable Generalized Laplacians
• DEF: G = (V, E, w) is orientable is ∃sequence of flips s.t. w > 0.
• DEF: LG is orientable if G is.• Note: Orientability is linear testable, greedy.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 109: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/109.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Orientable Generalized Laplacians
• Claim: If G is connected and not orientablethen L is SPD.
• Proof: Suppose xTLx = 0 and x 6= 0
• Pick a spanning tree T of G and orient it andflipping x.
• WLOG: x is the all-ones vector, a contra!,since G still has negative edges.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 110: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/110.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Orientable Generalized Laplacians
• Claim: If G is connected and not orientablethen L is SPD.
• Proof: Suppose xTLx = 0 and x 6= 0
• Pick a spanning tree T of G and orient it andflipping x.
• WLOG: x is the all-ones vector, a contra!,since G still has negative edges.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 111: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/111.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Orientable Generalized Laplacians
• Claim: If G is connected and not orientablethen L is SPD.
• Proof: Suppose xTLx = 0 and x 6= 0
• Pick a spanning tree T of G and orient it andflipping x.
• WLOG: x is the all-ones vector, a contra!,since G still has negative edges.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 112: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/112.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Orientable Generalized Laplacians
• Claim: If G is connected and not orientablethen L is SPD.
• Proof: Suppose xTLx = 0 and x 6= 0
• Pick a spanning tree T of G and orient it andflipping x.
• WLOG: x is the all-ones vector, a contra!,since G still has negative edges.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 113: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/113.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Two-Fold Covers• DEF: The 2-fold cover G = (V , W , w) ofG = (V, W, w) is:
• V = {V1, V1, . . . , Vn, Vn}• E: If wij > 0
add edges < Vi, Vj > and < Vi, Vj > withweight wij
• E: If wij < 0add edges < Vi, Vj > and < Vi, Vj > withweight −wij
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 114: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/114.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Two-Fold Covers• DEF: The 2-fold cover G = (V , W , w) ofG = (V, W, w) is:
• V = {V1, V1, . . . , Vn, Vn}
• E: If wij > 0add edges < Vi, Vj > and < Vi, Vj > withweight wij
• E: If wij < 0add edges < Vi, Vj > and < Vi, Vj > withweight −wij
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 115: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/115.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Two-Fold Covers• DEF: The 2-fold cover G = (V , W , w) ofG = (V, W, w) is:
• V = {V1, V1, . . . , Vn, Vn}• E: If wij > 0
add edges < Vi, Vj > and < Vi, Vj > withweight wij
• E: If wij < 0add edges < Vi, Vj > and < Vi, Vj > withweight −wij
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 116: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/116.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Two-Fold Covers• DEF: The 2-fold cover G = (V , W , w) ofG = (V, W, w) is:
• V = {V1, V1, . . . , Vn, Vn}• E: If wij > 0
add edges < Vi, Vj > and < Vi, Vj > withweight wij
• E: If wij < 0add edges < Vi, Vj > and < Vi, Vj > withweight −wij
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 117: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/117.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Example: Two-Fold Cover
+1
V1
−1
V2−1V3
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 118: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/118.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Two-Fold Covers and SolvingGeneralized Laplacians
• Let L = L(G) and L = G
• Note:
Lx = b then L
(x
−x
)=
(b
−b
)•
L
(xy
)=
(b
−b
)then L(x/2− y/2) = b
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 119: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/119.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Two-Fold Covers and SolvingGeneralized Laplacians
• Let L = L(G) and L = G• L is a regular Laplacian which we can solve
quickly.
• Note:
Lx = b then L
(x
−x
)=
(b
−b
)•
L
(xy
)=
(b
−b
)then L(x/2− y/2) = b
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 120: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/120.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Two-Fold Covers and SolvingGeneralized Laplacians
• Let L = L(G) and L = G
• Note:
Lx = b then L
(x
−x
)=
(b
−b
)
•
L
(xy
)=
(b
−b
)then L(x/2− y/2) = b
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 121: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/121.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Two-Fold Covers and SolvingGeneralized Laplacians
• Let L = L(G) and L = G
• Note:
Lx = b then L
(x
−x
)=
(b
−b
)•
L
(xy
)=
(b
−b
)then L(x/2− y/2) = b
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 122: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/122.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
SDD systems
• The 2-fold trick can be run without the factorof two in space and time.
• There should be uses of negative weights inrecommendation problems.
• Naive approach does not seem to work right.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 123: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/123.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
SDD systems
• The 2-fold trick can be run without the factorof two in space and time.
• There should be uses of negative weights inrecommendation problems.
• Naive approach does not seem to work right.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 124: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/124.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
SDD systems
• The 2-fold trick can be run without the factorof two in space and time.
• There should be uses of negative weights inrecommendation problems.
• Naive approach does not seem to work right.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 125: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/125.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Spectral Graph Partitioning• Idea: Pick a few low frequency eigenvectors.
• Use these vectors to embed the graph in Rd
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 126: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/126.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Spectral Graph Partitioning• Idea: Pick a few low frequency eigenvectors.• Use these vectors to embed the graph in Rd
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 127: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/127.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
The Blue Sky ProblemShi Malik applied to an image:
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 128: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/128.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
The Blue Sky ProblemShi Malik solution:
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 129: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/129.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
The Blue Sky ProblemSpectral Rounding applied to Image:
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 130: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/130.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Spectral RoundingEdge Reweighting
Algorithm:• Solve Lf = λ2Df .• Reweight graph edges getting L′ and D′.• Solve L′f = λ2D
′f
• Repeat while λ2 6= 0.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 131: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/131.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
SR: The reweighting scheme
• View D, f , and λ2 as a function of L
• Subject to Lf = λ2Df and fTDf = 1.• We get: ∂λ2
∂eij= (fi − fj)
2 − λ2(f2i + f 2
j )
• Take a “small” step in the direction of thegradient.
• If an edge goes negative set it to zero.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 132: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/132.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Medical Examples of SR
Breast Tumors
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 133: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/133.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
Medical Examples of SR
Breast Tumors
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications
![Page 134: Spectral Graph Theory, Linear Solvers, and Applications](https://reader030.vdocument.in/reader030/viewer/2022020706/61fc88be8d33c02b785e4a69/html5/thumbnails/134.jpg)
IntroductionGraph Based Methods
Iterative Methods for LaplaciansSymmetric Diagonally Dominate Systems
Spectral RoundingOpen Questions
• Find fast methods for any SPD system.• Find spectral methods that find better cut by
using more than one eigenvector.• Find solvers that work in the L2 norm.• A implementable solver with near linear time
guarantees.
Gary L. Miller Spectral Graph Theory, Linear Solvers, and Applications