an introduction to iterative projection methods eigenvalue problems luiza bondar the 23 rd of...
Post on 21-Dec-2015
216 views
TRANSCRIPT
![Page 1: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/1.jpg)
An introduction to iterative projection methods
Eigenvalue problems
Luiza Bondar
the 23rd of November -2005
4th Seminar
![Page 2: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/2.jpg)
Introduction (Erwin)
Perturbation analysis (Nico)
Direct (global) methods (Peter)
Introduction to projection methods (Luiza) (theoretical
background)
Krylov subspace methods 1 (Mark)
Krylov subspace methods 2 (Willem)
![Page 3: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/3.jpg)
Outline
• Introduction
• The power method
• Projection Methods
• Subspace iteration
• Summary
![Page 4: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/4.jpg)
Direct methods (Schur decomposition, QR iteration, Jacobi method,
method of Sturm sequences )
• compute all the eigenvalues and the corresponding eigenvectors
What if we DON’T need all the eigenvalues?
Example : compute the page rank of the www documents
Introduction
![Page 5: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/5.jpg)
WEB: a graph (pages are nodes links are edges )
Introduction
![Page 6: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/6.jpg)
Web graph: 1.4 bilion nodes (pages) 6.6 bilion edges (links)
page rank of page i : the probability that a surfer will visit the page i
The page rank is a dominant vector of a sparse 1.4 bilion X 1.4 bilionmatrix.
It makes little sense to compute all the eigenvectors.
page rank : vector with dimension N=1.4 bilion
Introduction
![Page 7: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/7.jpg)
The power method
• computes the dominant eigenvalue and an associated eigenvector
Some background
consider that A has p distinct eigenvalues.
1 2n
pM M M
dim i iM
n niP :
kki i i iA P P I D 0i i
iD semi-simple i 0iD
iis the algebraic multiplicity of i
iPis the projection onto iM
![Page 8: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/8.jpg)
The power method
consider that the dominant eigenvalue is unique and is semi-simple 1
initial vector such that
convergence ?NO YES
1 k
1 kx v
1 0P v 0
1Avk k 1
1k k
k v Av
0v
0
1v A vkk
k ( ) 1Avk compute an
dtake
![Page 9: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/9.jpg)
The power method
initial vector 0vn , 1 2
npM M M 0 0
1
v vp
ii
P
0
1v A vkk
k
kki i i iA P P I D use
11 0 0
2 1
1v v v
k pk
k i i i ikik
P P D P
then 1 01 0
1v v
vk PP
and 1k ( )1Av vk k k
0
convergence of each term in given by
Σ1
i
The power method is used by to compute the page rank.
![Page 10: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/10.jpg)
The power method
• the convergence of the method is given by
• the convergence might be very slow if are close from one another
• if the dominant eigenvalue is multiple but semi-simple, then the algorithm provides only one eigenvalue and a corresponding eigenvector
• does not converge if the dominant eigenvalue is complex and theoriginal matrix is real (2 eigenvalues with the same modulus)
2
1
1 2,
IMPROVEMENT : the shifted power method
LED TO : projection methods
![Page 11: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/11.jpg)
The power method
Shifted power method A A I x x
Example
• let be the dominant eigenvalue of a matrix that has an egenvalue • then the power method does not converge when applied to • but the power method converges for a shift (e.g. )
1 1 1i
A IA
A
Other variants of the power method
• inverse power method (iterates with )
• inverse power method with shift
-1A smallest eigenvalue
eigenvalue closest to the shift
![Page 12: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/12.jpg)
The power method
• inverse power method
A LU1 -1 -1
k-1v U L vkk
then converges to the smallest eigenvalue and converges to an
associated eigenvector k vk
• inverse power method with shift
-A I LU
1 1-
-1 -1 -1k-1 k-1v A I v U L vk
k k
then converges to and converges to an eigenvector associated with
k 1
vk
![Page 13: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/13.jpg)
The power method
• does not converge if the dominant eigenvalue is complex and the original matrix is real (2 eigenvalues with the same modulus)
But after a certain k
1v vk k,
IDEA: extract the vectors by performing a projection into the subspace
contains approximations to the complex par of eigenvectors
0 1
1 0A=
1 i
1
i
2 i
1
i
power method
1
1
1
1
vk 1vk
![Page 14: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/14.jpg)
Projection methods (Introduction)
1u vk
2 1u vk
ufind and such that Au u
1 1 2 2u u u
• impose 2 more constrains• one choice is to impose orthogonality conditions (Galerkin) i.e.,
1 , 2• introduce 2 degrees of freedom
1Au u u and
2Au u u
1u
2u
u
Auprojection method
![Page 15: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/15.jpg)
Projection methods (Introduction)
Generalizationn nA
, nK L
dim K=dim L=m
find and such that nu Au u
A projection technique seeks an approximate eigenpar and such that
Ku
K uuA ~~~
L uuA ~~~
• orthogonal projection
or• oblique projection
K: the right subspace, L: the left subspace
A way to construct K is Krylov subspace
(inspired by the power method)
10 0 0, , ,m mK v Av A v
![Page 16: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/16.jpg)
Projection methods (orthogonal)
Consider an orthonormal basis of K and
1 2, , , mv v v
1 2| | | ,mV v v v
Kuthe approximate can be written as u Vy
K uuA ~~~ , 0, Au u v K v , 0, 1, ,jv j m AVy vy
HV AVy y
n mV
:m HB V AV
i i
ix i iu Vx
• eigenvalue of then eigenvalue of
• eigenvector of then eigenvector of
mB
mB
A
A
Arnoldi’s method and the hermitian Lanczos algorithm are orthogonalprojection methods
![Page 17: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/17.jpg)
Projection methods (oblique)
L uuA ~~~ KuSearch for and such that
1 2, , , mv v v 1 2| | | ,mV v v vn mV
the approximate can be written as u Vy
1 2| | | ,mW w w wn mW 1 2, , , mw w w
orthonormal basis of K
orthonormal basis of L
and are such that (biorthogonal) V W HW V I
Ku
The condition leads to the approximate eigenvalue
problem
L uuA ~~~
HW AVy y
The nonhermitian Lanczos alghoritm is an oblique projection method.
![Page 18: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/18.jpg)
Projection methods (orthogonal)
How accurate can an orthogonal projection method be?
, uexact eigenpar
then 2 22
H Hk k kV AV I V u P A I P I P u
projection onto KkP
kP u
uK
u
ku-P u
![Page 19: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/19.jpg)
Projection methods (orthogonal)
Hermitian case
kP u
uK
u
2( )
sin 1 sin ,,
k kP A I P
d
kuu, u
AP u
2
2sin ,A uI u
![Page 20: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/20.jpg)
Subspace iteration
• generalization of the power method• start with an initial system of m vectors instead of
only one vector (power method)
• compute the matrix
0 1 2, , , mX x x x
0k
k X A X
If each of the m vectors is normalised in the same way as for the power method, then each of these vectors will converge to the SAMEeigenvector associated with the dominant eigenvalue (provided that )1 0, 1,Pxi i m
Note looses its linear independenceIDEA: restore the linear independence by performing aQR factorisation
0k
k X A X
![Page 21: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/21.jpg)
Subspace iteration
0 1 2, , , mX x x x
0 0 0X Q R :0 0X Q
-1X AXk k
-1 -1 1 -1H H HH X AX Q AQk k k k k
start with
QR factorize 0X take
compute
convergence ?X Q Rk k k
:X Qk k
recover the first m eigenvalues
and corresponding eigenvectors
of A from Hk
NO YES
![Page 22: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/22.jpg)
Subspace iteration
• the i-th column of converges to a Schur vector associated with the eigenvalue • the convergence of the column is given by the factor • the speed of convergence for an eigenvalue depends on how close isit to the next one
Variants of the subspace iteration method
• take the dimension of the subspace m larger than nev number of
eigenvalues wanted
• perform “locking” i.e., as soon as an eigenvalue has converged stop multiplying with A the corresponding vector in the subsequent
iterations
0Xi
1i
i
![Page 23: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/23.jpg)
Subspace iteration
Some very theoretical result on residual norm
Xk kS 00 XS P
Pk kSprojection onto
projection onto the subspace spanned by the eigenvectors associated with the first m eigenvalues of
Then for any eigenvalue of there is an unique
such that and ui A
A
assume that are linearly independent , 1, ,Pxi i m
0si SPs ui i
122
u u sI P
k
mk i i i k
i
0
![Page 24: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/24.jpg)
Summary
• The power method can be used to compute the dominant eigenvalue(real) and a corresponding eigenvector.
• Variants of the power method can compute the smallest eigenvalue orthe eigenvalue closest to a given number (shift).
• General projection methods consist in approximating the eigenvectors of a matrix with vectors belonging to a subspace of approximants with dimension smaller than the dimension of the matrix.
• Subspace iteration method is a generalization of the power method thatcomputes a given number of dominant eigenvalues and their corresponding eigenvectors.
![Page 25: An introduction to iterative projection methods Eigenvalue problems Luiza Bondar the 23 rd of November -2005 4 th Seminar](https://reader036.vdocument.in/reader036/viewer/2022081515/56649d5e5503460f94a3e666/html5/thumbnails/25.jpg)
Last minute questions answered by
Tycho van NoordenSorin Pop