an introduction to iterative projection methods eigenvalue problems luiza bondar the 23 rd of...
Post on 21-Dec-2015
216 Views
Preview:
TRANSCRIPT
An introduction to iterative projection methods
Eigenvalue problems
Luiza Bondar
the 23rd of November -2005
4th Seminar
Introduction (Erwin)
Perturbation analysis (Nico)
Direct (global) methods (Peter)
Introduction to projection methods (Luiza) (theoretical
background)
Krylov subspace methods 1 (Mark)
Krylov subspace methods 2 (Willem)
Outline
• Introduction
• The power method
• Projection Methods
• Subspace iteration
• Summary
Direct methods (Schur decomposition, QR iteration, Jacobi method,
method of Sturm sequences )
• compute all the eigenvalues and the corresponding eigenvectors
What if we DON’T need all the eigenvalues?
Example : compute the page rank of the www documents
Introduction
WEB: a graph (pages are nodes links are edges )
Introduction
Web graph: 1.4 bilion nodes (pages) 6.6 bilion edges (links)
page rank of page i : the probability that a surfer will visit the page i
The page rank is a dominant vector of a sparse 1.4 bilion X 1.4 bilionmatrix.
It makes little sense to compute all the eigenvectors.
page rank : vector with dimension N=1.4 bilion
Introduction
The power method
• computes the dominant eigenvalue and an associated eigenvector
Some background
consider that A has p distinct eigenvalues.
1 2n
pM M M
dim i iM
n niP :
kki i i iA P P I D 0i i
iD semi-simple i 0iD
iis the algebraic multiplicity of i
iPis the projection onto iM
The power method
consider that the dominant eigenvalue is unique and is semi-simple 1
initial vector such that
convergence ?NO YES
1 k
1 kx v
1 0P v 0
1Avk k 1
1k k
k v Av
0v
0
1v A vkk
k ( ) 1Avk compute an
dtake
The power method
initial vector 0vn , 1 2
npM M M 0 0
1
v vp
ii
P
0
1v A vkk
k
kki i i iA P P I D use
11 0 0
2 1
1v v v
k pk
k i i i ikik
P P D P
then 1 01 0
1v v
vk PP
and 1k ( )1Av vk k k
0
convergence of each term in given by
Σ1
i
The power method is used by to compute the page rank.
The power method
• the convergence of the method is given by
• the convergence might be very slow if are close from one another
• if the dominant eigenvalue is multiple but semi-simple, then the algorithm provides only one eigenvalue and a corresponding eigenvector
• does not converge if the dominant eigenvalue is complex and theoriginal matrix is real (2 eigenvalues with the same modulus)
2
1
1 2,
IMPROVEMENT : the shifted power method
LED TO : projection methods
The power method
Shifted power method A A I x x
Example
• let be the dominant eigenvalue of a matrix that has an egenvalue • then the power method does not converge when applied to • but the power method converges for a shift (e.g. )
1 1 1i
A IA
A
Other variants of the power method
• inverse power method (iterates with )
• inverse power method with shift
-1A smallest eigenvalue
eigenvalue closest to the shift
The power method
• inverse power method
A LU1 -1 -1
k-1v U L vkk
then converges to the smallest eigenvalue and converges to an
associated eigenvector k vk
• inverse power method with shift
-A I LU
1 1-
-1 -1 -1k-1 k-1v A I v U L vk
k k
then converges to and converges to an eigenvector associated with
k 1
vk
The power method
• does not converge if the dominant eigenvalue is complex and the original matrix is real (2 eigenvalues with the same modulus)
But after a certain k
1v vk k,
IDEA: extract the vectors by performing a projection into the subspace
contains approximations to the complex par of eigenvectors
0 1
1 0A=
1 i
1
i
2 i
1
i
power method
1
1
1
1
vk 1vk
Projection methods (Introduction)
1u vk
2 1u vk
ufind and such that Au u
1 1 2 2u u u
• impose 2 more constrains• one choice is to impose orthogonality conditions (Galerkin) i.e.,
1 , 2• introduce 2 degrees of freedom
1Au u u and
2Au u u
1u
2u
u
Auprojection method
Projection methods (Introduction)
Generalizationn nA
, nK L
dim K=dim L=m
find and such that nu Au u
A projection technique seeks an approximate eigenpar and such that
Ku
K uuA ~~~
L uuA ~~~
• orthogonal projection
or• oblique projection
K: the right subspace, L: the left subspace
A way to construct K is Krylov subspace
(inspired by the power method)
10 0 0, , ,m mK v Av A v
Projection methods (orthogonal)
Consider an orthonormal basis of K and
1 2, , , mv v v
1 2| | | ,mV v v v
Kuthe approximate can be written as u Vy
K uuA ~~~ , 0, Au u v K v , 0, 1, ,jv j m AVy vy
HV AVy y
n mV
:m HB V AV
i i
ix i iu Vx
• eigenvalue of then eigenvalue of
• eigenvector of then eigenvector of
mB
mB
A
A
Arnoldi’s method and the hermitian Lanczos algorithm are orthogonalprojection methods
Projection methods (oblique)
L uuA ~~~ KuSearch for and such that
1 2, , , mv v v 1 2| | | ,mV v v vn mV
the approximate can be written as u Vy
1 2| | | ,mW w w wn mW 1 2, , , mw w w
orthonormal basis of K
orthonormal basis of L
and are such that (biorthogonal) V W HW V I
Ku
The condition leads to the approximate eigenvalue
problem
L uuA ~~~
HW AVy y
The nonhermitian Lanczos alghoritm is an oblique projection method.
Projection methods (orthogonal)
How accurate can an orthogonal projection method be?
, uexact eigenpar
then 2 22
H Hk k kV AV I V u P A I P I P u
projection onto KkP
kP u
uK
u
ku-P u
Projection methods (orthogonal)
Hermitian case
kP u
uK
u
2( )
sin 1 sin ,,
k kP A I P
d
kuu, u
AP u
2
2sin ,A uI u
Subspace iteration
• generalization of the power method• start with an initial system of m vectors instead of
only one vector (power method)
• compute the matrix
0 1 2, , , mX x x x
0k
k X A X
If each of the m vectors is normalised in the same way as for the power method, then each of these vectors will converge to the SAMEeigenvector associated with the dominant eigenvalue (provided that )1 0, 1,Pxi i m
Note looses its linear independenceIDEA: restore the linear independence by performing aQR factorisation
0k
k X A X
Subspace iteration
0 1 2, , , mX x x x
0 0 0X Q R :0 0X Q
-1X AXk k
-1 -1 1 -1H H HH X AX Q AQk k k k k
start with
QR factorize 0X take
compute
convergence ?X Q Rk k k
:X Qk k
recover the first m eigenvalues
and corresponding eigenvectors
of A from Hk
NO YES
Subspace iteration
• the i-th column of converges to a Schur vector associated with the eigenvalue • the convergence of the column is given by the factor • the speed of convergence for an eigenvalue depends on how close isit to the next one
Variants of the subspace iteration method
• take the dimension of the subspace m larger than nev number of
eigenvalues wanted
• perform “locking” i.e., as soon as an eigenvalue has converged stop multiplying with A the corresponding vector in the subsequent
iterations
0Xi
1i
i
Subspace iteration
Some very theoretical result on residual norm
Xk kS 00 XS P
Pk kSprojection onto
projection onto the subspace spanned by the eigenvectors associated with the first m eigenvalues of
Then for any eigenvalue of there is an unique
such that and ui A
A
assume that are linearly independent , 1, ,Pxi i m
0si SPs ui i
122
u u sI P
k
mk i i i k
i
0
Summary
• The power method can be used to compute the dominant eigenvalue(real) and a corresponding eigenvector.
• Variants of the power method can compute the smallest eigenvalue orthe eigenvalue closest to a given number (shift).
• General projection methods consist in approximating the eigenvectors of a matrix with vectors belonging to a subspace of approximants with dimension smaller than the dimension of the matrix.
• Subspace iteration method is a generalization of the power method thatcomputes a given number of dominant eigenvalues and their corresponding eigenvectors.
Last minute questions answered by
Tycho van NoordenSorin Pop
top related