EIGEN-VALUES, EIGEN-VECTORS
QR FACTORIZATION (1)
ELM1222 Numerical Analysis
1
Some of the contents are adopted from
Laurene V. Fausett, Applied Numerical Analysis using MATLAB. Prentice Hall Inc., 1999
ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Eigen-values, Eigen-vectors
Why do we need eigen-values, eigen-vectors?
β’ Eigen-values and eigen-vectors give us useful and important information
about a matrix ( a special matrix representing a system, or a data):
Some examples:
1. determine whether or not a matrix is positive definite(matrix A is positive
definite if and only if the eigenvalues of A are positive)
2. determine whether or not a matrix is invertible as well as to indicate how
sensitive the determination of the inverse will be to numerical errors.
3. provide important representation for matrices known as the eigenvalue
decomposition
2 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Eigen-values, Eigen-vectors
β’ Eigenvalues of an matrix A are obtained by solving its
β’ characteristic equation
β’ ππ + ππβ1ππβ1 + ππβ2ππβ2 + β― + π1π1 + π0 = 0
β’ For large values of n, polynomial equations like this one are difficult and time-
consuming to solve.
β’ Moreover, numerical techniques for approximating roots of polynomial
equations of high degree are sensitive to rounding errors.
β’ We need alternative methods for approximating eigenvalues
β’ As presented here, the method can be used only to find the eigenvalue of A
that is largest in absolute value βthe dominant eigenvalue of A.
β’ Although this restriction may seem severe, dominant eigenvalues are of
primary interest in many physical applications.
3 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Dominant Eigen-value
β’ Let π1, π2, π3,β¦, ππ be the eigen-values of an A matrix with size ππ₯π.
β’ If the eigen values in magnitude will be sorted like
π1 < π2 < π3 < β― < ππ , ππ is called the dominant eigen-value
Example 1:
Find the dominant eigen value of the following matrix.
2 β121 β5
The characteristic equation will be
(2 β π)(β5 β π) + 12 = 0
β10 + 3 π + π 2 + 12 = 0
π = {β1, β2}
The dominant one is -2 and corresponding eigen-vector is π₯ = π‘ 3 1 π where π‘ β 0
4 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
NUMERICAL TECHNIQUES FOR EIGEN-
VALUES, EIGEN-VECTOR APPROXIMATION
5 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Power Method
β’ Accelerated Power Method
β’ Shifted Power Method
β’ Inverse Power Method
Power Method
β’ Like the Jacobi and Gauss-Seidel methods, the power method for
approximating eigenvalues is iterative.
β’ We chose an initial approximation of one of the dominant eigenvectors of A.
β’ This initial approximation can be a nonzero vector z
β’ We will have
w = Az
If z is an eigen-vector, then for any component we will have
Ξ» zk = wk
If z is not an eigen-vector we will use w as the next approximation of z but
in the scaled form such that the largest component of z will be 1
6 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Power Method
The iteration pattern will be as following;
w(1) = A z(1),
---
z(2) = w(1)
π€(1)π
=A z(1)
π€(1)π w(2) = A z(2) = A
A z(1)
π€(1)π
= A2 z(1)
π€(1)π
,
z(3) =w(2)
π€(2)π
=A z(2)
π€(2)π
= A2 z(1)
π€(2)π. π€(1)
π
w(3) = A z(3) = AA z(2)
π€(2)π
= A3 z(1)
π€(2)π. π€(1)
π
---
w(i) = A z(i), z(i) =w(i)
w(i)π
7 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Power Method
Example 2:
Given the A matrix find the dominant eigen-value and corresponding eigen-
vector using Power method
8 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
π³ = 1 1 1 π Initial eigen-vector
First Iteration
π° = ππ³ = 27 19 20 π π°π = 27
π³ = π°/π€1 = 1.000 0.7037 0.7407 π
Second Iteration
π° = ππ³ = 25.1852 15.1111 16.0000 π π°π = 25.1852
π³ = π°/π€1 = 1.000 0.6000 0.6353 π
Power Method
9 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Third Iteration
π° = ππ³ = 24.5647 13.6471 14.3059 π π°π = 24.5647
π³ = π°/π€1 = 1.000 0.5556 0.5824 π
Fourth Iteration
π° = ππ³ = 24.3065 12.9655 13.4253 π π°π = 24.3065 π³ = π°/π€1 = 1.000 0.5334 0.5523 π
π β π€1, π = 24.3065
π³ = 1.000 0.5334 0.5523 π
ππ³ β Ξ»π³ = β0.1249 β0.3653 β0.5123 π
ππ³ β Ξ»π³ β = 0.5123
Accelerated Power Method
β’ In some cases, when A is symmetric power method with
β’ Rayleigh quotient converges more rapidly than classic power method.
β’ If x is an eigen-vector matrix A then its corresponding eigenvalue is given by
Ξ» =π§πAz
π§π. z=
π§πw
π§π. z
β’ This is called Rayleigh quotient
10 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Accelerated Power Method
Example 3:
Given the A matrix find the dominant eigen-value and corresponding eigen-
vector using Rayleigh quotient
11 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Shifted Power Method
β’ If we already know an eigenvalue Ξ» of a matrix A , we can find another
eigenvalue of A by applying the power method to the matrix B = A βΞ»I .
β’ Denote the dominant eigenvalue of the shifted matrix B as ΞΌ
Example 4
If one eigen-value of the following matrix is 6 to find another eigen value apply
power method to the shifted matrix
12 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Shifted Power Method
13 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Initialize iterations with
π³ = 1 1 1 π
Apply Rayleigh quotient approximation
First Iteration
π° = ππ³ = β3 β5 0 π
π =π³Tπ°
π³Tπ³= β
3
8
w2 = β5 π³ = π°/π€2 = 3/5 1 0 π
Second Iteration
π° = ππ³ = β13/5 β21/5 0 π
π =π³Tπ°
π³Tπ³= β
72
17
w2 = β21
5
π³ = π°/π€2 = 13/21 1 0 π
ππ³ β Ξ»π³ π < 0.0001 we can use this as a stopping criterion
Inverse power method
β’ Provides an estimate of the eigenvalue of A that is of smallest magnitude
β’ Based on the fact that eigenvalues of B = Aβ1 are the reciprocals of the
eigenvalues of A.
β’ Thus, we apply the power method to B = Aβ1 to find its dominant eigenvalue
ΞΌ .
β’ Then, reciprocal of ΞΌ (i.e. π = 1/π) will give the smallest magnitude.
β’ It is not desirable to actually compute Aβ1 .instead where the power method
is normally Aβ1 z = w, we will use the form A w = z
14 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Inverse power method
Example 5: Given the following A matrix calculate the smallest eigen value
using inverse Power Method
15 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Initialize iterations with
π³ = 1 1 1 π
First Iteration
ππ° = π³ π° = 0.0286 0.0651 0. 0573 π
π =π€2
π§2= 0.0651
π =1
π= 15.3601
w2 = β5 π³ = π°/π€2 = 0.4400 1 0.8801 π
Inverse power method
16 ELM1222 Numerical Analysis | Dr Muharrem Mercimek
Second Iteration
ππ° = π³ π° = 0.0042 0.0842 0. 0617 π
π =π€2
π§2= 0.0842
(π€2 is the largest component of w)
π =1
π= 11.8777
π³ = π°/π€2 = β0.0495 1 0.7324 π
Third Iteration
ππ° = π³ π° = β0.0336 0.1029 0. 0639 π
π =π€2
π§2= 0.1029
π =1
π= 9.7153
The largest eigen-value for πβ1
The smallest eigen-value for A
Summary of Distance Metrics
17 ELM1222 Numerical Analysis | Dr Muharrem Mercimek