ENGG2013 Unit 17
Diagonalization Eigenvector and eigenvalue
Mar, 2011.
EXAMPLE 1
kshum ENGG2013 2
Q6 in midterm• u(t): unemployment rate in the t-th month.• e(t)= 1-u(t)• The unemployment rate in the next month is
given by a matrix multiplication
• Equilibrium: Solve
kshum ENGG2013 3
Unemployment rate at equilibrium = 0.2
Equilibrium
kshum ENGG2013 4
Unstable Stable
If stable, how fast does it converge to the equilibrium point?
kshum ENGG2013 5
0.2 0.2
Fast convergenceSlow convergence
Question
• Suppose that the initial unemployment rate at the first month is x(1), (for example x(1)=0.25), and suppose that the unemployment evolves by matrix multiplication
Find an analytic expression for x(t), for all t.
kshum ENGG2013 6
EXAMPLE 2
kshum ENGG2013 7
How to count?
• Count the number of binary strings of length n with no consecutive ones.
kshum ENGG2013 8
SOLVING RECURRENCE RELATION
kshum ENGG2013 9
Fibonacci numbers
• F1 = 1
• F2 = 1
• For n > 2, Fn = Fn-1+Fn-2.
• The Fibonacci numbers are– 1,1,3,5,8,13,21,34,55,89,144
kshum ENGG2013 10
http
://e
n.w
ikip
edia
.org
/wik
i/Fib
onac
ci_n
umb
er
A matrix formulation
• Define a vector
• Initial vector
• Find the recurrence relation in matrix form
kshum ENGG2013 11
A general question
• Given initial condition
and for t 2
Find v(t) for all t.
kshum ENGG2013 12
Matrix power
• Need to raise a matrix to a very high power
kshum ENGG2013 13
A trivial special case
• Diagonal matrix
• The solution is easy to find
• Raising a diagonal matrix to the power t is easy.
kshum ENGG2013 14
Decoupled equations
• When the equation is diagonal, we have two separate equation, each in one variable
kshum ENGG2013 15
DIAGONALIZATION
kshum ENGG2013 16
Problem reduction
• A square matrix M is called diagonalizable if we can find an invertible matrix, say P, such that the product P–1 M P is a diagonal matrix.
• A diagonalizable matrix can be raised to a high power easily. – Suppose that P–1 M P = D, D diagonal. – M = P D P–1.– Mn = (P D P–1) (P D P–1) (P D P–1) … (P D P–1) = P Dn P–1.
kshum ENGG2013 17
Example of diagonalizable matrix
• Let
• A is diagonalizable because we can find a matrix
such that
kshum ENGG2013 18
Now we know how fast it converges to 0.2
• The matrix can be diagonalized
kshum ENGG2013 19
Convergence to equilibrium
• The trajectory of the unemployment rate– the initial point is set to 0.1
kshum ENGG2013 20
1 2 3 4 5 6 7 8 9 100.1
0.11
0.12
0.13
0.14
0.15
0.16
0.17
0.18
0.19
0.2
month (t)
Une
mpl
oym
ent
rate
EIGENVECTOR AND EIGENVALUE
kshum ENGG2013 21
How to diagonalize?
• How to determine whether a matrix M is diagonalizable?
• How to find a matrix P which diagonalizes a matrix M?
kshum ENGG2013 22
From diagonalization to eigenvector
• By definition a matrix M is diagonalizable ifP–1 M P = D
for some invertible matrix P, and diagonal matrix D.
or equivalently,
kshum ENGG2013 23
The columns of P are special• Suppose that
kshum ENGG2013 24
Definition
• Given a square matrix A, a non-zero vector v is called an eigenvector of A, if we an find a real number (which may be zero), such that
• This number is called an eigenvalue of A, corresponding to the eigenvector v.
kshum ENGG2013 25
Matrix-vector product Scalar product of a vector
Important notes
• If v is an eigenvector of A with eigenvalue , then any non-zero scalar multiple of v also satisfies the definition of eigenvector.
kshum ENGG2013 26
k 0
Geometric meaning• A linear transformation L(x,y) given by: L(x,y) = (x+2y, 3x-4y)
• If the input is x=1, y=2 for example, the output is x = 5, y = -5.
kshum 27
x x + 2yy 3x – 4y
Invariant direction• An Eigenvector points at a direction which is invariant under the linear
transformation induced by the matrix.• The eigenvalue is interpreted as the magnification factor.• L(x,y) = (x+2y, 3x-4y)• If input is (2,1), output is magnified by a factor of 2, i.e., the eigenvalue is 2.
kshum 28
Another invariant direction• L(x,y) = (x+2y, 3x-4y)• If input is (-1/3,1), output is (5/3,-5). The length is increased by a factor of 5, and
the direction is reversed. The corresponding eigenvalue is -5.
kshum 29
Eigenvalue and eigenvector of
First eigenvalue = 2, with eigenvector
where k is any nonzero real number.
Second eigenvalue = -5, with eigenvector
where k is any nonzero real number.
kshum ENGG2013 30
Summary
• Motivation: want to solve recurrence relations.
• Formulation using matrix multiplication• Need to raise a matrix to an arbitrary power• Raising a matrix to some power can be easily
done if the matrix is diagonalizable.• Diagonalization can be done by eigenvalue
and eigenvector.
kshum ENGG2013 31