math 504 fall 2016 notes week 13, lecture...
TRANSCRIPT
Math 504 Fall 2016 NotesWeek 13, Lecture 1
Emre Mengi
Department of MathematicsKoç University
Istanbul, Turkey
Emre Mengi Week 13, Lecture 1
Outline
Background about Eigenvalues
Computation of All Eigenvalues
OverviewReduction into Hessenberg FormThe QR Algorithm
Emre Mengi Week 13, Lecture 1
Background
Emre Mengi Week 13, Lecture 1
Similarity Transformation
For an invertible S ∈ Cn×n, the transformation
A 7−→ SAS−1
is called a similarity transformation.
Remark: A and SAS−1 have the same characteristicpolynomials, and the same set of eigenvalues.
Unitary similarity transformationFor a unitary Q ∈ Cn×n
A 7−→ QAQ∗
Emre Mengi Week 13, Lecture 1
Similarity Transformation
For an invertible S ∈ Cn×n, the transformation
A 7−→ SAS−1
is called a similarity transformation.
Remark: A and SAS−1 have the same characteristicpolynomials, and the same set of eigenvalues.
Unitary similarity transformationFor a unitary Q ∈ Cn×n
A 7−→ QAQ∗
Emre Mengi Week 13, Lecture 1
Similarity Transformation
For an invertible S ∈ Cn×n, the transformation
A 7−→ SAS−1
is called a similarity transformation.
Remark: A and SAS−1 have the same characteristicpolynomials, and the same set of eigenvalues.
Unitary similarity transformationFor a unitary Q ∈ Cn×n
A 7−→ QAQ∗
Emre Mengi Week 13, Lecture 1
Eigenspace and Multiplicities
Eigenspace associated with an eigenvalue λ
Eλ = Null(A− λI)
Geometric multiplicity of an eigenvalue λ = dim Eλ
Algebraic multiplicity of an eigenvalue λ = multiplicityof λ as a root of det(A− zI)
PropositionAlgebraic multiplicity of λ ≥ Geometric multiplicity of λ
Emre Mengi Week 13, Lecture 1
Eigenspace and Multiplicities
Eigenspace associated with an eigenvalue λ
Eλ = Null(A− λI)
Geometric multiplicity of an eigenvalue λ = dim Eλ
Algebraic multiplicity of an eigenvalue λ = multiplicityof λ as a root of det(A− zI)
PropositionAlgebraic multiplicity of λ ≥ Geometric multiplicity of λ
Emre Mengi Week 13, Lecture 1
Eigenspace and Multiplicities
Eigenspace associated with an eigenvalue λ
Eλ = Null(A− λI)
Geometric multiplicity of an eigenvalue λ = dim Eλ
Algebraic multiplicity of an eigenvalue λ = multiplicityof λ as a root of det(A− zI)
PropositionAlgebraic multiplicity of λ ≥ Geometric multiplicity of λ
Emre Mengi Week 13, Lecture 1
Eigenspace and Multiplicities
Eigenspace associated with an eigenvalue λ
Eλ = Null(A− λI)
Geometric multiplicity of an eigenvalue λ = dim Eλ
Algebraic multiplicity of an eigenvalue λ = multiplicityof λ as a root of det(A− zI)
PropositionAlgebraic multiplicity of λ ≥ Geometric multiplicity of λ
Emre Mengi Week 13, Lecture 1
Computation of All Eigenvalues
Emre Mengi Week 13, Lecture 1
Overview
1 Reduction into Hessenberg form: Form unitary Q ∈ Cn×n
such thatQ∗AQ = H
is Hessenberg.2 The QR Algorithm: Form unitary Q1, . . . ,Qk ∈ Cn×n such
thatQ∗
k . . .Q∗1HQ1 . . .Qk =: Hk
becomes upper triangular in the limit as k →∞ under mildassumptions.
Note : A,H,Hk all have the same eigenvalues.
Emre Mengi Week 13, Lecture 1
Overview
1 Reduction into Hessenberg form: Form unitary Q ∈ Cn×n
such thatQ∗AQ = H
is Hessenberg.2 The QR Algorithm: Form unitary Q1, . . . ,Qk ∈ Cn×n such
thatQ∗
k . . .Q∗1HQ1 . . .Qk =: Hk
becomes upper triangular in the limit as k →∞ under mildassumptions.
Note : A,H,Hk all have the same eigenvalues.
Emre Mengi Week 13, Lecture 1
Overview
1 Reduction into Hessenberg form: Form unitary Q ∈ Cn×n
such thatQ∗AQ = H
is Hessenberg.2 The QR Algorithm: Form unitary Q1, . . . ,Qk ∈ Cn×n such
thatQ∗
k . . .Q∗1HQ1 . . .Qk =: Hk
becomes upper triangular in the limit as k →∞ under mildassumptions.
Note : A,H,Hk all have the same eigenvalues.
Emre Mengi Week 13, Lecture 1
Reduction into Hessenberg Form
3× 3 illustration
A =
x x xx x xx x x
7−→︸︷︷︸left multiply with Q1
x x xx x x0 x x
7−→︸︷︷︸
right multiply with Q∗1=Q1
x x xx x x0 x x
= H
where H = Q1AQ∗1.
Q1 =
[1 00 H1
]H1 = I − 2u(1)(u(1))∗, u(1) =
A(2 : 3,1)− ‖A(2 : 3,1)‖2e(1)∥∥A(2 : 3,1)− ‖A(2 : 3,1)‖2e(1)∥∥
2
Emre Mengi Week 13, Lecture 1
Reduction into Hessenberg Form
3× 3 illustration
A =
x x xx x xx x x
7−→︸︷︷︸left multiply with Q1
x x xx x x0 x x
7−→︸︷︷︸
right multiply with Q∗1=Q1
x x xx x x0 x x
= H
where H = Q1AQ∗1.
Q1 =
[1 00 H1
]H1 = I − 2u(1)(u(1))∗, u(1) =
A(2 : 3,1)− ‖A(2 : 3,1)‖2e(1)∥∥A(2 : 3,1)− ‖A(2 : 3,1)‖2e(1)∥∥
2
Emre Mengi Week 13, Lecture 1
Reduction into Hessenberg Form
kth column
A(k) =
[H(k) D(k)
0 M(k)
]7−→︸︷︷︸
left multiply
[H(k) D(k)
0 HkM(k)
]= A(k)
7−→︸︷︷︸right multiply
[A(k)(:,1 : k) A(k)(:, k + 1 : n)Hk
]
where H(k) ∈ Ck×(k−1) is Hessenberg.
Hk = I − 2u(k)[u(k)]∗
u(k) =h(k) − ‖h(k)‖2e(1))
‖h(k) − ‖h(k)‖2e(1)‖2, h(k) = A(k)(k + 1 : n, k)
Emre Mengi Week 13, Lecture 1
Reduction into Hessenberg Form
kth column
A(k) =
[H(k) D(k)
0 M(k)
]7−→︸︷︷︸
left multiply
[H(k) D(k)
0 HkM(k)
]= A(k)
7−→︸︷︷︸right multiply
[A(k)(:,1 : k) A(k)(:, k + 1 : n)Hk
]
where H(k) ∈ Ck×(k−1) is Hessenberg.
Hk = I − 2u(k)[u(k)]∗
u(k) =h(k) − ‖h(k)‖2e(1))
‖h(k) − ‖h(k)‖2e(1)‖2, h(k) = A(k)(k + 1 : n, k)
Emre Mengi Week 13, Lecture 1
Reduction into Hessenberg Form
Input: A ∈ Cn×n
Output: Hessenberg H ∈ Cn×n and the vectorsu(1), . . . ,u(n−2).
1: for k = 1,2, . . . ,n − 2 do2: v ← A(k + 1 : n, k)3: u(k) ←
(v − ‖v‖e(1)) /‖v − ‖v‖e(1)‖2
4: A(k+1 : n, k : n)← A(k+1 : n, k : n)−2u(k)((u(k))∗A(k+1 : n, k : n))5: A(:, k + 1 : n)← A(: .k + 1 : n)− (A(:, k + 1 : n)u(k))(2u(k))∗
6: end for7: R ← A
# of flops ∼ 10n3/3.
Emre Mengi Week 13, Lecture 1
Reduction into Hessenberg Form
Input: A ∈ Cn×n
Output: Hessenberg H ∈ Cn×n and the vectorsu(1), . . . ,u(n−2).
1: for k = 1,2, . . . ,n − 2 do2: v ← A(k + 1 : n, k)3: u(k) ←
(v − ‖v‖e(1)) /‖v − ‖v‖e(1)‖2
4: A(k+1 : n, k : n)← A(k+1 : n, k : n)−2u(k)((u(k))∗A(k+1 : n, k : n))5: A(:, k + 1 : n)← A(: .k + 1 : n)− (A(:, k + 1 : n)u(k))(2u(k))∗
6: end for7: R ← A
# of flops ∼ 10n3/3.
Emre Mengi Week 13, Lecture 1
Reduction into Hessenberg Form
Input: A ∈ Cn×n
Output: Hessenberg H ∈ Cn×n and the vectorsu(1), . . . ,u(n−2).
1: for k = 1,2, . . . ,n − 2 do2: v ← A(k + 1 : n, k)3: u(k) ←
(v − ‖v‖e(1)) /‖v − ‖v‖e(1)‖2
4: A(k+1 : n, k : n)← A(k+1 : n, k : n)−2u(k)((u(k))∗A(k+1 : n, k : n))5: A(:, k + 1 : n)← A(: .k + 1 : n)− (A(:, k + 1 : n)u(k))(2u(k))∗
6: end for7: R ← A
# of flops ∼ 10n3/3.
Emre Mengi Week 13, Lecture 1
The QR Algorithm
Generate a sequence {Hk} such that H0 = H and,Hk , Hk+1 are related as follows:(1) Compute a QR factorization
Hk − µk I = Qk+1Rk+1
for some shift µk .(2) Hk+1 := Rk+1Qk+1 + µk I
Similarity:Hk+1 = Q∗
k+1Hk Qk+1 (i.e., Hk , Hk+1 have the same eigenvalues.)
Emre Mengi Week 13, Lecture 1
The QR Algorithm
Generate a sequence {Hk} such that H0 = H and,Hk , Hk+1 are related as follows:(1) Compute a QR factorization
Hk − µk I = Qk+1Rk+1
for some shift µk .(2) Hk+1 := Rk+1Qk+1 + µk I
Similarity:Hk+1 = Q∗
k+1Hk Qk+1 (i.e., Hk , Hk+1 have the same eigenvalues.)
Emre Mengi Week 13, Lecture 1
The QR Algorithm
Generate a sequence {Hk} such that H0 = H and,Hk , Hk+1 are related as follows:(1) Compute a QR factorization
Hk − µk I = Qk+1Rk+1
for some shift µk .(2) Hk+1 := Rk+1Qk+1 + µk I
Similarity:Hk+1 = Q∗
k+1Hk Qk+1 (i.e., Hk , Hk+1 have the same eigenvalues.)
Emre Mengi Week 13, Lecture 1
The QR Algorithm
Preservation of the Hessenberg structure
Given Hk Hessenberg,
Qk+1 = (Hk − µk I)R−1k+1
is also Hessenberg(since Hessenberg× upper triangular is Hessenberg).
Since Qk+1 is Hessenberg,
Hk+1 = Rk+1Qk+1 + µk I
is also Hesseneberg(since upper triangular × Hessenberg is also Hessenberg).
Emre Mengi Week 13, Lecture 1
The QR Algorithm
Preservation of the Hessenberg structure
Given Hk Hessenberg,
Qk+1 = (Hk − µk I)R−1k+1
is also Hessenberg(since Hessenberg× upper triangular is Hessenberg).
Since Qk+1 is Hessenberg,
Hk+1 = Rk+1Qk+1 + µk I
is also Hesseneberg(since upper triangular × Hessenberg is also Hessenberg).
Emre Mengi Week 13, Lecture 1
The QR Algorithm
Preservation of the Hessenberg structure
Given Hk Hessenberg,
Qk+1 = (Hk − µk I)R−1k+1
is also Hessenberg(since Hessenberg× upper triangular is Hessenberg).
Since Qk+1 is Hessenberg,
Hk+1 = Rk+1Qk+1 + µk I
is also Hesseneberg(since upper triangular × Hessenberg is also Hessenberg).
Emre Mengi Week 13, Lecture 1
The QR Algorithm
Hence, the sequence {Hk} is Hessenberg.(1) The QR factorization Hk − µk I = Qk+1Rk+1 can be
computed at a cost of O(n2) flops,(2) Hk+1 = Rk+1Qk+1 + µk I can be formed at a cost of O(n2).
Choice of shifts
Rayleigh shiftsµk = Hk (n,n)
Wilkinson shiftsµk = eigenvalue of Hk (n − 1 : n,n − 1 : n) closest to Hk (n,n)
Emre Mengi Week 13, Lecture 1
The QR Algorithm
Hence, the sequence {Hk} is Hessenberg.(1) The QR factorization Hk − µk I = Qk+1Rk+1 can be
computed at a cost of O(n2) flops,(2) Hk+1 = Rk+1Qk+1 + µk I can be formed at a cost of O(n2).
Choice of shifts
Rayleigh shiftsµk = Hk (n,n)
Wilkinson shiftsµk = eigenvalue of Hk (n − 1 : n,n − 1 : n) closest to Hk (n,n)
Emre Mengi Week 13, Lecture 1