chapter 2 tridiagonal matrices - mathunipddottmath/corsi2012/lecturenotes/meurant/chap2.pdf · the...
TRANSCRIPT
We have seen that orthogonal polynomials can be defined throughthe Jacobi matrices of their three-term recurrence coefficients
The zeros of orthogonal polynomials are given by eigenvalues oftridiagonal matrices
Hence, it is useful to study properties of tridiagonal matrices
Similarity
Let
Tk =
α1 ω1
β1 α2 ω2
. . .. . .
. . .
βk−2 αk−1 ωk−1
βk−1 αk
and βi 6= ωi , i = 1, . . . , k − 1
Proposition
Assume that the coefficients ωj , j = 1, . . . , k − 1 are different fromzero and the products βj ωj are positive. Then, the matrix Tk issimilar to a symmetric tridiagonal matrix. Therefore, itseigenvalues are real
Proof.Consider D−1
k TkDk which is similar to Tk , Dk diagonal matrixwith diagonal elements δj
Take
δ1 = 1, δ2j =
βj−1 · · ·β1
ωj−1 · · ·ω1, j = 2, . . . k
Let
Jk =
α1 β1
β1 α2 β2
. . .. . .
. . .
βk−2 αk−1 βk−1
βk−1 αk
where the values βj , j = 1, . . . , k − 1 are assumed to be nonzero
Proposition
det(Jk+1) = αk+1 det(Jk)− β2k det(Jk−1)
with initial conditions
det(J1) = α1, det(J2) = α1α2 − β21 .
The eigenvalues of Jk are the zeros of det(Jk − λI )
The zeros do not depend on the signs of the coefficientsβj , j = 1, . . . , k − 1
We suppose βj > 0 and we have a Jacobi matrix
Cholesky-like factorizations
Let ∆k be a diagonal matrix with diagonal elements δj ,j = 1, . . . , k and
Lk =
1l1 1
. . .. . .
lk−2 1lk−1 1
Jk = Lk∆kLT
k
The entries of Lk and ∆k are given by
δ1 = α1, l1 = β1/δ1
δj = αj −β2
j−1
δj−1, j = 2, . . . , k, lj = βj/δj , j = 2, . . . , k − 1
The factorization can be completed if no δj is zero forj = 1, . . . , k − 1
This does not happen if Jk is positive definite, all the elements δj
are positive and the genuine Cholesky factorization can beobtained from ∆k
Jk = LCk (LC
k )T
with LCk = Lk∆
1/2k which is
LCk =
√δ1
β1√δ1
√δ2
. . .. . .
βk−2√δk−2
√δk−1
βk−1√δk−1
√δk
The factorization can also be written as
Jk = LDk ∆−1
k (LDk )T
with
LDk =
δ1
β1 δ2
. . .. . .
βk−2 δk−1
βk−1 δk
Clearly, the only elements we have to compute and store are the
diagonal elements δj , j = 1, . . . , k
To solve a linear system Jkx = c , we successively solve
LDk y = c , (LD
k )T x = ∆ky
The previous factorizations proceed from top to bottom (LU)We can also proceed from bottom to top (UL)
Jk = LTk D−1
k Lk
with
Lk =
d
(k)1
β1 d(k)2. . .
. . .
βk−2 d(k)k−1
βk−1 d(k)k
and Dk a diagonal matrix with elements d
(k)j
d(k)k = αk , d
(k)j = αj −
β2j
d(k)j+1
, j = k − 1, . . . , 1
From LU and UL factorizations we can obtain all the so-called“twisted” factorizations of Jk
Jk = MkΩkMTk
Mk is lower bidiagonal at the top for rows with index smaller thanl and upper bidiagonal at the bottom for rows with index largerthan l
ω1 = α1, ωj = αj −β2
j−1
ωj−1, j = 2, . . . , l − 1
ωk = αk , ωj = αj −β2
j
ωj+1, j = k − 1, . . . , l + 1
ωl = αl −β2
l−1
ωl−1−
β2l
ωl+1
Eigenvalues
The eigenvalues of Jk are the zeros of det(Jk − λI )
det(Jk − λI ) = δ1(λ) · · · δk(λ) = d(k)1 (λ) · · · d (k)
k (λ)
This shows that
δk(λ) =det(Jk − λI )
det(Jk−1 − λI ), d
(k)1 (λ) =
det(Jk − λI )
det(J2,k − λI )
TheoremThe eigenvalues θ
(k+1)i of Jk+1 strictly interlace the eigenvalues of
Jk
θ(k+1)1 < θ
(k)1 < θ
(k+1)2 < θ
(k)2 < · · · < θ
(k)k < θ
(k+1)k+1
(Cauchy interlacing theorem)
Proof.Eigenvector x = (y ζ)T of Jk+1 corresponding to θ
Jky + βkζek = θy
βkyk + αk+1ζ = θζ
Eliminating y from these relations, we obtain
(αk+1 − β2k((ek)T (Jk − θI )−1ek))ζ = θζ
αk+1 − β2k
k∑j=1
ξ2j
θ(k)j − θ
− θ = 0
where ξj is the last component of the jth eigenvector of Jk
The zeros of this function interlace the poles θ(k)j
See next slide
Results on eigenvector components
Let χj ,k(λ) be the determinant of Jj ,k − λI
The first components of the eigenvectors z i of Jk are
(z i1)
2 =
∣∣∣∣∣∣χ2,k(θ(k)i )
χ′1,k(θ(k)i )
∣∣∣∣∣∣ ,that is,
(z i1)
2 =θ(k)i − θ
(2,k)1
θ(k)i − θ
(k)1
· · ·θ(k)i − θ
(2,k)i−1
θ(k)i − θ
(k)i−1
θ(2,k)i − θ
(k)i
θ(k)i+1 − θ
(k)i
· · ·θ(2,k)k−1 − θ
(k)i
θ(k)k − θ
(k)i
.
The last components of the eigenvectors z i of Jk are
(z ik)2 =
∣∣∣∣∣∣χ1,k−1(θ(k)i )
χ′1,k(θ(k)i )
∣∣∣∣∣∣ ,that is,
(z ik)2 =
θ(k)i − θ
(k−1)1
θ(k)i − θ
(k)1
· · ·θ(k)i − θ
(k−1)i−1
θ(k)i − θ
(k)i−1
θ(k−1)i − θ
(k)i
θ(k)i+1 − θ
(k)i
· · ·θ(k−1)k−1 − θ
(k)i
θ(k)k − θ
(k)i
.
Inverse
What is the inverse of a tridiagonal matrix?
TheoremThere exist two sequences of numbers ui, vi, i = 1, . . . , k suchthat
J−1k =
u1v1 u1v2 u1v3 . . . u1vk
u1v2 u2v2 u2v3 . . . u2vk
u1v3 u2v3 u3v3 . . . u3vk...
......
. . ....
u1vk u2vk u3vk . . . ukvk
Moreover, u1 can be chosen arbitrarily, for instance u1 = 1
see Baranger and Duc-Jacquet; MeurantHence, we just have to compute J−1
k e1 = u1v and J−1k ek = vku
SolveJkv = e1
Use UL factorization of Jk
v1 =1
d(k)1
, vj = (−1)j−1 β1 · · ·βj−1
d(k)1 · · · d (k)
j
, j = 2, . . . , k
ForvkJku = ek
use LU factorization
uk =1
δkvk, uk−j = (−1)j
βk−j · · ·βk−1
δk−j · · · δkvk, j = 1, . . . , k − 1
TheoremThe inverse of the symmetric tridiagonal matrix Jk is characterizedas
(J−1k )i ,j = (−1)j−iβi · · ·βj−1
d(k)j+1 · · · d
(k)k
δi · · · δk, ∀i , ∀j > i
(J−1k )i ,i =
d(k)i+1 · · · d
(k)k
δi · · · δk, ∀i
Proof.
ui = (−1)−(i+1) 1
β1 · · ·βi−1
d(k)1 · · · d (k)
k
δi · · · δk
The diagonal elements of J−1k can also be obtained using twisted
factorizations
TheoremLet l be a fixed index and ωj the diagonal elements of thecorresponding twisted factorizationThen
(J−1k )l ,l =
1
ωl
In the sequel we will be interested in (J−1k )1,1
(J−1k )1,1 =
1
d(k)1
The (1,1) entry of the inverse
Can we compute (J−1k )1,1 incrementally?
Theorem
(J−1k+1)1,1 = (J−1
k )1,1 +(β1 · · ·βk)2
(δ1 · · · δk)2δk+1
Proof.
Jk+1 =
(Jk βkek
βk(ek)T αk+1
)The upper left block of J−1
k+1 is the inverse of the Schurcomplement (
Jk −β2
k
αk+1ek(ek)T
)−1
Inverse of a rank-1 modification of Jk
Use the Sherman–Morrison formula
(A + αxyT )−1 = A−1 − αA−1xyTA−1
1 + αyTA−1x
This gives(Jk −
β2k
αk+1ek(ek)T
)−1
= J−1k +
(J−1k ek)((ek)T J−1
k )αk+1
β2k− (ek)T J−1
k ek
Let lk = J−1k ek
(J−1k+1)1,1 = (J−1
k )1,1 +β2
k(lk1 )2
αk+1 − β2k lkk
lk1 = (−1)k−1 β1 · · ·βk−1
δ1 · · · δk, lkk =
1
δk
To simplify the formulas, we note that
αk+1 − β2k lkk = αk+1 −
β2k
δk= δk+1
We start with (J−11 )1,1 = π1 = 1/α1 and c1 = 1
t = β2k πk , δk+1 = αk+1 − t, πk+1 =
1
δk+1, ck+1 = t ck πk
This gives(J−1
k+1)1,1 = (J−1k )1,1 + ck+1πk+1
The QD algorithm
The QD algorithm is a method introduced by Heinz Rutishauser tocompute the eigenvalues of a tridiagonal matrix
See also Stiefel, Henrici, Fernando and Parlett, Parlett and Laurie
Let us start with the LR algorithm
LR algorithm for eigenvalues
- Start from the Cholesky factorization Jk = LkLTk of the
tridiagonal positive definite matrix Jk
- Compute Jk = LTk Lk ⇒ Jk = L−1
k JkLk
The matrix Jk is similar to the matrix Jk
- Iterate the process obtaining a series of matrices J(i)k with
J(0)k = Jk , J
(1)k = Jk , . . .
This is the basis of the LR algorithm of Rutishauser
The off-diagonal elements tend to zero and in the limit we obtainthe eigenvalues of Jk on the diagonal
Can we compute Lk , the Cholesky factor of Jk = LTk Lk without
explicitly computing Jk?
We have
Jk =
δ1 +β2
1δ1
β1
√δ2δ1
β1
√δ2δ1
δ2 +β2
2δ2
β2
√δ3δ2
. . .. . .
. . .
βk−2
√δk−1
δk−2δk−1 +
β2k−1
δk−1βk−1
√δk
δk−1
βk−1
√δk
δk−1δk
Let√
εj be the subdiagonal entries of LCk
Therefore, εj = β2j /δj
The diagonal entries of the Cholesky-like factorization of Jk aregiven by
δ1 = δ1 +β2
1
δ1= δ1 + ε1
δj = δj +β2
j
δj−
β2j−1δj/δj−1
δj−1
= δj + εj −εj−1δj
δj−1
, j = 2, . . . , k − 1
δk = δk −β2
k−1δk/δk−1
δk−1
= δk −εk−1δk
δk−1
Let εj = β2j δj+1/(δj δj)
The diagonal entries of Lk are√
δj , the subdiagonal entries are√εj and
εj = εjδj+1
δj
The expression for δj can be written as
δj = δj + εj − εj−1
The QD algorithm is the following; given δj and εj :
ε0 = 0for j=1:k-1δj = (δj − εj−1) + εjεj = εjδj+1/δj
endδk = δk − εk−1
Then, in the LR algorithm, we do δj = δj , εj = εj and we iterateuntil the off-diagonal elements are small enough
The differential QD algorithm (dqd)The QD algorithm can be modified by removing subtractions
δj =δj
δj−1
(δj−1 − εj−1) + εj
Let tj = δj − εj . Then
tj = tj−1δj
δj−1
, δj = tj + εj
t = δ1
for j=1:k-1δj = t + εjf = δj+1/δj
εj = f εjt = ft
endδk = t
The differential QD algorithm with shift
The algorithm with shift dqds(µ) is:
t = δ1 − µfor j=1:k-1δj = t + εjf = δj+1/δj
εj = f εjt = ft − µ
endδk = t
The Cholesky factorization of a shifted matrix
Assume we know the factorization of Jk = LDLT . We would liketo compute the factorization of Jk − µI = LDLT
δ1 = α1−µ, δj = αj−µ−β2
j−1
δj−1
, j = 2, . . . , k, lj =βj
δj
, j = 1, . . . , k−1
Butα1 = δ1, αj = δj + δj−1lj−1, j = 2, . . . , k
βj = δj lj , j = 1, . . . , k − 1
We can eliminate αj and βj from the formulas giving δj and lj
δ1 = δ1 − µ, δj = δj − µ + δj−1l2j−1 −
δ2j−1l
2j−1
δj−1
, j = 2, . . . , k
and
lj =δj lj
δj
, j = 1, . . . , k − 1
Finally
δj = δj − µ + δj−1l2j−1 − δj−1lj−1 lj−1, j = 2, . . . , k
This algorithm has been named stqds by Dhillon and Parlett
The dstqds algorithm
Let sj = δj − δj ; then
sj = lj−1 lj−1
(lj−1δj−1
lj−1
− δj−1
)− µ
= lj−1 lj−1(δj−1 − δj−1)− µ
= lj−1 lj−1sj−1 − µ
s = −µfor j=1:k-1δj = s + δj
lj = (δj lj)/δj
s = lj ljs − µendδk = s + δk
Historical note
I Augustin Louis Cauchy, (1789 Paris, France-1857 Sceaux,France), French mathematician
I Issai Schur, (1875 Mogilev, Russia-1941 Tel Aviv, Israel),German mathematician
I Andre-Louis Cholesky, (1875 Montguyon, France-1918Bagneux (Soissons), France), French geodesist
I Eduard Ludwig Stiefel, (1909-1978 Zurich, Switzerland),Swiss mathematician
I Heinz Rutishauser, (1918 Weinfelden, Switzerland-1970Zurich, Swtizerland), Swiss mathematician
I Peter Henrici, (1923 Basel, Switzerland-1987 Zurich,Switzerland), Swiss mathematician
I Jack Sherman, (?), American statistician
I Winifred J. Morrison, (?), American statistician
I Beresford Neill Parlett, (? ?-), American mathematician
J. Baranger and M. Duc-Jacquet, Matricestridiagonales symetriques et matrices factorisables, RIRO, v 3(1971), pp. 61–66
I.S. Dhillon and B.N. Parlett, Relatively robustrepresentations of symmetric tridiagonals, Linear Alg. Appl.,v 309 (2000), pp. 121–151
K.V. Fernando and B.N. Parlett, Accurate singularvalues and differential qd algorithms, Numer. Math., v 67(1994), pp. 191–229
G.H. Golub and C. Van Loan, Matrix Computations,Third Edition, Johns Hopkins University Press, (1996)
P. Henrici, The quotient-difference algorithm, NBSAppl. Math. Series, v 49 (1958), pp. 23–46
G. Meurant, A review of the inverse of tridiagonal andblock tridiagonal matrices, SIAM J. Matrix Anal. Appl., v 13n 3 (1992), pp. 707–728
B.N. Parlett, The new qd algorithms, Acta Numerica, v 15(1995), pp. 459-491
H. Rutishauser, Der Quotienten-Differenzen-Algorithmus,Zeitschrift fur Angewandte Mathematik und Physik (ZAMP),v 5 n 3 (1954), pp. 233-251
H. Rutishauser, Solution of eigenvalue problems with theLR-transformation , Nat. Bur. Standards Appl. Math. Ser.,v 49 (1958), pp. 47–81