linear algebra - imag · 2015-09-25 · linear algebra vectors and matrices eigenvalues and...
TRANSCRIPT
![Page 1: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/1.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Linear Algebra
Brigitte Bidegaray-Fesquet
Univ. Grenoble Alpes, Laboratoire Jean Kuntzmann, Grenoble
MSIAM, 23–24 September 2015
![Page 2: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/2.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Overview
1 Vectors and matricesElementary operationsGram–Schmidt orthonormalizationMatrix normConditioningSpecific matricesTridiagonalisationLU and QR factorizations
2 Eigenvalues and eigenvectorsPower iteration algorithmDeflationGalerkinJacobiQR
3 Numerical solution of linear systemsDirect methodsIterative methodsPreconditioning
4 StorageBand storageSparse storage
5 Bandwidth reductionCuthill–McKee algorithm
6 References
![Page 3: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/3.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Overview
1 Vectors and matricesElementary operationsGram–Schmidt orthonormalizationMatrix normConditioningSpecific matricesTridiagonalisationLU and QR factorizations
2 Eigenvalues and eigenvectorsPower iteration algorithmDeflationGalerkinJacobiQR
3 Numerical solution of linear systemsDirect methodsIterative methodsPreconditioning
4 StorageBand storageSparse storage
5 Bandwidth reductionCuthill–McKee algorithm
6 References
![Page 4: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/4.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Elementary operations on vectors
Cn resp. Rn: linear space of vectors with n entries in C resp. R.Generically: F n, where F is a field.
Linear combination of vectors
~w = α~u + β~v , α, β ∈ C or R.
f o r i = 1 to nw( i ) = a l p h a ∗ u ( i ) + b e t a ∗ v ( i )
end f o r
Scalar product of 2 vectors
~u · ~v =nP
i=1
ui vi
uv = 0f o r i = 1 to n
uv = uv + u ( i ) ∗ v ( i )end f o r
`2 norm of a vector
‖~u‖2 =
snP
i=1
u2i
uu = 0f o r i = 1 to n
uu = uu + u ( i ) ∗ u ( i )end f o rnorm = s q r t ( uu )
![Page 5: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/5.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Elementary operations on matrices
Mnp(F ): linear space of matrices with n × p entries in F .
Linear combination of matrices
C = αA + βB, α, β ∈ F .
f o r i = 1 to nf o r j = 1 to p
C( i , j ) = a l p h a ∗ A( i , j ) + b e t a ∗ B( i , j )end f o r
end f o r
Matrix–vector product
~w = A~u, wi =pP
j=1
Aijuj
f o r i = 1 to nwi = 0f o r j = 1 to p
wi = wi + A( i , j ) ∗ u ( j )end f o rw( i ) = wi
end f o r
Matrix–matrix product
C = AB, Cij =pP
k=1
AikBkj
f o r i = 1 to nf o r j = 1 to q
c i j = 0f o r k = 1 to p
c i j = c i j + A( i , k ) ∗ B( k , j )end f o rC( i , j ) = c i j
end f o rend f o r
![Page 6: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/6.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Gram–Schmidt orthonormalization
Let {~v1, . . . , ~vp} be a free family of vectors.It generates the vector space Ep with dimension p.We want to construct {~e1, . . . ,~ep}, an orthonormal basis of Ep.
Gram–Schmidt algorithm
~u1 = ~v1 ~e1 =~u1
‖~u1‖2
~u2 = ~v2 − (~v2 · ~e1)~e1 ~e2 =~u2
‖~u2‖2
. . . . . .
~up = ~vp −p−1Pk=1
(~vp · ~ek)~ek ~ep =~up
‖~up‖2
~u1 = ~v1
(~v2 · ~e1)~e1~e1
~v2~u2
~e2
![Page 7: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/7.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Matrix norms
Definition
‖A‖ ≥ 0, ∀A ∈Mnn(F ), F = C or R.‖A‖ = 0⇔ A = 0.
‖λA‖ = |λ|‖A‖, ∀A ∈Mnn(F ), ∀λ ∈ F .
‖A + B‖ ≤ ‖A‖+ ‖B‖, ∀A,B ∈Mnn(F ) (triangle inequality).
‖AB‖ ≤ ‖A‖‖B‖, ∀A,B ∈Mnn(F ) (specific for matrix norms).
Subordinate matrix norms
‖A‖p = max‖x‖p 6=0
‖Ax‖p‖x‖p
= max‖x‖p=1
‖Ax‖p, ∀x ∈ F n, where ‖~x‖p = p
snP
i=1
xpi .
in particular: ‖A‖1 = maxj
Pi
|Aij | and ‖A‖∞ = maxi
Pj
|Aij |.
Matrix-vector product estimate
‖A‖p ≥‖Ax‖p‖x‖p
and hence ‖Ax‖p ≤ ‖A‖p‖x‖p for all x ∈ F n.
![Page 8: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/8.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Matrix norms
Definition
‖A‖ ≥ 0, ∀A ∈Mnn(F ), F = C or R.‖A‖ = 0⇔ A = 0.
‖λA‖ = |λ|‖A‖, ∀A ∈Mnn(F ), ∀λ ∈ F .
‖A + B‖ ≤ ‖A‖+ ‖B‖, ∀A,B ∈Mnn(F ) (triangle inequality).
‖AB‖ ≤ ‖A‖‖B‖, ∀A,B ∈Mnn(F ) (specific for matrix norms).
Subordinate matrix norms
‖A‖p = max‖x‖p 6=0
‖Ax‖p‖x‖p
= max‖x‖p=1
‖Ax‖p, ∀x ∈ F n, where ‖~x‖p = p
snP
i=1
xpi .
in particular: ‖A‖1 = maxj
Pi
|Aij | and ‖A‖∞ = maxi
Pj
|Aij |.
Matrix-vector product estimate
‖A‖p ≥‖Ax‖p‖x‖p
and hence ‖Ax‖p ≤ ‖A‖p‖x‖p for all x ∈ F n.
![Page 9: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/9.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Matrix norms
Definition
‖A‖ ≥ 0, ∀A ∈Mnn(F ), F = C or R.‖A‖ = 0⇔ A = 0.
‖λA‖ = |λ|‖A‖, ∀A ∈Mnn(F ), ∀λ ∈ F .
‖A + B‖ ≤ ‖A‖+ ‖B‖, ∀A,B ∈Mnn(F ) (triangle inequality).
‖AB‖ ≤ ‖A‖‖B‖, ∀A,B ∈Mnn(F ) (specific for matrix norms).
Subordinate matrix norms
‖A‖p = max‖x‖p 6=0
‖Ax‖p‖x‖p
= max‖x‖p=1
‖Ax‖p, ∀x ∈ F n, where ‖~x‖p = p
snP
i=1
xpi .
in particular: ‖A‖1 = maxj
Pi
|Aij | and ‖A‖∞ = maxi
Pj
|Aij |.
Matrix-vector product estimate
‖A‖p ≥‖Ax‖p‖x‖p
and hence ‖Ax‖p ≤ ‖A‖p‖x‖p for all x ∈ F n.
![Page 10: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/10.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Matrix conditioning
Definition
Cond(A) = ‖A−1‖‖A‖.
Properties
Cond(A) ≥ 1,Cond(A−1) = Cond(A),Cond(αA) = Cond(A).
For the Euclidian norm
Cond2(A) =|λmax||λmin|
.
![Page 11: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/11.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Matrix conditioning
Definition
Cond(A) = ‖A−1‖‖A‖.
Properties
Cond(A) ≥ 1,Cond(A−1) = Cond(A),Cond(αA) = Cond(A).
For the Euclidian norm
Cond2(A) =|λmax||λmin|
.
![Page 12: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/12.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Matrix conditioning
Definition
Cond(A) = ‖A−1‖‖A‖.
Properties
Cond(A) ≥ 1,Cond(A−1) = Cond(A),Cond(αA) = Cond(A).
For the Euclidian norm
Cond2(A) =|λmax||λmin|
.
![Page 13: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/13.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Conditioning and linear systems
Problem
(S0) A~x = ~b, (Sper) (A + δA)(~x + δ~x) = (~b + δ~b).
(Sper)−(S0): Aδ~x + δA(~x + δ~x) = δ~b,
δ~x = A−1“δ~b − δA(~x + δ~x)
”,
‖δ~x‖ ≤ ‖A−1‖‚‚‚δ~b − δA(~x + δ~x)
‚‚‚ (for a subordinate matrix norm),
‖δ~x‖ ≤ ‖A−1‖“‖δ~b‖+ ‖δA‖‖~x + δ~x‖
”,
‖δ~x‖‖~x + δ~x‖ ≤ ‖A
−1‖
‖δ~b‖‖~x + δ~x‖ + ‖δA‖
!.
Result
‖δ~x‖‖~x + δ~x‖ ≤ Cond(A)
‖δ~b‖
‖A‖‖~x + δ~x‖ +‖δA‖‖A‖
!.
relative error on x = Cond(A) (relative error on ~b + relative error on A).
![Page 14: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/14.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Conditioning and linear systems
Problem
(S0) A~x = ~b, (Sper) (A + δA)(~x + δ~x) = (~b + δ~b).
(Sper)−(S0): Aδ~x + δA(~x + δ~x) = δ~b,
δ~x = A−1“δ~b − δA(~x + δ~x)
”,
‖δ~x‖ ≤ ‖A−1‖‚‚‚δ~b − δA(~x + δ~x)
‚‚‚ (for a subordinate matrix norm),
‖δ~x‖ ≤ ‖A−1‖“‖δ~b‖+ ‖δA‖‖~x + δ~x‖
”,
‖δ~x‖‖~x + δ~x‖ ≤ ‖A
−1‖
‖δ~b‖‖~x + δ~x‖ + ‖δA‖
!.
Result
‖δ~x‖‖~x + δ~x‖ ≤ Cond(A)
‖δ~b‖
‖A‖‖~x + δ~x‖ +‖δA‖‖A‖
!.
relative error on x = Cond(A) (relative error on ~b + relative error on A).
![Page 15: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/15.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Hermitian, orthogonal. . .
Transposed matrix: (tA)ij = Aji .Adjoint matrix: (A∗)ij = Aji .
Symmetric matrix
tA = A.
Hermitian matrix
A∗ = A and hence tA = A.
Orthogonal matrix (in Mnn(R))
tAA = I .
Unitary matrix (in Mnn(C))
A∗A = I .
Similar matrices (”semblables” in French)
A and B are similar if ∃P/B = P−1AP.
![Page 16: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/16.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Hermitian, orthogonal. . .
Transposed matrix: (tA)ij = Aji .Adjoint matrix: (A∗)ij = Aji .
Symmetric matrix
tA = A.
Hermitian matrix
A∗ = A and hence tA = A.
Orthogonal matrix (in Mnn(R))
tAA = I .
Unitary matrix (in Mnn(C))
A∗A = I .
Similar matrices (”semblables” in French)
A and B are similar if ∃P/B = P−1AP.
![Page 17: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/17.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Hermitian, orthogonal. . .
Transposed matrix: (tA)ij = Aji .Adjoint matrix: (A∗)ij = Aji .
Symmetric matrix
tA = A.
Hermitian matrix
A∗ = A and hence tA = A.
Orthogonal matrix (in Mnn(R))
tAA = I .
Unitary matrix (in Mnn(C))
A∗A = I .
Similar matrices (”semblables” in French)
A and B are similar if ∃P/B = P−1AP.
![Page 18: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/18.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Hermitian, orthogonal. . .
Transposed matrix: (tA)ij = Aji .Adjoint matrix: (A∗)ij = Aji .
Symmetric matrix
tA = A.
Hermitian matrix
A∗ = A and hence tA = A.
Orthogonal matrix (in Mnn(R))
tAA = I .
Unitary matrix (in Mnn(C))
A∗A = I .
Similar matrices (”semblables” in French)
A and B are similar if ∃P/B = P−1AP.
![Page 19: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/19.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Hermitian, orthogonal. . .
Transposed matrix: (tA)ij = Aji .Adjoint matrix: (A∗)ij = Aji .
Symmetric matrix
tA = A.
Hermitian matrix
A∗ = A and hence tA = A.
Orthogonal matrix (in Mnn(R))
tAA = I .
Unitary matrix (in Mnn(C))
A∗A = I .
Similar matrices (”semblables” in French)
A and B are similar if ∃P/B = P−1AP.
![Page 20: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/20.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Profiles
Lower triangular Upper triangular Tridiagonal
Lower Hessenberg Upper Hessenberg
![Page 21: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/21.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Householder matrices
Definition
H~v = I − 2~v t~v
‖~v‖22
Properties
1 H~v is orthogonal.
2 If ~v = ~a− ~b 6= ~0 and ‖~a‖2 = ‖~b‖2,
then H~v~a = ~b.
t~v~v = ‖~a‖2 − 2t~a~b + ‖~b‖2 = 2‖~a‖2 − 2t~a~b = 2t~a~v = 2t~v~a
H~v~a = ~a− 2~v t~v~a‖~v‖2
= ~a− ~v = ~b.
Application
Let ~a ∈ K n, we look for H~v such that H~v~a = t(α, 0, . . . , 0).
Solution: take ~b = t(α, 0, . . . , 0) with α = ‖~a‖2, and ~v = ~a− ~b. Then
H~v~a = ~b.
![Page 22: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/22.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Householder matrices
Definition
H~v = I − 2~v t~v
‖~v‖22
Properties
1 H~v is orthogonal.
2 If ~v = ~a− ~b 6= ~0 and ‖~a‖2 = ‖~b‖2,
then H~v~a = ~b.
t~v~v = ‖~a‖2 − 2t~a~b + ‖~b‖2 = 2‖~a‖2 − 2t~a~b = 2t~a~v = 2t~v~a
H~v~a = ~a− 2~v t~v~a‖~v‖2
= ~a− ~v = ~b.
Application
Let ~a ∈ K n, we look for H~v such that H~v~a = t(α, 0, . . . , 0).
Solution: take ~b = t(α, 0, . . . , 0) with α = ‖~a‖2, and ~v = ~a− ~b. Then
H~v~a = ~b.
![Page 23: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/23.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Householder matrices
Definition
H~v = I − 2~v t~v
‖~v‖22
Properties
1 H~v is orthogonal.
2 If ~v = ~a− ~b 6= ~0 and ‖~a‖2 = ‖~b‖2,
then H~v~a = ~b.
t~v~v = ‖~a‖2 − 2t~a~b + ‖~b‖2 = 2‖~a‖2 − 2t~a~b = 2t~a~v = 2t~v~a
H~v~a = ~a− 2~v t~v~a‖~v‖2
= ~a− ~v = ~b.
Application
Let ~a ∈ K n, we look for H~v such that H~v~a = t(α, 0, . . . , 0).
Solution: take ~b = t(α, 0, . . . , 0) with α = ‖~a‖2, and ~v = ~a− ~b. Then
H~v~a = ~b.
![Page 24: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/24.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Householder matrices
Definition
H~v = I − 2~v t~v
‖~v‖22
Properties
1 H~v is orthogonal.
2 If ~v = ~a− ~b 6= ~0 and ‖~a‖2 = ‖~b‖2,
then H~v~a = ~b.
t~v~v = ‖~a‖2 − 2t~a~b + ‖~b‖2 = 2‖~a‖2 − 2t~a~b = 2t~a~v = 2t~v~a
H~v~a = ~a− 2~v t~v~a‖~v‖2
= ~a− ~v = ~b.
Application
Let ~a ∈ K n, we look for H~v such that H~v~a = t(α, 0, . . . , 0).
Solution: take ~b = t(α, 0, . . . , 0) with α = ‖~a‖2, and ~v = ~a− ~b. Then
H~v~a = ~b.
![Page 25: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/25.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Householder matrices
Definition
H~v = I − 2~v t~v
‖~v‖22
Properties
1 H~v is orthogonal.
2 If ~v = ~a− ~b 6= ~0 and ‖~a‖2 = ‖~b‖2,
then H~v~a = ~b.
t~v~v = ‖~a‖2 − 2t~a~b + ‖~b‖2 = 2‖~a‖2 − 2t~a~b = 2t~a~v = 2t~v~a
H~v~a = ~a− 2~v t~v~a‖~v‖2
= ~a− ~v = ~b.
Application
Let ~a ∈ K n, we look for H~v such that H~v~a = t(α, 0, . . . , 0).
Solution: take ~b = t(α, 0, . . . , 0) with α = ‖~a‖2, and ~v = ~a− ~b. Then
H~v~a = ~b.
![Page 26: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/26.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Householder matrices
Definition
H~v = I − 2~v t~v
‖~v‖22
Properties
1 H~v is orthogonal.
2 If ~v = ~a− ~b 6= ~0 and ‖~a‖2 = ‖~b‖2,
then H~v~a = ~b.
t~v~v = ‖~a‖2 − 2t~a~b + ‖~b‖2 = 2‖~a‖2 − 2t~a~b = 2t~a~v = 2t~v~a
H~v~a = ~a− 2~v t~v~a‖~v‖2
= ~a− ~v = ~b.
Application
Let ~a ∈ K n, we look for H~v such that H~v~a = t(α, 0, . . . , 0).
Solution: take ~b = t(α, 0, . . . , 0) with α = ‖~a‖2, and ~v = ~a− ~b. Then
H~v~a = ~b.
![Page 27: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/27.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Householder tridiagonalisation
Aim
A: symmetric matrix.Construct a sequence A(1) = A, . . . ,A(n) tridiagonal and A(n)n = HAtH.
A(2) = A(3) = A(4) = A(5) = . . . A(n) =
First step
A(1) ≡
A(1)11
t~a(1)12
~a(1)21 A(1)
!H(1) ≡
„1 t~0~0 H(1)
«A(2) ≡
A
(1)11
t(H(1)~a(1)21 )
H(1)~a(1)21 H(1)A(1)t H(1)
!.
Choose H(1) such that H(1)~a(1)21 = t(α, 0, . . . , 0)n−1 = α(~e1)n−1.
α = ‖~a(1)21 ‖2, ~u1 = ~a
(1)21 − α(~e1)n−1, H(1) = H~u1 .
Complexity
Order2
3n3 products.
![Page 28: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/28.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Householder tridiagonalisation
Aim
A: symmetric matrix.Construct a sequence A(1) = A, . . . ,A(n) tridiagonal and A(n)n = HAtH.
A(2) = A(3) = A(4) = A(5) = . . . A(n) =
First step
A(1) ≡
A(1)11
t~a(1)12
~a(1)21 A(1)
!H(1) ≡
„1 t~0~0 H(1)
«A(2) ≡
A
(1)11
t(H(1)~a(1)21 )
H(1)~a(1)21 H(1)A(1)t H(1)
!.
Choose H(1) such that H(1)~a(1)21 = t(α, 0, . . . , 0)n−1 = α(~e1)n−1.
α = ‖~a(1)21 ‖2, ~u1 = ~a
(1)21 − α(~e1)n−1, H(1) = H~u1 .
Complexity
Order2
3n3 products.
![Page 29: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/29.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Householder tridiagonalisation
Aim
A: symmetric matrix.Construct a sequence A(1) = A, . . . ,A(n) tridiagonal and A(n)n = HAtH.
A(2) = A(3) = A(4) = A(5) = . . . A(n) =
First step
A(1) ≡
A(1)11
t~a(1)12
~a(1)21 A(1)
!H(1) ≡
„1 t~0~0 H(1)
«A(2) ≡
A
(1)11
t(H(1)~a(1)21 )
H(1)~a(1)21 H(1)A(1)t H(1)
!.
Choose H(1) such that H(1)~a(1)21 = t(α, 0, . . . , 0)n−1 = α(~e1)n−1.
α = ‖~a(1)21 ‖2, ~u1 = ~a
(1)21 − α(~e1)n−1, H(1) = H~u1 .
Complexity
Order2
3n3 products.
![Page 30: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/30.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Householder tridiagonalisation
Aim
A: symmetric matrix.Construct a sequence A(1) = A, . . . ,A(n) tridiagonal and A(n)n = HAtH.
A(2) = A(3) = A(4) = A(5) = . . . A(n) =
First step
A(1) ≡
A(1)11
t~a(1)12
~a(1)21 A(1)
!H(1) ≡
„1 t~0~0 H(1)
«A(2) ≡
A
(1)11
t(H(1)~a(1)21 )
H(1)~a(1)21 H(1)A(1)t H(1)
!.
Choose H(1) such that H(1)~a(1)21 = t(α, 0, . . . , 0)n−1 = α(~e1)n−1.
α = ‖~a(1)21 ‖2, ~u1 = ~a
(1)21 − α(~e1)n−1, H(1) = H~u1 .
Complexity
Order2
3n3 products.
![Page 31: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/31.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Givens tridiagonalization
Let Gpq(c, s) =
0BBBBBBBB@
1 01
c s1
−s c1
0 1
1CCCCCCCCAwith c2 + s2 = 1.
tGAG =If A is symmetric: else: leads to Hessenberg matrix
0
0
0
0
0
0
0
0
. . . 0
∗
0
∗
0
0
∗∗
Complexity
Order4
3n3 products.
![Page 32: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/32.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Principles of LU factorization
=
11
11
11
11
11
1 0
×
0
A L U
Some regular matrix (with non-zero determinant) are notLU-transformable, e.g. ([0 1; 1 1]) is not.
If it exists, the LU decomposition of A is not unique.It is unique if A is non-singular.
A is non-singular and LU-transformable⇐⇒ all the determinants of the fundamental principal minors arenon zero (and in this case the decomposition is unique).
![Page 33: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/33.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Principles of LU factorization
=
11
11
11
11
11
1 0
×
0
A L U
Some regular matrix (with non-zero determinant) are notLU-transformable, e.g. ([0 1; 1 1]) is not.
If it exists, the LU decomposition of A is not unique.It is unique if A is non-singular.
A is non-singular and LU-transformable⇐⇒ all the determinants of the fundamental principal minors arenon zero (and in this case the decomposition is unique).
![Page 34: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/34.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Principles of LU factorization
=
11
11
11
11
11
1 0
×
0
A L U
Some regular matrix (with non-zero determinant) are notLU-transformable, e.g. ([0 1; 1 1]) is not.
If it exists, the LU decomposition of A is not unique.It is unique if A is non-singular.
A is non-singular and LU-transformable⇐⇒ all the determinants of the fundamental principal minors arenon zero (and in this case the decomposition is unique).
![Page 35: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/35.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Doolittle LU factorization – principle
It proceeds line by line.8>><>>:A11 = L11U11 L11 = 1A12 = L11U12
. . . ⇒ {U1j}j=1,...,n
A1n = L11U1n8>><>>:A21 = L21U11 ⇒ L21
A22 = L21U12 + U22
. . . ⇒ {U2j}j=2,...,n
A2n = L21U1n + U2n8>>>><>>>>:A31 = L31U11 ⇒ L31
A32 = L31U12 + L32U22 ⇒ L32
A33 = L31U13 + L32U23 + U33
. . . ⇒ {U3j}j=3,...,n
A3n = L31U1n + L32U2n + U3n
. . .
![Page 36: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/36.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Doolittle LU factorization – algorithm
Doolittle algorithm
Lij =
Aij −j−1Pk=1
LikUkj
UjjUij = Aij −
i−1Pk=1
LikUkj
f o r i = 1 to nf o r j = 1 to i−1
sum = 0f o r k=1 to j−1
sum = sum + L ( i , k )∗U( k , j )end f o rL ( i , j ) = (A( i , j )−sum )/U( j , j )
end f o rL ( i , i ) = 1f o r j = i to n
sum = 0f o r k = 1 to i−1
sum = sum + L ( i , k )∗U( k , j )end f o rU( i , j ) = A( i , j ) − sum
end f o rend f o r
Complexity
Order n3 products
![Page 37: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/37.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Cholesky factorization for an Hermitian matrix
Principle
A = C tC
Cholesky algorithm
Cii =
sAii −
i−1Pk=1
CikCik Cij =
Aij −j−1Pk=1
CikCjk
Cjj, j 6= i
C( 1 , 1 ) = s q r t (A( 1 , 1 ) )f o r i = 2 to n
f o r j = 1 to i−1sum = 0f o r k = 1 to j−1
sum = sum + C( i , k )∗C( j , k )end f o rC( i , j ) = (A( i , j )−sum )/C( j , j )
end f o rsum=0f o r k = 1 to i−1
sum = sum + C( i , k )∗C( i , k )end f o rC( i , i ) = s q r t (A( i , i ) − sum )
end f o r
Complexity
Order n3 products
![Page 38: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/38.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
LU factorization – profiles
Non-zero elements
In blue: AIn red: superposition of L and U
The interior of the profile is filled!
![Page 39: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/39.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
QR factorization – principle
Principle
A = QR, where Q orthogonal and R right (upper) triangular.Gm . . .G2G1A = R,A = tG1
tG2 . . .tGm| {z }
Q
R
Let Gij(c, s) =
0BBBBBBBB@
1 01
c s1
−s c1
0 1
1CCCCCCCCAwith c2 + s2 = 1.
× ×× ×
× =(i, j)
(GA)ij = −sAjj + cAij
c =Ajjq
A2ij + A2
jj
s =Aijq
A2ij + A2
jj
![Page 40: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/40.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
QR factorization – algorithm
Algorithm
R = AQ = I d // s i z e of Af o r i = 2 to n
f o r j = 1 to i−1r o o t = s q r t (R( i , j )∗R( i , j )+R( j , j )∗R( j , j ) )i f r o o t != 0
c = R( j , j )/ r o o ts = R( i , j )/ r o o t
e l s ec = 1s = 0
end i fC o n s t r u c t G j iR = G j i ∗R // m a t r i x p r o d u c tQ = Q∗ t r a n s p o s e ( G j i ) // m a t r i x p r o d u c t
end f o rend f o r
Complexity
Order n3 products
![Page 41: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/41.jpg)
Linear Algebra
Vectors and matrices
Elementary operations
Gram–Schmidtorthonormalization
Matrix norm
Conditioning
Specific matrices
Tridiagonalisation
LU and QRfactorizations
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
QR factorization – Python example
A =
0BBBB@3 2 1 0 04 3 2 1 05 4 3 2 16 5 4 3 27 6 5 4 3
1CCCCA
R =
[email protected] 9.467 7.316 5.164 3.271
3.437 10−16 6.086 10−01 1.217 1.826 1.7044.476 10−17 1.989 10−18 2.324 10−15 3.768 10−15 −3.775 10−01
−6.488 10−16 1.082 10−17 0.000 1.618 10−16 −6.764 10−02
−6.671 10−16 −2.548 10−17 0.000 −3.082 10−33 −5.029 10−01
1CCCCA
Q =
[email protected] −0.7303 −0.3775 −0.0676 −0.50290.3443 −0.4260 −0.0062 −0.1589 0.8210.4303 −0.1217 0.5407 0.7050 −0.10300.5164 0.1826 0.4472 −0.6627 −0.24660.6025 0.4869 −0.6042 0.1842 0.0311
1CCCCA
![Page 42: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/42.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Overview
1 Vectors and matricesElementary operationsGram–Schmidt orthonormalizationMatrix normConditioningSpecific matricesTridiagonalisationLU and QR factorizations
2 Eigenvalues and eigenvectorsPower iteration algorithmDeflationGalerkinJacobiQR
3 Numerical solution of linear systemsDirect methodsIterative methodsPreconditioning
4 StorageBand storageSparse storage
5 Bandwidth reductionCuthill–McKee algorithm
6 References
![Page 43: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/43.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Power iteration algorithm – Python experiment
A =
„10 0−9 1
«Eigenvalues and eigenvectors:
λ1 = 1, λ2 = 10, ~v1 =
„01
«, ~v2 =
„1−1
«.
Construct the series~xk = A~xk−1
~x0 =
„21
«, ~x1 =
„20−17
«, ~x2 =
„200−197
«, ~x3 =
„2000−1997
«. . .
~x tends to the direction of the eigenvector associated to the highermodulus eigenvalue.
”~xk/~xk−1” tends to the higher modulus eigenvalue.
![Page 44: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/44.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Power iteration algorithm – Algorithm
Computation of the eigenvalue with higher modulus.A may be diagonalizable or not, the dominant eigenvalue can be uniqueor not.
Algorithm
choose q ( 0 )f o r k = 1 to c o n v e r g e n c e
x ( k ) = A ∗ q ( k−1)q ( k ) = x ( k ) / norm ( x ( k ) )
end f o rlambdamax = x ( k ) ( j )/ q ( k−1)( j )
Attention: good choice of component j .
![Page 45: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/45.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Power iteration algorithm – Python example
A =
0@10 0 00 5 00 0 2
1ARotations:
R1 =
0@ cos(1) 0 sin(1)0 1 0
− sin(1) 0 cos(1)
1A ,R2 =
0@1 0 00 cos(2) sin(2)0 − sin(2) cos(2)
1A
B = R2R1AtR1tR2 =
0@ 4.33541265 −3.30728724 1.51360499−3.30728724 7.20313893 −1.008283181.51360499 −1.00828318 5.46144841
1AEigenvalues and eigenvectors:
λ1 = 2, λ2 = 5, λ3 = 10,
~v1 =
0@−0.8415−0.49130.2248
1A , ~v2 =
[email protected] 10−16
0.41610.9093
1A , ~v3 =
0@−0.54030.7651−0.3502
1A .
![Page 46: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/46.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Power iteration algorithm – Remarks
1 Convergence results depend on the fact that
the matrix is diagonalizable or notthe dominant eigenvalue is multiple or not
2 The choice of the norm is not explicit: usually max norm oreuclidian norm
3 ~q0 should not be orthogonal to the eigen-subspace associated tothe dominant eigenvalue.
![Page 47: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/47.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Power iteration algorithm – Remarks
1 Convergence results depend on the fact thatthe matrix is diagonalizable or not
the dominant eigenvalue is multiple or not
2 The choice of the norm is not explicit: usually max norm oreuclidian norm
3 ~q0 should not be orthogonal to the eigen-subspace associated tothe dominant eigenvalue.
![Page 48: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/48.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Power iteration algorithm – Remarks
1 Convergence results depend on the fact thatthe matrix is diagonalizable or notthe dominant eigenvalue is multiple or not
2 The choice of the norm is not explicit: usually max norm oreuclidian norm
3 ~q0 should not be orthogonal to the eigen-subspace associated tothe dominant eigenvalue.
![Page 49: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/49.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Power iteration algorithm – Remarks
1 Convergence results depend on the fact thatthe matrix is diagonalizable or notthe dominant eigenvalue is multiple or not
2 The choice of the norm is not explicit: usually max norm oreuclidian norm
3 ~q0 should not be orthogonal to the eigen-subspace associated tothe dominant eigenvalue.
![Page 50: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/50.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Power iteration algorithm – Remarks
1 Convergence results depend on the fact thatthe matrix is diagonalizable or notthe dominant eigenvalue is multiple or not
2 The choice of the norm is not explicit: usually max norm oreuclidian norm
3 ~q0 should not be orthogonal to the eigen-subspace associated tothe dominant eigenvalue.
![Page 51: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/51.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Inverse iteration algorithm – Algorithm
Computation of the eigenvalue with smallest modulus.A may be diagonalizable or not, the dominant eigenvalue can be uniqueor not.Based on the fact that
λmin(A) =“λmax(A−1)
”−1
.
Algorithm
choose q ( 0 )f o r k = 1 to c o n v e r g e n c e
s o l v e A ∗ x ( k ) = q ( k−1)q ( k ) = x ( k ) / norm ( x ( k ) )
end f o rlambdamin = q ( k−1)( j ) / x ( k ) ( j )
![Page 52: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/52.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Inverse iteration algorithm – Python example
A =
0@10 0 00 5 00 0 2
1ARotations:
R1 =
0@ cos(1) 0 sin(1)0 1 0
− sin(1) 0 cos(1)
1A ,R2 =
0@1 0 00 cos(2) sin(2)0 − sin(2) cos(2)
1A
B = R2R1AtR1tR2 =
0@ 4.33541265 −3.30728724 1.51360499−3.30728724 7.20313893 −1.008283181.51360499 −1.00828318 5.46144841
1AEigenvalues and eigenvectors:
λ1 = 2, λ2 = 5, λ3 = 10,
~v1 =
0@−0.8415−0.49130.2248
1A , ~v2 =
[email protected] 10−16
0.41610.9093
1A , ~v3 =
0@−0.54030.7651−0.3502
1A .
![Page 53: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/53.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Generalized inverse iteration algorithm – Algorithm
Computation of the closest eigenvalue to a given µ.The eigenvalues of A− µI are the λi − µ,where λi are the eigenvalues of A.⇒ apply the inverse iteration algorithm to A− µI .
Algorithm
choose q ( 0 )f o r k = 1 to c o n v e r g e n c e
s o l v e (A−mu∗ I ) ∗ x ( k ) = q ( k−1)q ( k ) = x ( k ) / norm ( x ( k ) )
end f o rlambda = q ( k−1)( j ) / x ( k ) ( j ) + mu
![Page 54: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/54.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Generalized inverse iteration algorithm – Python example
A =
0@10 0 00 5 00 0 2
1A µ = 4.
Rotations:
R1 =
0@ cos(1) 0 sin(1)0 1 0
− sin(1) 0 cos(1)
1A ,R2 =
0@1 0 00 cos(2) sin(2)0 − sin(2) cos(2)
1A
B = R2R1AtR1tR2 =
0@ 4.33541265 −3.30728724 1.51360499−3.30728724 7.20313893 −1.008283181.51360499 −1.00828318 5.46144841
1AEigenvalues and eigenvectors:
λ1 = 2, λ2 = 5, λ3 = 10,
~v1 =
0@−0.8415−0.49130.2248
1A , ~v2 =
[email protected] 10−16
0.41610.9093
1A , ~v3 =
0@−0.54030.7651−0.3502
1A .
![Page 55: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/55.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Deflation – Algorithm and Python example
Computation of all the eigenvalues in modulus decreasing order.
When an eigenelement (λ, q) is found, it is removed from furthercomputation by replacing A← A− λ~qt~q.
Algorithm
f o r i = 1 to nchoose q ( 0 )f o r k = 1 to c o n v e r g e n c e
x ( k ) = A ∗ q ( k−1)q ( k ) = x ( k ) / norm ( x ( k ) )
end f o rlambda = x ( k ) ( j ) / q ( k−1)( j )A = A − lambda ∗ q ∗ t r a n s p o s e ( q )
// e l i m i n a t e s d i r e c t i o n qend f o r
![Page 56: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/56.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Galerkin method – Algorithm
Let H be a subspace of dimension m, generated by the orthonormalbasis (~q1, . . . , ~qm).Construct the rectangular matrix Q = (~q1, . . . , ~qm).Remark: Q∗Q = Idm
Goal
Look for eigenvectors in H.
If ~u ∈ H, ~u =mP
i=1
αi~qi (unique).
~u = Q~U, where ~U = t(α1, . . . , αm).
A~u = λ~u ⇔ AQ~U = λQ~U.Project on H: Q∗AQ~U = λQ∗Q~U = λ~U.
⇒ We look for eigenelements of B = Q∗AQ.
Vocabulary:• {λi , ~ui} are the Ritz elements,• B is the Rayleigh matrix.
![Page 57: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/57.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Jacobi method – Algorithm
Goal
Diagonalize the (real symmetric) matrix.
Until a ”reasonably diagonal” matrix is obtained:
Choose the largest off-diagonal element (largest modulus)
Construct a rotation matrix that annihilates this term
In the end, the eigenvalues are the diagonal elements.
![Page 58: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/58.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Jacobi method – Algorithm
Goal
Diagonalize the (real symmetric) matrix.
Until a ”reasonably diagonal” matrix is obtained:
Choose the largest off-diagonal element (largest modulus)
Construct a rotation matrix that annihilates this term
In the end, the eigenvalues are the diagonal elements.
![Page 59: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/59.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Jacobi method – Algorithm
Goal
Diagonalize the (real symmetric) matrix.
Until a ”reasonably diagonal” matrix is obtained:
Choose the largest off-diagonal element (largest modulus)
Construct a rotation matrix that annihilates this term
In the end, the eigenvalues are the diagonal elements.
![Page 60: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/60.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Jacobi method – Algorithm
Goal
Diagonalize the (real symmetric) matrix.
Until a ”reasonably diagonal” matrix is obtained:
Choose the largest off-diagonal element (largest modulus)
Construct a rotation matrix that annihilates this term
In the end, the eigenvalues are the diagonal elements.
![Page 61: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/61.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
QR method – Algorithm
Algorithm
A( 1 ) = Af o r k = 1 to c o n v e r g e n c e
[Q( k ) ,R( k ) ] = Q R f a c t o r (A( k ) )A( k+1) = R( k )∗Q( k )
end f o r
The eigenvalues are the diagonal elements of the last matrix Ak+1.
Properties
Ak+1 = RkQk = Q∗k QkRkQk = Q∗k AkQk
⇒ Ak+1 and Ak are similar.
If Ak is tridiagonal or Hessenberg, Ak+1 also is⇒ First restrict to this case keeping similar matrices.
![Page 62: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/62.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
QR method – Convergence and Python example
Theorem
Let V ∗ be the matrix of left eigenvectors of A (A∗~u∗ = λ~u∗).If• the principal minors of V are non-zero.• the eigen-values of A are such that |λ1| > · · · > |λn|.Then the QR method converges Ak+1 tends to an upper triangular formand (Ak)ii tends to λi .
![Page 63: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/63.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Eigenvalues – Summary
We want to know all the eigenvalues
QR method — better than JacobiPreprocessing: find a similar tridiagonal or Heisenberg matrix(Householder or Givens algorithm).
We only want one eigenvector whose eigenvalue is known (or anapproximation)
Power iteration algorithm and variants. . .
We only want a sub-set of eigenelements
We know the eigenvalues and look for eigenvectors: deflation andvariants
We know the subspace for eigenvectors: Galerkin and variants
![Page 64: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/64.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Eigenvalues – Summary
We want to know all the eigenvalues
QR method — better than JacobiPreprocessing: find a similar tridiagonal or Heisenberg matrix(Householder or Givens algorithm).
We only want one eigenvector whose eigenvalue is known (or anapproximation)
Power iteration algorithm and variants. . .
We only want a sub-set of eigenelements
We know the eigenvalues and look for eigenvectors: deflation andvariants
We know the subspace for eigenvectors: Galerkin and variants
![Page 65: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/65.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Eigenvalues – Summary
We want to know all the eigenvalues
QR method — better than JacobiPreprocessing: find a similar tridiagonal or Heisenberg matrix(Householder or Givens algorithm).
We only want one eigenvector whose eigenvalue is known (or anapproximation)
Power iteration algorithm and variants. . .
We only want a sub-set of eigenelements
We know the eigenvalues and look for eigenvectors: deflation andvariants
We know the subspace for eigenvectors: Galerkin and variants
![Page 66: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/66.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Eigenvalues – Summary
We want to know all the eigenvalues
QR method — better than JacobiPreprocessing: find a similar tridiagonal or Heisenberg matrix(Householder or Givens algorithm).
We only want one eigenvector whose eigenvalue is known (or anapproximation)
Power iteration algorithm and variants. . .
We only want a sub-set of eigenelements
We know the eigenvalues and look for eigenvectors: deflation andvariants
We know the subspace for eigenvectors: Galerkin and variants
![Page 67: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/67.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Eigenvalues – Summary
We want to know all the eigenvalues
QR method — better than JacobiPreprocessing: find a similar tridiagonal or Heisenberg matrix(Householder or Givens algorithm).
We only want one eigenvector whose eigenvalue is known (or anapproximation)
Power iteration algorithm and variants. . .
We only want a sub-set of eigenelements
We know the eigenvalues and look for eigenvectors: deflation andvariants
We know the subspace for eigenvectors: Galerkin and variants
![Page 68: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/68.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Eigenvalues – Summary
We want to know all the eigenvalues
QR method — better than JacobiPreprocessing: find a similar tridiagonal or Heisenberg matrix(Householder or Givens algorithm).
We only want one eigenvector whose eigenvalue is known (or anapproximation)
Power iteration algorithm and variants. . .
We only want a sub-set of eigenelements
We know the eigenvalues and look for eigenvectors: deflation andvariants
We know the subspace for eigenvectors: Galerkin and variants
![Page 69: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/69.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Power iterationalgorithm
Deflation
Galerkin
Jacobi
QR
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Eigenvalues – Summary
We want to know all the eigenvalues
QR method — better than JacobiPreprocessing: find a similar tridiagonal or Heisenberg matrix(Householder or Givens algorithm).
We only want one eigenvector whose eigenvalue is known (or anapproximation)
Power iteration algorithm and variants. . .
We only want a sub-set of eigenelements
We know the eigenvalues and look for eigenvectors: deflation andvariants
We know the subspace for eigenvectors: Galerkin and variants
![Page 70: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/70.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Overview
1 Vectors and matricesElementary operationsGram–Schmidt orthonormalizationMatrix normConditioningSpecific matricesTridiagonalisationLU and QR factorizations
2 Eigenvalues and eigenvectorsPower iteration algorithmDeflationGalerkinJacobiQR
3 Numerical solution of linear systemsDirect methodsIterative methodsPreconditioning
4 StorageBand storageSparse storage
5 Bandwidth reductionCuthill–McKee algorithm
6 References
![Page 71: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/71.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Principles
A~x = ~b
Elimination methods
The solution to the system remains unchanged if
lines are permuted,
line i is replaced by a linear combination
`i ←nP
k=1
µk`k , with µi 6= 0.
Factorisation methods
A = LULU~x = ~bWe solve two triangular systemsL~y = ~bU~x = ~y .
![Page 72: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/72.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Principles
A~x = ~b
Elimination methods
The solution to the system remains unchanged if
lines are permuted,
line i is replaced by a linear combination
`i ←nP
k=1
µk`k , with µi 6= 0.
Factorisation methods
A = LULU~x = ~bWe solve two triangular systemsL~y = ~bU~x = ~y .
![Page 73: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/73.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Principles
A~x = ~b
Elimination methods
The solution to the system remains unchanged if
lines are permuted,
line i is replaced by a linear combination
`i ←nP
k=1
µk`k , with µi 6= 0.
Factorisation methods
A = LULU~x = ~bWe solve two triangular systemsL~y = ~bU~x = ~y .
![Page 74: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/74.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Principles
A~x = ~b
Elimination methods
The solution to the system remains unchanged if
lines are permuted,
line i is replaced by a linear combination
`i ←nP
k=1
µk`k , with µi 6= 0.
Factorisation methods
A = LULU~x = ~bWe solve two triangular systemsL~y = ~bU~x = ~y .
![Page 75: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/75.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Lower triangular matrix
xi =
bi −i−1Pk=1
Aikxk
Aii
Algorithm
i f A(1 ,1)==0 then s t o px ( 1 ) = b ( 1 ) /A( 1 , 1 )f o r i = 2 to n
i f A( i , i )==0 then s t o pax = 0f o r k = 1 to i−1
ax = ax + A( i , k )∗ x ( k )end f o rx ( i ) = ( b ( i )−ax )/A( i , i )
end f o r
Complexity
Order n2/2 products.
![Page 76: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/76.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Upper triangular matrix
xi =
bi −nP
k=i+1
Aikxk
Aii
Algorithm
i f A( n , n)==0 then s t o px ( n ) = b ( n )/A( n , n )f o r i = n−1 to 1
i f A( i , i )==0 then s t o pax = 0f o r k = i +1 to n
ax = ax + A( i , k )∗ x ( k )end f o rx ( i ) = ( b ( i )−ax )/A( i , i )
end f o r
Complexity
Order n2/2 products.
![Page 77: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/77.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Gauss elimination – Principle
Aim
Transform A to upper triangular matrix.
At rank p − 1:
Aij = 0 if i > j , j < p.
`i ← `i − Aip`p
App
f o r p = 1 to np i v o t = A( p , p )i f p i v o t == 0 s t o pl i n e ( p ) = l i n e ( p )/ p i v o tf o r i = p+1 to n
Aip = A( i , p )l i n e ( i ) = l i n e ( i ) − Aip ∗ l i n e ( p )
end f o rend f o rx = s o l v e (A, b ) // upper t r i a n g u l a r
Complexity
Still order n3 products.
![Page 78: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/78.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Gauss–Jordan elimination – Principle
Aim
Transform A to identity.
At rank p − 1:
Aii = 1 if i < p,
Aij = 0 if i 6= j , j < p.
`i ← `i − Aip`p
App
f o r p = 1 to np i v o t = A( p , p )i f p i v o t == 0 s t o pl i n e ( p ) = l i n e ( p )/ p i v o tf o r i = 1 to n , i !=p
Aip = A( i , p )l i n e ( i ) = l i n e ( i ) − Aip ∗ l i n e ( p )
end f o rend f o rx = b
Attention
take into account le right-hand side in the ”line”.
what if App = 0?
![Page 79: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/79.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Gauss–Jordan elimination – Algorithm
// unknown e n t r i e s number ingf o r i = 1 to n
num( i ) = iend f o r
f o r p=1 to n// maximal p i v o tpmax = abs (A( p , p ) )imax = pjmax = pf o r i = p to n
f o r j = p to ni f abs (A( i , j ) ) > pmax then
pmax = abs (A( i , j ) )imax = ijmax = j
end i fend f o r
end f o r// l i n e p e r m u t a t i o nf o r j = p to n
permute (A( p , j ) ,A( imax , j )end f o rpermute ( b ( p ) , b ( imax ) )// column p e r m u t a t i o nf o r i = p to n
permute (A( i , p ) ,A( i , jmax )end f o rpermute (num( p ) , num( jmax ) )
p i v o t = A( p , p )i f p i v o t == 0 stop , rank (A) = p−1f o r j = p to n
A( p , j ) = A( p , j )/ p i v o tend f o rb ( p ) = b ( p )/ p i v o tf o r i = 1 to n , i !=p
Aip = A( i , p )f o r j = p to n
A( i , j ) = A( i , j ) − Aip ∗ A( p , j )end f o rb ( i ) = b ( i ) − Aip∗b ( p )
end f o rend f o r // l o o p on p
f o r i = 1 to nx (num( i ) ) = b ( i )
end f o r
Complexity
Order n3 products.
Remark
Also computes the rank of thematrix.
![Page 80: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/80.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Factorization methods — Thomas algorithm — principle
LU decomposition for tridiagonal matrices.
=
11
11
11
11
11 ×
11
11
11
11
11
A L UWe suppose that Lij and Uij are known for i < p. Then
Ap,p−1 = Lp,p−1Up−1,p−1,
Ap,p = Lp,p−1Up−1,p + Up,p,
Ap,p+1 = Up,p+1.
⇒
Lp,p−1 = Ap,p−1/Up−1,p−1,
Up,p = Ap,p − Lp,p−1Up−1,p = Ap,p − Ap,p−1Up−1,p/Up−1,p−1,
Up,p+1 = Ap,p+1.
![Page 81: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/81.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Factorization methods — Thomas algorithm — algorithm
Algorithmm
// f a c t o r i z a t i o nU( 1 , 1 ) = A( 1 , 1 )U( 1 , 2 ) = A( 1 , 2 )f o r i = 2 to n
i f U( i −1, i −1) = 0 then s t o pL ( i , i −1) = A( i , i −1)/U( i −1, i −1)U( i , i ) = A( i , i ) − L ( i , i −1)∗U( i −1, i )U( i , i +1) = A( i , i +1)
end f o r// c o n s t r u c t i o n of t h e s o l u t i o ny = s o l v e ( L , b ) // l o w e r t r i a n g u l a rx = s o l v e (U, y ) // upper t r i a n g u l a r
Complexity
Order 5n products.
![Page 82: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/82.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Factorization methods — general matrices
For general matrices:
Factorize the matrix
LU algorithmCholeski algorithm
Solve upper triangular system
Solve lower triangular system.
![Page 83: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/83.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Factorization methods — general matrices
For general matrices:
Factorize the matrix
LU algorithm
Choleski algorithm
Solve upper triangular system
Solve lower triangular system.
![Page 84: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/84.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Factorization methods — general matrices
For general matrices:
Factorize the matrix
LU algorithmCholeski algorithm
Solve upper triangular system
Solve lower triangular system.
![Page 85: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/85.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Factorization methods — general matrices
For general matrices:
Factorize the matrix
LU algorithmCholeski algorithm
Solve upper triangular system
Solve lower triangular system.
![Page 86: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/86.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Factorization methods — general matrices
For general matrices:
Factorize the matrix
LU algorithmCholeski algorithm
Solve upper triangular system
Solve lower triangular system.
![Page 87: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/87.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Iterative methods — Principle
A= E + D + F
To solve A~x = ~b, write A = M − Nand iterate M~xk+1 − N~xk = ~b, i.e. ~xk+1 = M−1N~xk + M−1~b.
Attention
M should be easy to invert.
M−1N should lead to a stable algorithm.
Jacobi M = D, N = −(E + F ),
Gauss–Seidel M = D + E , N = −F ,
Successive Over Relaxation M =D
ω+ E , N =
„1
ω− 1
«D − F .
![Page 88: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/88.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Iterative methods — Principle
A= E + D + F
To solve A~x = ~b, write A = M − Nand iterate M~xk+1 − N~xk = ~b, i.e. ~xk+1 = M−1N~xk + M−1~b.
Attention
M should be easy to invert.
M−1N should lead to a stable algorithm.
Jacobi M = D, N = −(E + F ),
Gauss–Seidel M = D + E , N = −F ,
Successive Over Relaxation M =D
ω+ E , N =
„1
ω− 1
«D − F .
![Page 89: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/89.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Iterative methods — Principle
A= E + D + F
To solve A~x = ~b, write A = M − Nand iterate M~xk+1 − N~xk = ~b, i.e. ~xk+1 = M−1N~xk + M−1~b.
Attention
M should be easy to invert.
M−1N should lead to a stable algorithm.
Jacobi M = D, N = −(E + F ),
Gauss–Seidel M = D + E , N = −F ,
Successive Over Relaxation M =D
ω+ E , N =
„1
ω− 1
«D − F .
![Page 90: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/90.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Iterative methods — Principle
A= E + D + F
To solve A~x = ~b, write A = M − Nand iterate M~xk+1 − N~xk = ~b, i.e. ~xk+1 = M−1N~xk + M−1~b.
Attention
M should be easy to invert.
M−1N should lead to a stable algorithm.
Jacobi M = D, N = −(E + F ),
Gauss–Seidel M = D + E , N = −F ,
Successive Over Relaxation M =D
ω+ E , N =
„1
ω− 1
«D − F .
![Page 91: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/91.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Iterative methods — Principle
A= E + D + F
To solve A~x = ~b, write A = M − Nand iterate M~xk+1 − N~xk = ~b, i.e. ~xk+1 = M−1N~xk + M−1~b.
Attention
M should be easy to invert.
M−1N should lead to a stable algorithm.
Jacobi M = D, N = −(E + F ),
Gauss–Seidel M = D + E , N = −F ,
Successive Over Relaxation M =D
ω+ E , N =
„1
ω− 1
«D − F .
![Page 92: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/92.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Iterative methods — Principle
A= E + D + F
To solve A~x = ~b, write A = M − Nand iterate M~xk+1 − N~xk = ~b, i.e. ~xk+1 = M−1N~xk + M−1~b.
Attention
M should be easy to invert.
M−1N should lead to a stable algorithm.
Jacobi M = D, N = −(E + F ),
Gauss–Seidel M = D + E , N = −F ,
Successive Over Relaxation M =D
ω+ E , N =
„1
ω− 1
«D − F .
![Page 93: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/93.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Jacobi method
Algorithm
choose x ( k=0)f o r k = 0 to c o n v e r g e n c e
f o r i = 1 to nr h s = b ( i )f o r j = 1 to n , j != i
r h s = r h s − A( i , j )∗ x ( j , k )end f o rx ( i , k+1) = r h s / A( i , i )
end f o rt e s t = norm ( x ( k+1)−x ( k))< e p s i l o n
end f o r ( wh i l e not t e s t )
xk+1i =
1
Aii
0@bi −nX
j=1,j 6=i
Aijxkj
1A~xk+1 = D−1(~b − (E + F )~xk)
= D−1(~b + (D − A)~xk)
= D−1~b + (I − D−1A)~xk .
Remarks
simple,
two copies of the variable ~xk+1
and ~xk ,
insensible to permutations,
converges if the diagonal isstrictly dominant.
![Page 94: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/94.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Gauss–Seidel method
Algorithm
choose x ( k=0)f o r k = 0 to c o n v e r g e n c e
f o r i = 1 to nr h s = b ( i )f o r j = 1 to i−1
r h s = r h s − A( i , j )∗ x ( j , k+1)end f o rf o r j = i +1 to n
r h s = r h s − A( i , j )∗ x ( j , k )end f o rx ( i , k+1) = r h s / A( i , i )
end f o rt e s t = norm ( x ( k+1)−x ( k))< e p s i l o n
end f o r ( wh i l e not t e s t )
Remarks
still simple,
one copy of the variable ~x ,
sensible to permutations,
often converges better thanJacobi.
xk+1i =
1
Aii
bi −
i−1Xj=1
Aijxk+1j −
nXj=i+1
Aijxkj
!
![Page 95: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/95.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
SOR method
xk+1i =
ω
Aii
bi −
i−1Xj=1
Aijxk+1j −
nXj=i+1
Aijxkj
!+ (1− ω)xk
i
~xk+1 =
„D
ω+ E
«−1
~b +
„D
ω+ E
«−1 »„1
ω− 1
«D − F
–~xk
Remarks
still simple,
one copy of the variable ~x ,
Necessary condition for convergence: 0 < ω < 2.
![Page 96: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/96.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
SOR method
xk+1i =
ω
Aii
bi −
i−1Xj=1
Aijxk+1j −
nXj=i+1
Aijxkj
!+ (1− ω)xk
i
~xk+1 =
„D
ω+ E
«−1
~b +
„D
ω+ E
«−1 »„1
ω− 1
«D − F
–~xk
Remarks
still simple,
one copy of the variable ~x ,
Necessary condition for convergence: 0 < ω < 2.
![Page 97: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/97.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
SOR method
xk+1i =
ω
Aii
bi −
i−1Xj=1
Aijxk+1j −
nXj=i+1
Aijxkj
!+ (1− ω)xk
i
~xk+1 =
„D
ω+ E
«−1
~b +
„D
ω+ E
«−1 »„1
ω− 1
«D − F
–~xk
Remarks
still simple,
one copy of the variable ~x ,
Necessary condition for convergence: 0 < ω < 2.
![Page 98: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/98.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
SOR method
xk+1i =
ω
Aii
bi −
i−1Xj=1
Aijxk+1j −
nXj=i+1
Aijxkj
!+ (1− ω)xk
i
~xk+1 =
„D
ω+ E
«−1
~b +
„D
ω+ E
«−1 »„1
ω− 1
«D − F
–~xk
Remarks
still simple,
one copy of the variable ~x ,
Necessary condition for convergence: 0 < ω < 2.
![Page 99: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/99.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
SOR method
xk+1i =
ω
Aii
bi −
i−1Xj=1
Aijxk+1j −
nXj=i+1
Aijxkj
!+ (1− ω)xk
i
~xk+1 =
„D
ω+ E
«−1
~b +
„D
ω+ E
«−1 »„1
ω− 1
«D − F
–~xk
Remarks
still simple,
one copy of the variable ~x ,
Necessary condition for convergence: 0 < ω < 2.
![Page 100: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/100.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
SOR method
xk+1i =
ω
Aii
bi −
i−1Xj=1
Aijxk+1j −
nXj=i+1
Aijxkj
!+ (1− ω)xk
i
~xk+1 =
„D
ω+ E
«−1
~b +
„D
ω+ E
«−1 »„1
ω− 1
«D − F
–~xk
Remarks
still simple,
one copy of the variable ~x ,
Necessary condition for convergence: 0 < ω < 2.
![Page 101: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/101.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — general principle
For A symmetric definite positive!!
Principle
Construct a series of approximations of the solution to the system
~xk+1 = ~xk + αk~pk ,
where ~pk descent direction and αk to be determined.
The solution ~x minimizes the functional J(~x) = t~xA~x − 2t~b~x .
∂J
∂xi(~x) =
∂
∂xi
0@Xj,k
xjAjkxk − 2X
j
bjxj
1A=
Xk
Aikxk +X
j
xjAji − 2bi
= 2“
A~x − ~b”
i,
∂J
∂xi(~x) = 0.
![Page 102: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/102.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — determining αk
~x also minimizes the functional E(~x) = t(~x − ~x)A~(~x −~x), and E(~x) = 0.For a given ~pk , which α minimizes E(~xk+1)?
E(~xk + α~pk) = t(~xk + α~pk − ~x)A(~xk + α~pk − ~x),
∂
∂αE(~xk + α~pk) = t~pkA(~xk + α~pk − ~x) + t(~xk + α~pk − ~x)A~pk
= 2t(~xk + α~pk − ~x)A~pk .
t(~xk + αk~pk − ~x)A~pk = 0t~xkA~pk + αk
t~pkA~pk − t~xA~pk = 0t~pkA~xk + αk t~pkA~pk − t~pkA~x = 0.
αk =t~pkA~xk − t~pkA~x
t~pkA~pk
![Page 103: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/103.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — functional profiles (good cases)
1.0 0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.01.5
1.0
0.5
0.0
0.5
1.0
1.5
2.0
2.5
A =
„2 00 2
«, ~b = A
„21
«Cond(A) = 1
A =
„2 1.5
1.5 2
«, ~b = A
„21
«Cond(A) = 7
0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.02.5
2.0
1.5
1.0
0.5
0.0
0.5
1.0
![Page 104: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/104.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — functional profiles (bad cases)
1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0
1.5
1.0
0.5
0.0
0.5
1.0
1.5
2.0
Nonpositive case
A =
„2 88 2
«, ~b = A
„21
«
Nonsymmetric case
A =
„2 −33 2
«, ~b = A
„21
«
1.0 0.5 0.0 0.5 1.0 1.5 2.0 2.5
2.0
1.5
1.0
0.5
0.0
0.5
1.0
1.5
![Page 105: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/105.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — optimal parameter (principle)
Principle
Choose ~pk = ~r k ≡ ~b − A~xk .
Choose αk is such that ~r k+1 is orthogonal to ~pk .
~r k+1 = ~b − A~xk+1 = ~b − A(~xk + α~pk) = ~r k − αkA~pk ,
0 = t~pk~r k+1 = t~pk~r k − αk t~pkA~pk .
αk =t~pk~r k
t~pkA~pk.
E(~xk+1) = (1− γk)E(~xk)
with γk =(t~pk~r k)2
(t~pkA~pk)(t~r kA−1~r k)≥ 1
Cond(A)
|t~pk~r k |‖~pk‖‖~r k‖ .
![Page 106: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/106.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — optimal parameter (principle)
Principle
Choose ~pk = ~r k ≡ ~b − A~xk .
Choose αk is such that ~r k+1 is orthogonal to ~pk .
~r k+1 = ~b − A~xk+1 = ~b − A(~xk + α~pk) = ~r k − αkA~pk ,
0 = t~pk~r k+1 = t~pk~r k − αk t~pkA~pk .
αk =t~pk~r k
t~pkA~pk.
E(~xk+1) = (1− γk)E(~xk)
with γk =(t~pk~r k)2
(t~pkA~pk)(t~r kA−1~r k)≥ 1
Cond(A)
|t~pk~r k |‖~pk‖‖~r k‖ .
![Page 107: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/107.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — optimal parameter (principle)
Principle
Choose ~pk = ~r k ≡ ~b − A~xk .
Choose αk is such that ~r k+1 is orthogonal to ~pk .
~r k+1 = ~b − A~xk+1 = ~b − A(~xk + α~pk) = ~r k − αkA~pk ,
0 = t~pk~r k+1 = t~pk~r k − αk t~pkA~pk .
αk =t~pk~r k
t~pkA~pk.
E(~xk+1) = (1− γk)E(~xk)
with γk =(t~pk~r k)2
(t~pkA~pk)(t~r kA−1~r k)≥ 1
Cond(A)
|t~pk~r k |‖~pk‖‖~r k‖ .
![Page 108: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/108.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — optimal parameter (principle)
Principle
Choose ~pk = ~r k ≡ ~b − A~xk .
Choose αk is such that ~r k+1 is orthogonal to ~pk .
~r k+1 = ~b − A~xk+1 = ~b − A(~xk + α~pk) = ~r k − αkA~pk ,
0 = t~pk~r k+1 = t~pk~r k − αk t~pkA~pk .
αk =t~pk~r k
t~pkA~pk.
E(~xk+1) = (1− γk)E(~xk)
with γk =(t~pk~r k)2
(t~pkA~pk)(t~r kA−1~r k)≥ 1
Cond(A)
|t~pk~r k |‖~pk‖‖~r k‖ .
![Page 109: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/109.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — optimal parameter (principle)
Principle
Choose ~pk = ~r k ≡ ~b − A~xk .
Choose αk is such that ~r k+1 is orthogonal to ~pk .
~r k+1 = ~b − A~xk+1 = ~b − A(~xk + α~pk) = ~r k − αkA~pk ,
0 = t~pk~r k+1 = t~pk~r k − αk t~pkA~pk .
αk =t~pk~r k
t~pkA~pk.
E(~xk+1) = (1− γk)E(~xk)
with γk =(t~pk~r k)2
(t~pkA~pk)(t~r kA−1~r k)≥ 1
Cond(A)
|t~pk~r k |‖~pk‖‖~r k‖ .
![Page 110: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/110.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — optimal parameter (principle)
Principle
Choose ~pk = ~r k ≡ ~b − A~xk .
Choose αk is such that ~r k+1 is orthogonal to ~pk .
~r k+1 = ~b − A~xk+1 = ~b − A(~xk + α~pk) = ~r k − αkA~pk ,
0 = t~pk~r k+1 = t~pk~r k − αk t~pkA~pk .
αk =t~pk~r k
t~pkA~pk.
E(~xk+1) = (1− γk)E(~xk)
with γk =(t~pk~r k)2
(t~pkA~pk)(t~r kA−1~r k)≥ 1
Cond(A)
|t~pk~r k |‖~pk‖‖~r k‖ .
![Page 111: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/111.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — optimal parameter (algorithm)
Algorithm
choose x ( k=1)f o r k = 1 to c o n v e r g e n c e
r ( k ) = b − A ∗ x ( k )p ( k ) = r ( k )a l p h a ( k ) = r ( k ) . p ( k ) / p ( k ) . A ∗ p ( k )x ( k+1) = x ( k ) + a l p h a ( k ) ∗ p ( k )
end f o r // r ( k ) s m a l l
![Page 112: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/112.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — conjugate gradient (principle)
Principle
Choose ~pk = ~r k + βk~pk−1.
Choose βk to minimize the error, i.e. maximize the factor γk
Properties
t~r k~pj = 0 ∀j < k,
Span(~r 1,~r 2, . . . ,~r k) = Span(~r 1,A~r 1, . . . ,Ak−1~r 1)
Span(~p1, ~p2, . . . , ~pk) = Span(~r 1,A~r 1, . . . ,Ak−1~r 1)t~pkA~pj = 0 ∀j < kt~r kA~pj = 0 ∀j < k
The algorithm converges in at most n iterations.
![Page 113: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/113.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — conjugate gradient (algorithm)
Algorithm
choose x ( k=1)p ( 1 ) = r ( 1 ) = b − A∗x ( 1 )f o r k = 1 to c o n v e r g e n c e
a l p h a ( k ) = r ( k ) . p ( k ) / p ( k ) . A ∗ p ( k )x ( k+1) = x ( k ) + a l p h a ( k ) ∗ p ( k )r ( k+1) = r ( k ) − a l p h a ( k ) ∗ A ∗ p ( k )b e t a ( k+1) = r ( k+1) . r ( k+1) / r ( k ) . r ( k )p ( k+1) = r ( k+1) + b e t a ( k+1) ∗ p ( k )
end f o r // r ( k ) s m a l l
![Page 114: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/114.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — GMRES
For generic matrices AGMRES: General Minimal RESidual method
Take a ”fair” approximation ~xk of the solution
Construct the m-dimensional set of free vectors
{~r k ,A~r k , . . . ,Am−1~r k}
This spans the Krylov space Hkm.
Construct an orthonormal basis for Hkm – e.g. via Gram-Schmidt
{~v1, . . . , ~vm}
Look for a new approximation ~xk+1 ∈ Hkm:
~xk+1 =mX
j=1
Xj~vj = [V ]~X
We obtain a system of n equations with m unknowns
A~xk+1 = A[V ]~X = ~b.
![Page 115: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/115.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — GMRES
For generic matrices AGMRES: General Minimal RESidual method
Take a ”fair” approximation ~xk of the solution
Construct the m-dimensional set of free vectors
{~r k ,A~r k , . . . ,Am−1~r k}
This spans the Krylov space Hkm.
Construct an orthonormal basis for Hkm – e.g. via Gram-Schmidt
{~v1, . . . , ~vm}
Look for a new approximation ~xk+1 ∈ Hkm:
~xk+1 =mX
j=1
Xj~vj = [V ]~X
We obtain a system of n equations with m unknowns
A~xk+1 = A[V ]~X = ~b.
![Page 116: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/116.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — GMRES
For generic matrices AGMRES: General Minimal RESidual method
Take a ”fair” approximation ~xk of the solution
Construct the m-dimensional set of free vectors
{~r k ,A~r k , . . . ,Am−1~r k}
This spans the Krylov space Hkm.
Construct an orthonormal basis for Hkm – e.g. via Gram-Schmidt
{~v1, . . . , ~vm}
Look for a new approximation ~xk+1 ∈ Hkm:
~xk+1 =mX
j=1
Xj~vj = [V ]~X
We obtain a system of n equations with m unknowns
A~xk+1 = A[V ]~X = ~b.
![Page 117: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/117.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — GMRES
For generic matrices AGMRES: General Minimal RESidual method
Take a ”fair” approximation ~xk of the solution
Construct the m-dimensional set of free vectors
{~r k ,A~r k , . . . ,Am−1~r k}
This spans the Krylov space Hkm.
Construct an orthonormal basis for Hkm – e.g. via Gram-Schmidt
{~v1, . . . , ~vm}
Look for a new approximation ~xk+1 ∈ Hkm:
~xk+1 =mX
j=1
Xj~vj = [V ]~X
We obtain a system of n equations with m unknowns
A~xk+1 = A[V ]~X = ~b.
![Page 118: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/118.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — GMRES
For generic matrices AGMRES: General Minimal RESidual method
Take a ”fair” approximation ~xk of the solution
Construct the m-dimensional set of free vectors
{~r k ,A~r k , . . . ,Am−1~r k}
This spans the Krylov space Hkm.
Construct an orthonormal basis for Hkm – e.g. via Gram-Schmidt
{~v1, . . . , ~vm}
Look for a new approximation ~xk+1 ∈ Hkm:
~xk+1 =mX
j=1
Xj~vj = [V ]~X
We obtain a system of n equations with m unknowns
A~xk+1 = A[V ]~X = ~b.
![Page 119: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/119.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — GMRES
For generic matrices AGMRES: General Minimal RESidual method
Take a ”fair” approximation ~xk of the solution
Construct the m-dimensional set of free vectors
{~r k ,A~r k , . . . ,Am−1~r k}
This spans the Krylov space Hkm.
Construct an orthonormal basis for Hkm – e.g. via Gram-Schmidt
{~v1, . . . , ~vm}
Look for a new approximation ~xk+1 ∈ Hkm:
~xk+1 =mX
j=1
Xj~vj = [V ]~X
We obtain a system of n equations with m unknowns
A~xk+1 = A[V ]~X = ~b.
![Page 120: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/120.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — GMRES (cont’d)
Project on Hkm
[tV ]A[V ]~X = [tV ]~b.
Solve this system of m equations with m unknowns
~xk+1 = [V ]~X .
and so on. . .
To work well GMRES should be preconditioned!
![Page 121: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/121.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — GMRES (cont’d)
Project on Hkm
[tV ]A[V ]~X = [tV ]~b.
Solve this system of m equations with m unknowns
~xk+1 = [V ]~X .
and so on. . .
To work well GMRES should be preconditioned!
![Page 122: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/122.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — GMRES (cont’d)
Project on Hkm
[tV ]A[V ]~X = [tV ]~b.
Solve this system of m equations with m unknowns
~xk+1 = [V ]~X .
and so on. . .
To work well GMRES should be preconditioned!
![Page 123: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/123.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — GMRES (cont’d)
Project on Hkm
[tV ]A[V ]~X = [tV ]~b.
Solve this system of m equations with m unknowns
~xk+1 = [V ]~X .
and so on. . .
To work well GMRES should be preconditioned!
![Page 124: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/124.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — GMRES (cont’d)
Project on Hkm
[tV ]A[V ]~X = [tV ]~b.
Solve this system of m equations with m unknowns
~xk+1 = [V ]~X .
and so on. . .
To work well GMRES should be preconditioned!
![Page 125: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/125.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Descent method — GMRES (cont’d)
Project on Hkm
[tV ]A[V ]~X = [tV ]~b.
Solve this system of m equations with m unknowns
~xk+1 = [V ]~X .
and so on. . .
To work well GMRES should be preconditioned!
![Page 126: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/126.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Preconditioning – principle
Principle
Replace system A~x = ~b by C−1A~x = C−1~bwhere Cond(C−1A)� Cond(A).
Which matrix C?
C should be easily invertible, typically the product of two triangularmatrices.
C = diag(A), simplest but well. . .
incomplete Cholesky or LU factorization
. . .
![Page 127: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/127.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Preconditioning – principle
Principle
Replace system A~x = ~b by C−1A~x = C−1~bwhere Cond(C−1A)� Cond(A).
Which matrix C?
C should be easily invertible, typically the product of two triangularmatrices.
C = diag(A), simplest but well. . .
incomplete Cholesky or LU factorization
. . .
![Page 128: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/128.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Preconditioning – principle
Principle
Replace system A~x = ~b by C−1A~x = C−1~bwhere Cond(C−1A)� Cond(A).
Which matrix C?
C should be easily invertible, typically the product of two triangularmatrices.
C = diag(A), simplest but well. . .
incomplete Cholesky or LU factorization
. . .
![Page 129: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/129.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Preconditioning – principle
Principle
Replace system A~x = ~b by C−1A~x = C−1~bwhere Cond(C−1A)� Cond(A).
Which matrix C?
C should be easily invertible, typically the product of two triangularmatrices.
C = diag(A), simplest but well. . .
incomplete Cholesky or LU factorization
. . .
![Page 130: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/130.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Preconditioning – principle
Principle
Replace system A~x = ~b by C−1A~x = C−1~bwhere Cond(C−1A)� Cond(A).
Which matrix C?
C should be easily invertible, typically the product of two triangularmatrices.
C = diag(A), simplest but well. . .
incomplete Cholesky or LU factorization
. . .
![Page 131: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/131.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Preconditioning – symmetry issues
Symmetry
Even if A and C are symmetric, C−1A may not be symmetric.What if symmetry is needed?
Let C−1/2 such that C−1/2C−1/2 = C−1.Then C−1/2AC−1/2 is similar to C−1A.
We consider the system
C +1/2(C−1A)C−1/2C +1/2~x = C +1/2C−1~b
(C−1/2AC−1/2)C +1/2~x = C−1/2~b
Solve(C−1/2AC−1/2)~y = C−1/2~b
and then~y = C +1/2~x .
![Page 132: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/132.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Preconditioning – symmetry issues
Symmetry
Even if A and C are symmetric, C−1A may not be symmetric.What if symmetry is needed?
Let C−1/2 such that C−1/2C−1/2 = C−1.Then C−1/2AC−1/2 is similar to C−1A.
We consider the system
C +1/2(C−1A)C−1/2C +1/2~x = C +1/2C−1~b
(C−1/2AC−1/2)C +1/2~x = C−1/2~b
Solve(C−1/2AC−1/2)~y = C−1/2~b
and then~y = C +1/2~x .
![Page 133: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/133.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Direct methods
Iterative methods
Preconditioning
Storage
Bandwidth reduction
References
Preconditioning – preconditioned conjugate gradient
Algorithm
choose x ( k=1)r ( 1 ) = b − A∗x ( 1 )s o l v e Cz ( 1 ) = r ( 1 )p ( 1 ) = r ( 1 )f o r k = 1 to c o n v e r g e n c e
a l p h a ( k ) = r ( k ) . z ( k ) / p ( k ) . A ∗ p ( k )x ( k+1) = x ( k ) + a l p h a ( k ) ∗ p ( k )r ( k+1) = r ( k ) − a l p h a ( k ) ∗ A ∗ p ( k )s o l v e C z ( k+1) = r ( k+1)b e t a ( k+1) = r ( k+1) . z ( k+1) / r ( k ) . z ( k )p ( k+1) = z ( k+1) + b e t a ( k+1) ∗ p ( k )
end f o r
At each iteration a system C~z = ~r is solved.
![Page 134: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/134.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Band storage
Sparse storage
Bandwidth reduction
References
Overview
1 Vectors and matricesElementary operationsGram–Schmidt orthonormalizationMatrix normConditioningSpecific matricesTridiagonalisationLU and QR factorizations
2 Eigenvalues and eigenvectorsPower iteration algorithmDeflationGalerkinJacobiQR
3 Numerical solution of linear systemsDirect methodsIterative methodsPreconditioning
4 StorageBand storageSparse storage
5 Bandwidth reductionCuthill–McKee algorithm
6 References
![Page 135: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/135.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Band storage
Sparse storage
Bandwidth reduction
References
Storage – main issues
Problems involve often a large number of variables, of degrees offreedom, say 106.
To store a full matrix for a 106-order system, 1012 real numbers (ifreal) are needed... In simple precision this necessitates 4 To ofmemory.
But high order problems are often very sparse.
We therefore use a storage structure which consists in only storingrelevant, non-zero, data.
Access to one element Aij should be very efficient.
![Page 136: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/136.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Band storage
Sparse storage
Bandwidth reduction
References
Storage – main issues
Problems involve often a large number of variables, of degrees offreedom, say 106.
To store a full matrix for a 106-order system, 1012 real numbers (ifreal) are needed... In simple precision this necessitates 4 To ofmemory.
But high order problems are often very sparse.
We therefore use a storage structure which consists in only storingrelevant, non-zero, data.
Access to one element Aij should be very efficient.
![Page 137: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/137.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Band storage
Sparse storage
Bandwidth reduction
References
Storage – main issues
Problems involve often a large number of variables, of degrees offreedom, say 106.
To store a full matrix for a 106-order system, 1012 real numbers (ifreal) are needed... In simple precision this necessitates 4 To ofmemory.
But high order problems are often very sparse.
We therefore use a storage structure which consists in only storingrelevant, non-zero, data.
Access to one element Aij should be very efficient.
![Page 138: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/138.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Band storage
Sparse storage
Bandwidth reduction
References
Storage – main issues
Problems involve often a large number of variables, of degrees offreedom, say 106.
To store a full matrix for a 106-order system, 1012 real numbers (ifreal) are needed... In simple precision this necessitates 4 To ofmemory.
But high order problems are often very sparse.
We therefore use a storage structure which consists in only storingrelevant, non-zero, data.
Access to one element Aij should be very efficient.
![Page 139: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/139.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Band storage
Sparse storage
Bandwidth reduction
References
Storage – main issues
Problems involve often a large number of variables, of degrees offreedom, say 106.
To store a full matrix for a 106-order system, 1012 real numbers (ifreal) are needed... In simple precision this necessitates 4 To ofmemory.
But high order problems are often very sparse.
We therefore use a storage structure which consists in only storingrelevant, non-zero, data.
Access to one element Aij should be very efficient.
![Page 140: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/140.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Band storage
Sparse storage
Bandwidth reduction
References
CDS: Compressed Diagonal Storage
L(A) = maxi
Li (A)
whereLi (A) = max
j/Aij 6=0|i − j |
![Page 141: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/141.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Band storage
Sparse storage
Bandwidth reduction
References
CRS: Compressed Row Storage
tabv
tabj
tabi
All the non-zero values of the matrix are stored in a table tab; theyare stored line by line in the increasing order of columns.
A table tabj, with same size than tab stores the column numberof the values in tabv.
A table tabi with size n + 1 stores the indices in tabj of the firstelement of each line. The last entry is the size of tabv.
CCS: Compressed Column Storage = Harwell Boeing
Generalization to symmetric matrices
![Page 142: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/142.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Band storage
Sparse storage
Bandwidth reduction
References
CRS: Compressed Row Storage
tabv
tabj
tabi
All the non-zero values of the matrix are stored in a table tab; theyare stored line by line in the increasing order of columns.
A table tabj, with same size than tab stores the column numberof the values in tabv.
A table tabi with size n + 1 stores the indices in tabj of the firstelement of each line. The last entry is the size of tabv.
CCS: Compressed Column Storage = Harwell Boeing
Generalization to symmetric matrices
![Page 143: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/143.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Band storage
Sparse storage
Bandwidth reduction
References
CRS: Compressed Row Storage
tabv
tabj
tabi
All the non-zero values of the matrix are stored in a table tab; theyare stored line by line in the increasing order of columns.
A table tabj, with same size than tab stores the column numberof the values in tabv.
A table tabi with size n + 1 stores the indices in tabj of the firstelement of each line. The last entry is the size of tabv.
CCS: Compressed Column Storage = Harwell Boeing
Generalization to symmetric matrices
![Page 144: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/144.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Band storage
Sparse storage
Bandwidth reduction
References
CRS: Compressed Row Storage
tabv
tabj
tabi
All the non-zero values of the matrix are stored in a table tab; theyare stored line by line in the increasing order of columns.
A table tabj, with same size than tab stores the column numberof the values in tabv.
A table tabi with size n + 1 stores the indices in tabj of the firstelement of each line. The last entry is the size of tabv.
CCS: Compressed Column Storage = Harwell Boeing
Generalization to symmetric matrices
![Page 145: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/145.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Band storage
Sparse storage
Bandwidth reduction
References
CRS: Compressed Row Storage
tabv
tabj
tabi
All the non-zero values of the matrix are stored in a table tab; theyare stored line by line in the increasing order of columns.
A table tabj, with same size than tab stores the column numberof the values in tabv.
A table tabi with size n + 1 stores the indices in tabj of the firstelement of each line. The last entry is the size of tabv.
CCS: Compressed Column Storage = Harwell Boeing
Generalization to symmetric matrices
![Page 146: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/146.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Band storage
Sparse storage
Bandwidth reduction
References
CRS: Compressed Row Storage – exercise
Question
A =
0BB@0 4 1 62 0 5 00 9 7 00 0 3 8
1CCACRS storage?
Solution
tabi = {1, 4, 6, 8, 10}tabj = {2, 3, 4, 1, 3, 2, 3, 3, 4}tabv = {4, 1, 6, 2, 5, 9, 7, 3, 8}
![Page 147: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/147.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Band storage
Sparse storage
Bandwidth reduction
References
CRS: Compressed Row Storage – exercise
Question
A =
0BB@0 4 1 62 0 5 00 9 7 00 0 3 8
1CCACRS storage?
Solution
tabi = {1, 4, 6, 8, 10}tabj = {2, 3, 4, 1, 3, 2, 3, 3, 4}tabv = {4, 1, 6, 2, 5, 9, 7, 3, 8}
![Page 148: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/148.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Overview
1 Vectors and matricesElementary operationsGram–Schmidt orthonormalizationMatrix normConditioningSpecific matricesTridiagonalisationLU and QR factorizations
2 Eigenvalues and eigenvectorsPower iteration algorithmDeflationGalerkinJacobiQR
3 Numerical solution of linear systemsDirect methodsIterative methodsPreconditioning
4 StorageBand storageSparse storage
5 Bandwidth reductionCuthill–McKee algorithm
6 References
![Page 149: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/149.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – construction of a graph
Goal
Reduce the bandwidth of a large sparse matrix by renumbering theunknowns.
Construction
The nodes of the graph are the unknowns ofthe system. They are labelled with a numberfrom 1 to n.
The edges are the relations between theunknowns. Two unknowns i and j are linked ifAij 6= 0.
The distance d(i, j) between two nodes is theminimal number of edges to follow to join bothnodes.
The excentricity E(i) = maxj d(i, j)
Far neighbors are λ(i) = {j/d(i, j) = E(i)}
Graph diameter D = maxi E(i)
Peripheral nodes P = {j/E(j) = D}.
![Page 150: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/150.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – construction of a graph
Goal
Reduce the bandwidth of a large sparse matrix by renumbering theunknowns.
Construction
The nodes of the graph are the unknowns ofthe system. They are labelled with a numberfrom 1 to n.
The edges are the relations between theunknowns. Two unknowns i and j are linked ifAij 6= 0.
The distance d(i, j) between two nodes is theminimal number of edges to follow to join bothnodes.
The excentricity E(i) = maxj d(i, j)
Far neighbors are λ(i) = {j/d(i, j) = E(i)}
Graph diameter D = maxi E(i)
Peripheral nodes P = {j/E(j) = D}.
![Page 151: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/151.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – construction of a graph
Goal
Reduce the bandwidth of a large sparse matrix by renumbering theunknowns.
Construction
The nodes of the graph are the unknowns ofthe system. They are labelled with a numberfrom 1 to n.
The edges are the relations between theunknowns. Two unknowns i and j are linked ifAij 6= 0.
The distance d(i, j) between two nodes is theminimal number of edges to follow to join bothnodes.
The excentricity E(i) = maxj d(i, j)
Far neighbors are λ(i) = {j/d(i, j) = E(i)}
Graph diameter D = maxi E(i)
Peripheral nodes P = {j/E(j) = D}.
![Page 152: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/152.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – construction of a graph
Goal
Reduce the bandwidth of a large sparse matrix by renumbering theunknowns.
Construction
The nodes of the graph are the unknowns ofthe system. They are labelled with a numberfrom 1 to n.
The edges are the relations between theunknowns. Two unknowns i and j are linked ifAij 6= 0.
The distance d(i, j) between two nodes is theminimal number of edges to follow to join bothnodes.
The excentricity E(i) = maxj d(i, j)
Far neighbors are λ(i) = {j/d(i, j) = E(i)}
Graph diameter D = maxi E(i)
Peripheral nodes P = {j/E(j) = D}.
![Page 153: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/153.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – construction of a graph
Goal
Reduce the bandwidth of a large sparse matrix by renumbering theunknowns.
Construction
The nodes of the graph are the unknowns ofthe system. They are labelled with a numberfrom 1 to n.
The edges are the relations between theunknowns. Two unknowns i and j are linked ifAij 6= 0.
The distance d(i, j) between two nodes is theminimal number of edges to follow to join bothnodes.
The excentricity E(i) = maxj d(i, j)
Far neighbors are λ(i) = {j/d(i, j) = E(i)}
Graph diameter D = maxi E(i)
Peripheral nodes P = {j/E(j) = D}.
![Page 154: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/154.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – construction of a graph
Goal
Reduce the bandwidth of a large sparse matrix by renumbering theunknowns.
Construction
The nodes of the graph are the unknowns ofthe system. They are labelled with a numberfrom 1 to n.
The edges are the relations between theunknowns. Two unknowns i and j are linked ifAij 6= 0.
The distance d(i, j) between two nodes is theminimal number of edges to follow to join bothnodes.
The excentricity E(i) = maxj d(i, j)
Far neighbors are λ(i) = {j/d(i, j) = E(i)}
Graph diameter D = maxi E(i)
Peripheral nodes P = {j/E(j) = D}.
![Page 155: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/155.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – construction of a graph
Goal
Reduce the bandwidth of a large sparse matrix by renumbering theunknowns.
Construction
The nodes of the graph are the unknowns ofthe system. They are labelled with a numberfrom 1 to n.
The edges are the relations between theunknowns. Two unknowns i and j are linked ifAij 6= 0.
The distance d(i, j) between two nodes is theminimal number of edges to follow to join bothnodes.
The excentricity E(i) = maxj d(i, j)
Far neighbors are λ(i) = {j/d(i, j) = E(i)}
Graph diameter D = maxi E(i)
Peripheral nodes P = {j/E(j) = D}.
![Page 156: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/156.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – construction of a graph
Goal
Reduce the bandwidth of a large sparse matrix by renumbering theunknowns.
Construction
The nodes of the graph are the unknowns ofthe system. They are labelled with a numberfrom 1 to n.
The edges are the relations between theunknowns. Two unknowns i and j are linked ifAij 6= 0.
The distance d(i, j) between two nodes is theminimal number of edges to follow to join bothnodes.
The excentricity E(i) = maxj d(i, j)
Far neighbors are λ(i) = {j/d(i, j) = E(i)}
Graph diameter D = maxi E(i)
Peripheral nodes P = {j/E(j) = D}.
![Page 157: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/157.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – construction of a graph
Goal
Reduce the bandwidth of a large sparse matrix by renumbering theunknowns.
Construction
The nodes of the graph are the unknowns ofthe system. They are labelled with a numberfrom 1 to n.
The edges are the relations between theunknowns. Two unknowns i and j are linked ifAij 6= 0.
The distance d(i, j) between two nodes is theminimal number of edges to follow to join bothnodes.
The excentricity E(i) = maxj d(i, j)
Far neighbors are λ(i) = {j/d(i, j) = E(i)}
Graph diameter D = maxi E(i)
Peripheral nodes P = {j/E(j) = D}.
![Page 158: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/158.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – bandwidth reduction
This graph is used to renumber the unknonws.
Choose a first node and label it with 1.
Attribute the new numbers (2,3,. . . ) to the neighbors of node 1with have the less non-labelled neighbors.
Label the neighbors of node 2
and so on. . .
until all nodes are labelled.
once this is done the numbering is reversed: the first become thelast.
![Page 159: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/159.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – bandwidth reduction
This graph is used to renumber the unknonws.
Choose a first node and label it with 1.
Attribute the new numbers (2,3,. . . ) to the neighbors of node 1with have the less non-labelled neighbors.
Label the neighbors of node 2
and so on. . .
until all nodes are labelled.
once this is done the numbering is reversed: the first become thelast.
![Page 160: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/160.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – bandwidth reduction
This graph is used to renumber the unknonws.
Choose a first node and label it with 1.
Attribute the new numbers (2,3,. . . ) to the neighbors of node 1with have the less non-labelled neighbors.
Label the neighbors of node 2
and so on. . .
until all nodes are labelled.
once this is done the numbering is reversed: the first become thelast.
![Page 161: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/161.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – bandwidth reduction
This graph is used to renumber the unknonws.
Choose a first node and label it with 1.
Attribute the new numbers (2,3,. . . ) to the neighbors of node 1with have the less non-labelled neighbors.
Label the neighbors of node 2
and so on. . .
until all nodes are labelled.
once this is done the numbering is reversed: the first become thelast.
![Page 162: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/162.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – bandwidth reduction
This graph is used to renumber the unknonws.
Choose a first node and label it with 1.
Attribute the new numbers (2,3,. . . ) to the neighbors of node 1with have the less non-labelled neighbors.
Label the neighbors of node 2
and so on. . .
until all nodes are labelled.
once this is done the numbering is reversed: the first become thelast.
![Page 163: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/163.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – bandwidth reduction
This graph is used to renumber the unknonws.
Choose a first node and label it with 1.
Attribute the new numbers (2,3,. . . ) to the neighbors of node 1with have the less non-labelled neighbors.
Label the neighbors of node 2
and so on. . .
until all nodes are labelled.
once this is done the numbering is reversed: the first become thelast.
![Page 164: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/164.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – example 1
![Page 165: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/165.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
Cuthill–McKeealgorithm
References
Cuthill–McKee algorithm – example 2
![Page 166: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/166.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Overview
1 Vectors and matricesElementary operationsGram–Schmidt orthonormalizationMatrix normConditioningSpecific matricesTridiagonalisationLU and QR factorizations
2 Eigenvalues and eigenvectorsPower iteration algorithmDeflationGalerkinJacobiQR
3 Numerical solution of linear systemsDirect methodsIterative methodsPreconditioning
4 StorageBand storageSparse storage
5 Bandwidth reductionCuthill–McKee algorithm
6 References
![Page 167: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/167.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Bibliography
P. Lascaux, R. Theodor, Analyse numerique matricielle appliquee al’art de l’ingenieur Volumes 1 and 2, 2eme edition, Masson (1997).
Gene H. Golub, Charles F. van Loan, Matrix Computations, 3rdedition, Johns Hopkins University Press (1996).
![Page 168: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/168.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References
Bibliography
P. Lascaux, R. Theodor, Analyse numerique matricielle appliquee al’art de l’ingenieur Volumes 1 and 2, 2eme edition, Masson (1997).
Gene H. Golub, Charles F. van Loan, Matrix Computations, 3rdedition, Johns Hopkins University Press (1996).
![Page 169: Linear Algebra - imag · 2015-09-25 · Linear Algebra Vectors and matrices Eigenvalues and eigenvectors Numerical solution of linear systems Storage Bandwidth reduction References](https://reader033.vdocument.in/reader033/viewer/2022043012/5faabf778c639b6558151d2d/html5/thumbnails/169.jpg)
Linear Algebra
Vectors and matrices
Eigenvalues andeigenvectors
Numerical solution oflinear systems
Storage
Bandwidth reduction
References TheEndTheEnd