tu berlin … · matrices, graphs, and pdes: a journey of unexpected relations, bms, berlin 2014...
TRANSCRIPT
![Page 1: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/1.jpg)
Matrices, Graphs, and PDEs:A journey of unexpected relations
Mario Arioli1
1BMS Berlin visiting [email protected]
http://www3.math.tu-berlin.de/Vorlesungen/SS14/MatricesGraphsPDEs/Thanks Oliver
![Page 2: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/2.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Programme
1. Linear Algebra from a variational point of view
2. Short introduction to the Finite Element method (FEM) andadaptive FEM
3. Complex networks
2 / 215
![Page 3: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/3.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Programme
Linear Algebra from a variational point of view:
I finite dimensional spaces on IRN with a norm based on apositive definite matrix A: a finite dimensional Hilbert spacestheory
I duality and convergence in dual norm
I relations between finite-element approximation matrices andmeasure of the error in energy
I Golub-Kahan bidiagonalisation method and elliptic singularvalues
2 / 215
![Page 4: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/4.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Programme
Short introduction to the Finite Element method (FEM) andadaptive FEM:
I How to use the properties of finite dimensional Hilbert spacesin order to detect where we need to improve the mesh
I Interplay between mesh graphs and matrices
I Fiedler vectors and partitioning of graphs
I Elements of Domain Decomposition techniques
I Adaptive methods and a posteriori measures of the algebraicerrors within the Krylov iterative methods
2 / 215
![Page 5: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/5.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Programme
Complex networks
I Elementary introductions to random graphs (Erdos-Renyi,Barabasi-Albert, and Watts- Strogatz random models) andcomplex graphs2: random models vs real life models
I Embedding of a graph in RN: quantum graphs and1D-simplex domains
I Solution of systems of parabolic equations on a quantumgraph3: Hamiltonians on graphs
I Applications in material science: Dirac’s model on graphene
2 / 215
![Page 6: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/6.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Prologue
When I’m feeling sadI simply remember my favorite things
And then I don’t feel so bad
3 / 215
![Page 7: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/7.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Prologue
When I’m feeling sadI simply remember my favorite things
And then I don’t feel so bad
3 / 215
![Page 8: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/8.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Les boules
0 20 40 60 80 10010−40
10−20
100
1020
N
V
V =π
N
2
Γ( N
2 + 1)
4 / 215
![Page 9: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/9.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Les boules
0 2 4 6 8 10
100.4
100.5
100.6
100.7
N
V
V =π
N
2
Γ( N
2 + 1)
4 / 215
![Page 10: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/10.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Les boules
V (N) =π
N2
Γ(N2 + 1)
N V2 π3 4
3π
4 π2
2
5 8π2
15
6 π3
6
10 π5
150
4 / 215
![Page 11: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/11.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Les boules
V (N) =π
N2
Γ(N2 + 1)
IS the Volume of the N-dimensional Sphere and
limN→∞
V (N)→ 0
4 / 215
![Page 12: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/12.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Les boules
v =1√N∗ eN =
1√N
1√N...1√N
N
||v||2 = 1
BUTlim
N→∞v = 0
4 / 215
![Page 13: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/13.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Les boules
FUNCTIONAL ANALYSIS IS LINEAR ALGEBRA COPING WITHSTRANGE BALLS
4 / 215
![Page 14: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/14.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some friends
G =
1 −11 −1
1 −11 −1
1 −11 −1
1 −11 −1
1 −11 −1
ddt on the interval [0, 1]
5 / 215
![Page 15: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/15.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some friends
GTG =
1 −1−1 2 −1
−1 2 −1−1 2 −1
−1 2 −1−1 2 −1
−1 2 −1−1 2 −1
−1 2 −1−1 2 −1
−1 1
d2
(dt)2Laplacian with Neumann conditions on the interval [0, 1]
5 / 215
![Page 16: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/16.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some friends
−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
5 / 215
![Page 17: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/17.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some friends
G =
1 −11 −11 −11 −1
G is the INCIDENCE MATRIX of the graph and the grad operator.GTG is the LAPLACIAN on the graph.I− GTG is the ADJACENCY matrix of the graph.
5 / 215
![Page 18: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/18.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some friends
We do not need a regular graph. On each edge, we have a mapfrom the IRN space where it lives to the segment [0, 1] and we cansolve on each edge the local operator!!
5 / 215
![Page 19: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/19.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some friends
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
5 / 215
![Page 20: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/20.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some friendsWe can insert points on each edge
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
5 / 215
![Page 21: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/21.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some friends
0 2 4 6 8 10 12 14 16 18 20 22
0
2
4
6
8
10
12
14
16
18
20
22
nz = 61
5 / 215
![Page 22: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/22.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some friends
0 1 2 3 4 5 6 7 8 90
1
2
3
4
5
6
7
8
5 / 215
![Page 23: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/23.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some friends
0 1 2 3 4 5 6 7 8 90
1
2
3
4
5
6
7
8
GRAPHENE5 / 215
![Page 24: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/24.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
TRUTH may be misleading
In a finite dimensional space all norms are equivalent i.e.
6 / 215
![Page 25: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/25.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
TRUTH may be misleading
In a finite dimensional space all norms are equivalent i.e.
c(N)||v ||1 ≤ ||v ||2 ≤ C (N)||v ||1
6 / 215
![Page 26: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/26.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
TRUTH may be misleading
In a finite dimensional space all norms are equivalent i.e.
c(N)||v ||1 ≤ ||v ||2 ≤ C (N)||v ||1
Identify the norms for which we have
c||v ||1 ≤ ||v ||2 ≤ C ||v ||1
6 / 215
![Page 27: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/27.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
TRUTH may be misleading
In a finite dimensional space all norms are equivalent i.e.
c(N)||v ||1 ≤ ||v ||2 ≤ C (N)||v ||1
Identify the norms for which we have
c||v ||1 ≤ ||v ||2 ≤ C ||v ||1 i.e. || · ||1 ∼ || · ||2
6 / 215
![Page 28: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/28.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Lecture 1
7 / 215
![Page 29: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/29.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Finite dimensional Hilbert spaces and IRN
I (·, ·) : H× H→ IR scalar product and‖u‖H =
√(u, u) ∀u ∈ H norm.
I ∃ψii=1,...,N a basis for H
∀u ∈ H u =∑N
i=1 uiψi ui ∈ IR i = 1, . . . ,N
I Representation of scalar product in IRN .Let u =
∑Ni=1 uiψi and v =
∑Ni=1 viψi .
Then
(u, v) =N∑i=1
N∑j=1
uivj(ψi , ψj) = vTHu
where Hij = Hji = (ψi , ψj) and u, v ∈ IRN .Moreover, uTHu > 0 iff u 6= 0 and, thus H SPD.
8 / 215
![Page 30: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/30.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Finite dimensional Hilbert spaces and IRN
I (·, ·) : H× H→ IR scalar product and‖u‖H =
√(u, u) ∀u ∈ H norm.
I ∃ψii=1,...,N a basis for H
∀u ∈ H u =∑N
i=1 uiψi ui ∈ IR i = 1, . . . ,N
I Representation of scalar product in IRN .Let u =
∑Ni=1 uiψi and v =
∑Ni=1 viψi .
Then
(u, v) =N∑i=1
N∑j=1
uivj(ψi , ψj) = vTHu
where Hij = Hji = (ψi , ψj) and u, v ∈ IRN .Moreover, uTHu > 0 iff u 6= 0 and, thus H SPD.
8 / 215
![Page 31: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/31.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Finite dimensional Hilbert spaces and IRN
I (·, ·) : H× H→ IR scalar product and‖u‖H =
√(u, u) ∀u ∈ H norm.
I ∃ψii=1,...,N a basis for H
∀u ∈ H u =∑N
i=1 uiψi ui ∈ IR i = 1, . . . ,N
I Representation of scalar product in IRN .Let u =
∑Ni=1 uiψi and v =
∑Ni=1 viψi .
Then
(u, v) =N∑i=1
N∑j=1
uivj(ψi , ψj) = vTHu
where Hij = Hji = (ψi , ψj) and u, v ∈ IRN .Moreover, uTHu > 0 iff u 6= 0 and, thus H SPD.
8 / 215
![Page 32: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/32.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Dual space H?
I f ∈ H? : H→ IR (functional);
I f (αu + βv) = αf (u) + βf (v) ∀u, v ∈ H
I H? is the space of the linear functionals on H
‖f ‖?H = supu 6=0
f (u)
‖u‖H
I If H finite dimensional and u =∑N
i=1 uiψi , then
f (u) =∑N
i=1 ui f (ψi ) = fTu
I Dual vectorLet u ∈ H, u 6= 0, then ∃fu ∈ H? such that
fu(u) = ‖u‖H
(Hahn-Banach).
9 / 215
![Page 33: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/33.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Dual space H?
I f ∈ H? : H→ IR (functional);
I f (αu + βv) = αf (u) + βf (v) ∀u, v ∈ H
I H? is the space of the linear functionals on H
‖f ‖?H = supu 6=0
f (u)
‖u‖H
I If H finite dimensional and u =∑N
i=1 uiψi , then
f (u) =∑N
i=1 ui f (ψi ) = fTu
I Dual vectorLet u ∈ H, u 6= 0, then ∃fu ∈ H? such that
fu(u) = ‖u‖H
(Hahn-Banach).
9 / 215
![Page 34: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/34.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Dual space H?
I f ∈ H? : H→ IR (functional);
I f (αu + βv) = αf (u) + βf (v) ∀u, v ∈ H
I H? is the space of the linear functionals on H
‖f ‖?H = supu 6=0
f (u)
‖u‖H
I If H finite dimensional and u =∑N
i=1 uiψi , then
f (u) =∑N
i=1 ui f (ψi ) = fTu
I Dual vectorLet u ∈ H, u 6= 0, then ∃fu ∈ H? such that
fu(u) = ‖u‖H
(Hahn-Banach).
9 / 215
![Page 35: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/35.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Dual space H?
I f ∈ H? : H→ IR (functional);
I f (αu + βv) = αf (u) + βf (v) ∀u, v ∈ H
I H? is the space of the linear functionals on H
‖f ‖?H = supu 6=0
f (u)
‖u‖H
I If H finite dimensional and u =∑N
i=1 uiψi , then
f (u) =∑N
i=1 ui f (ψi ) = fTu
I Dual vectorLet u ∈ H, u 6= 0, then ∃fu ∈ H? such that
fu(u) = ‖u‖H
(Hahn-Banach).
9 / 215
![Page 36: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/36.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Dual space H?
I f ∈ H? : H→ IR (functional);
I f (αu + βv) = αf (u) + βf (v) ∀u, v ∈ H
I H? is the space of the linear functionals on H
‖f ‖?H = supu 6=0
f (u)
‖u‖H
I If H finite dimensional and u =∑N
i=1 uiψi , then
f (u) =∑N
i=1 ui f (ψi ) = fTu
I Dual vectorLet u ∈ H, u 6= 0, then ∃fu ∈ H? such that
fu(u) = ‖u‖H
(Hahn-Banach).
9 / 215
![Page 37: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/37.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Dual space H?
I Let H be a Hilbert finite dimensional space and H the realN × N matrix identifying the scalar product.
I
fu(u) = fTu = (uTHu)1/2
The dual vector of u has the following representation:
10 / 215
![Page 38: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/38.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Dual space H?
I Let H be a Hilbert finite dimensional space and H the realN × N matrix identifying the scalar product.
I
fu(u) = fTu = (uTHu)1/2
The dual vector of u has the following representation:
10 / 215
![Page 39: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/39.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Dual space H?
I Let H be a Hilbert finite dimensional space and H the realN × N matrix identifying the scalar product.
I
fu(u) = fTu = (uTHu)1/2
The dual vector of u has the following representation:
f =Hu
‖u‖H
and‖fu‖2
H? = uTHu = fTH−1f
10 / 215
![Page 40: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/40.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Dual space basis
I The general definitions of a dual basis for H is
φj(ψi ) =
1 i = j0 i 6= j
I The φi are linearly independent:
N∑i=1
βiφi (u) = 0 ∀u ∈ H=⇒N∑i=1
βiφi (ψi ) = 0=⇒βi = 0.
I f (ψi ) = γi and f (u) = f (∑N
i=1 uiψi ) =∑N
i=1 γiui
φi (u) = φ(N∑i=1
uiψi ) = ui=⇒f =N∑i=1
αiφi
11 / 215
![Page 41: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/41.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Dual space basis
I The general definitions of a dual basis for H is
φj(ψi ) =
1 i = j0 i 6= j
I The φi are linearly independent:
N∑i=1
βiφi (u) = 0 ∀u ∈ H=⇒N∑i=1
βiφi (ψi ) = 0=⇒βi = 0.
I f (ψi ) = γi and f (u) = f (∑N
i=1 uiψi ) =∑N
i=1 γiui
φi (u) = φ(N∑i=1
uiψi ) = ui=⇒f =N∑i=1
αiφi
11 / 215
![Page 42: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/42.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Dual space basis
I The general definitions of a dual basis for H is
φj(ψi ) =
1 i = j0 i 6= j
I The φi are linearly independent:
N∑i=1
βiφi (u) = 0 ∀u ∈ H=⇒N∑i=1
βiφi (ψi ) = 0=⇒βi = 0.
I f (ψi ) = γi and f (u) = f (∑N
i=1 uiψi ) =∑N
i=1 γiui
φi (u) = φ(N∑i=1
uiψi ) = ui=⇒f =N∑i=1
αiφi
11 / 215
![Page 43: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/43.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operator
I A : H −→ V where H and V finite dimensional Hilbertspaces. H and V are the SPD matrices of the scalar products
I
‖A ‖H,V = maxu 6=0
‖A u‖V‖u‖H
= ‖V1/2AH−1/2‖2
I The result follows from the generalized eigenvalue problem inIRN
ATVAu = λHu
I
κH(M) = ‖M‖H,H−1‖M−1‖H−1,H.
12 / 215
![Page 44: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/44.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operator
I A : H −→ V where H and V finite dimensional Hilbertspaces. H and V are the SPD matrices of the scalar products
I
‖A ‖H,V = maxu 6=0
‖A u‖V‖u‖H
= ‖V1/2AH−1/2‖2
I The result follows from the generalized eigenvalue problem inIRN
ATVAu = λHu
I
κH(M) = ‖M‖H,H−1‖M−1‖H−1,H.
12 / 215
![Page 45: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/45.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operator
I A : H −→ V where H and V finite dimensional Hilbertspaces. H and V are the SPD matrices of the scalar products
I
‖A ‖H,V = maxu 6=0
‖A u‖V‖u‖H
= ‖V1/2AH−1/2‖2
I The result follows from the generalized eigenvalue problem inIRN
ATVAu = λHu
I
κH(M) = ‖M‖H,H−1‖M−1‖H−1,H.
12 / 215
![Page 46: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/46.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operator
I A : H −→ V where H and V finite dimensional Hilbertspaces. H and V are the SPD matrices of the scalar products
I
‖A ‖H,V = maxu 6=0
‖A u‖V‖u‖H
= ‖V1/2AH−1/2‖2
I The result follows from the generalized eigenvalue problem inIRN
ATVAu = λHu
I
κH(M) = ‖M‖H,H−1‖M−1‖H−1,H.
12 / 215
![Page 47: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/47.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operator
I A : H −→ V where H and V finite dimensional Hilbertspaces. H and V are the SPD matrices of the scalar products
I
‖A ‖H,V = maxu 6=0
‖A u‖V‖u‖H
= ‖V1/2AH−1/2‖2
I The result follows from the generalized eigenvalue problem inIRN
ATVAu = λHu
I
κH(M) = ‖M‖H,H−1‖M−1‖H−1,H.
The interesting case is κH(M) independent of N
12 / 215
![Page 48: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/48.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Hilbert Space Setting: duality and adjoint
Given z ∈ H?, we have
〈z , u〉H?,H = zTu = zTH−1Hu = (u,H−1z)H,
w = H−1z Riesz vector corresponding to w =∑
j wjφj ∈ H.
13 / 215
![Page 49: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/49.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Hilbert Space Setting: duality and adjoint
Given z ∈ H?, we have
〈z , u〉H?,H = zTu = zTH−1Hu = (u,H−1z)H,
w = H−1z Riesz vector corresponding to w =∑
j wjφj ∈ H.
Let C : H 7→ FC ? : F? 7→ H? (adjoint operator)
〈C ?v , u〉H?,H , 〈v ,C u〉F?,F ∀v ∈ F?, u ∈ H.
13 / 215
![Page 50: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/50.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Hilbert Space Setting: duality and adjoint
Given z ∈ H?, we have
〈z , u〉H?,H = zTu = zTH−1Hu = (u,H−1z)H,
w = H−1z Riesz vector corresponding to w =∑
j wjφj ∈ H.
Let C : H 7→ FC ? : F? 7→ H? (adjoint operator)
〈C ?v , u〉H?,H , 〈v ,C u〉F?,F ∀v ∈ F?, u ∈ H.
Therefore, we have
〈C ?v , u〉H?,H = (Cu,F−1v)F = uTCTv.
13 / 215
![Page 51: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/51.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Hilbert Space Setting: normal equations
If we assume that F = H? then we have that the “normalequations operator” in the Hilbert space is an operator such that
C ? H −1 C : H 7→ H?,
and it is represented by the matrix
CTH−1C.
14 / 215
![Page 52: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/52.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Hilbert Space Setting: normal equations
If we assume that F = H? then we have that the “normalequations operator” in the Hilbert space is an operator such that
C ? H −1 C : H 7→ H?,
and it is represented by the matrix
CTH−1C.
If CT = C then the corresponding operator C is self-adjoint.Moreover, we have that the operator
H −1 C : H 7→ H
maps H into itself. (H −1 C
)i, (H−1C)i .
14 / 215
![Page 53: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/53.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operators
Let us consider now the Hilbert spaces
M := (IRn, ‖ · ‖M), N := (IRm, ‖ · ‖N),
and their dual spaces
M? := (IRn, ‖ · ‖M−1), N? := (IRm, ‖ · ‖N−1),
15 / 215
![Page 54: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/54.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operators
Let us consider now the Hilbert spaces
M := (IRn, ‖ · ‖M), N := (IRm, ‖ · ‖N),
and their dual spaces
M? := (IRn, ‖ · ‖M−1), N? := (IRm, ‖ · ‖N−1),
A : N→M?
〈A y , u〉M?,M , (u,M−1Ay)M = uTAy, y ∈ N,∀u ∈M,
15 / 215
![Page 55: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/55.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operators
Let us consider now the Hilbert spaces
M := (IRn, ‖ · ‖M), N := (IRm, ‖ · ‖N),
and their dual spaces
M? := (IRn, ‖ · ‖M−1), N? := (IRm, ‖ · ‖N−1),
A : N→M?
〈A y , u〉M?,M , (u,M−1Ay)M = uTAy, y ∈ N,∀u ∈M,
〈A ?u, y〉N?,N := (y,N−1ATu)N = yTATu, u ∈M,∀y ∈ N,
15 / 215
![Page 56: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/56.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operators
16 / 215
![Page 57: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/57.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operators
C =
[M AAT −N
]C : M×N 7→M? ×N?.
The scalar product in M×N is represented by the matrix
H =
[M
N
].
17 / 215
![Page 58: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/58.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Lecture 2
18 / 215
![Page 59: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/59.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear systems: variational framework
I Find u ∈ H such that for all v ∈ H
a(u, v) = L(v) (L(·) ∈ H? dual space of H)
I Existence and uniqueness: ∀v,w ∈ H
a(w, v) ≤ C1‖w‖H‖v‖Hsup
w∈H\0
a(w, v)
‖w‖H≥ C2‖v‖H
I H = (IRN , || · ||H) and H? = (IRN , || · ||H−1)H SPD
19 / 215
![Page 60: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/60.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear systems: variational framework
I Find u ∈ H such that for all v ∈ H
a(u, v) = L(v) (L(·) ∈ H? dual space of H)
I Existence and uniqueness: ∀v,w ∈ H
a(w, v) ≤ C1‖w‖H‖v‖Hsup
w∈H\0
a(w, v)
‖w‖H≥ C2‖v‖H
I H = (IRN , || · ||H) and H? = (IRN , || · ||H−1)H SPD
19 / 215
![Page 61: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/61.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear systems: variational framework
I Find u ∈ H such that for all v ∈ H
a(u, v) = L(v) (L(·) ∈ H? dual space of H)
I Existence and uniqueness: ∀v,w ∈ H
a(w, v) ≤ C1‖w‖H‖v‖Hsup
w∈H\0
a(w, v)
‖w‖H≥ C2‖v‖H
I H = (IRN , || · ||H) and H? = (IRN , || · ||H−1)H SPD
19 / 215
![Page 62: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/62.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Finite dimensional Banach spaces and IRN
I Other norms are possible on IRN :
I ‖u‖p = (∑N
i=1(ui )p)1/2 with 1 < p <∞
I ‖u‖1 = (∑N
i=1 |ui |)I ‖u‖∞ = maxi |ui |
I Hyper-norms on IRN of order k .
‖ · ‖~k : IRN → IRk
I ∀λ ∈ IR ‖λu‖~k = |λ|‖u‖~kII ∀u, v ∈ IRN ‖u + v‖~k ≤ ‖u‖~k + ‖v‖~k component-wise
III ‖u‖~k = 0k ⇒ u = 0N
20 / 215
![Page 63: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/63.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Finite dimensional Banach spaces and IRN
I Other norms are possible on IRN :I ‖u‖p = (
∑Ni=1(ui )
p)1/2 with 1 < p <∞I ‖u‖1 = (
∑Ni=1 |ui |)
I ‖u‖∞ = maxi |ui |
I Hyper-norms on IRN of order k .
‖ · ‖~k : IRN → IRk
I ∀λ ∈ IR ‖λu‖~k = |λ|‖u‖~kII ∀u, v ∈ IRN ‖u + v‖~k ≤ ‖u‖~k + ‖v‖~k component-wise
III ‖u‖~k = 0k ⇒ u = 0N
20 / 215
![Page 64: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/64.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Finite dimensional Banach spaces and IRN
I Other norms are possible on IRN :I ‖u‖p = (
∑Ni=1(ui )
p)1/2 with 1 < p <∞I ‖u‖1 = (
∑Ni=1 |ui |)
I ‖u‖∞ = maxi |ui |I Hyper-norms on IRN of order k .
‖ · ‖~k : IRN → IRk
I ∀λ ∈ IR ‖λu‖~k = |λ|‖u‖~kII ∀u, v ∈ IRN ‖u + v‖~k ≤ ‖u‖~k + ‖v‖~k component-wise
III ‖u‖~k = 0k ⇒ u = 0N
20 / 215
![Page 65: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/65.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operator hyper-norm
I Let ‖u‖~k and ‖v‖~p two hyper-norms on IRn and A a linearoperator between (IRn, ‖ · ‖~k) and (IRn, ‖ · ‖~p)
I The norm is defined as
‖A ‖~k,~p = M ∈ IRk×p
M =
‖A11‖ . . . ‖A1k‖... . . .
...‖Ap1‖ . . . ‖Apk‖
I
IRn =
p⊕j=1
Wj =k⊕
i=1
Vi Wi ∩Wj = 0 Vi ∩Vj = 0
21 / 215
![Page 66: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/66.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operator hyper-norm
I Let ‖u‖~k and ‖v‖~p two hyper-norms on IRn and A a linearoperator between (IRn, ‖ · ‖~k) and (IRn, ‖ · ‖~p)
I The norm is defined as
‖A ‖~k,~p = M ∈ IRk×p
M =
‖A11‖ . . . ‖A1k‖... . . .
...‖Ap1‖ . . . ‖Apk‖
I
IRn =
p⊕j=1
Wj =k⊕
i=1
Vi Wi ∩Wj = 0 Vi ∩Vj = 0
21 / 215
![Page 67: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/67.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operator hyper-norm
I Let ‖u‖~k and ‖v‖~p two hyper-norms on IRn and A a linearoperator between (IRn, ‖ · ‖~k) and (IRn, ‖ · ‖~p)
I The norm is defined as
‖A ‖~k,~p = M ∈ IRk×p
M =
‖A11‖ . . . ‖A1k‖... . . .
...‖Ap1‖ . . . ‖Apk‖
I
IRn =
p⊕j=1
Wj =k⊕
i=1
Vi Wi ∩Wj = 0 Vi ∩Vj = 0
21 / 215
![Page 68: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/68.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Rigal-Gaches (1967) theorem
∃∆A,∃δb such that:(A + ∆A)u = (b + δb) and
‖∆A‖~k,~p ≤ S ∈ IRk×p, ‖δb‖~k ≤ t ∈ IRk
⇔‖r‖~k ≤ S‖u‖~p + twhere r is defined byr = Au− b
22 / 215
![Page 69: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/69.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Rigal-Gaches (1967) proof
A =
A11 . . . A1k... . . .
...Ap1 . . . Apk
r =
r1...
rk
u =
u1...
up
∆Aij = − (S‖u‖~p)i
(S‖u‖~p + ‖t‖~k)iri (Zi
j)T
where(Zi
j) = (S‖u‖~p)izk
and zk is the dual vector of uk ( zTk uk = (‖u‖~p)k ;
∆bi =(‖t‖~k)i
(S‖u‖~p + ‖t‖~k)iri
23 / 215
![Page 70: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/70.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Backward error
We have the following equivalence in a general Hilbert (true alsofor a Banach):
∃b ∈ BL(H),∃δL ∈ H? such that:a(u, v) + b(u, v) = (L + δL)(v),∀v ∈ H, and‖b(·, ·)‖BL(H) ≤ α, ‖δL‖H? ≤ β
⇔‖ρu‖H? ≤ α‖u‖H + βwhere ρu ∈ H? is defined by〈ρu, v〉H?,H = a(u, v)− L(v),
∀v ∈ H
A., Noulard, and Russo (2001)
24 / 215
![Page 71: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/71.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Backward error (proof)
The proof will be given assuming that H is only a Banach space, therebyshowing that the theorem holds, even in a more general situation. Forthis reason, in this proof, (and only here), we will use the notation ofduality pairs.⇒: This is obvious.⇐: We will build two perturbations of a and L, respectively b and δL,such that :
a(u, v) + b(u, v) = L(v) + δL(v),∀v ∈ H.
We set:∀u ∈ H, 〈ρu, v〉H?,H = b(u, v)− L(v),∀v ∈ H;
we have ρu ∈ H?.
25 / 215
![Page 72: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/72.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Backward error (proof)
We will denote by Ju ∈ (H?)? = H?? the element of the bi-dual of H,which is associated to u in the canonic injection
J : H −→ V ??I ⊂ H??
u 7−→ Ju
defined by 〈Ju, f 〉H??,H? = 〈f , u〉H?,H, ∀f ∈ H?. It is well-known that J isa linear isometry (see e.g. H. BREZIS, Analyse Fonctionnelle, Theorieet Applications, Masson, Paris, 1983.[III.4 p. 39]). We then have
‖Ju‖H?? = ‖u‖H = sup‖f ‖H?≤1 〈Ju, f 〉H??,H? =
sup‖f ‖H?≤1 〈f , u〉H?,H = 〈fu, u〉H?,H ,
for a certain fu ∈ H?. One must be aware of the fact that, here, we
cannot associate a vector v ∈ H to fu, unless H is reflexive. In other
words we cannot find a v ∈ H such that ‖fu‖H? = 〈fu, v〉H?,H, because
‖fu‖H? is a sup and not a max. It is a max if (and only if) H is reflexive.
26 / 215
![Page 73: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/73.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Backward error (proof)Now, as has been done for the perturbation of a system of linearequations, we define:
b(u, v) = − α
α‖u‖H + β〈Ju, fu〉H??,H 〈ρu, v〉H?,H
and
δL(v) =β
α‖u‖H + β〈ρu, v〉H?,H .
It is obvious that b is continuous and bilinear from H× H to IR, andδL ∈ H?; an easy computation shows that
δL(v)− b(u, v) =(β
α‖u‖H + β+
α
α‖u‖H + β〈Ju, fu〉H??,H?
)〈ρu, v〉H?,H = 〈ρu, v〉H?,H
as required. Moreover, if we suppose that ‖ρu‖H? ≤ α‖u‖H + β, thenwe have:
‖b‖BL(H) ≤ α, ‖δL‖H? ≤ β.
27 / 215
![Page 74: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/74.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Backward error (Remark)
If H is a reflexive Banach space, we can give a more expressive form tothe perturbation term b. In fact, in this case, we can identify Ju and uand obtain that
b(u, v) = − αα‖u‖+β 〈Ju, fu〉H??,H? 〈ρu, v〉H?,H
= − αα‖u‖+β 〈fu, u〉H?,H 〈ρu, v〉H?,H
= − αα‖u‖+β 〈fu ⊗ ρu, (u, v)〉 ,
in analogy with the finite dimensional case.
28 / 215
![Page 75: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/75.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: conjugate gradient method
A symmetric positive definite
H = (IRN , || · ||A) and H? = (IRN , || · ||A−1)
At each step k the conjugate gradient method minimizes the energynorm of the error δu(k) = u− u(k) on a Krylov space u(0) +Kk :
minu(k)∈ u(0)+Kk
‖δu(k)‖2A
‖δu(k)‖A = ‖ρu(k)‖H? = ‖r(k)‖A−1
r(k) = b− Au(k)
Arioli Numer. Math. (2003)
29 / 215
![Page 76: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/76.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: conjugate gradient method
u(k) = u(k−1) + αk−1p(k−1) αk−1 =r(k−1)T r(k−1)
p(k−1)TAp(k−1),
r(k) = r(k−1) − αk−1Ap(k−1)
p(k) = r(k) + βk−1p(k−1), βk−1 =r(k)T r(k)
r(k−1)T r(k−1),
where u(0) = 0 and r(0) = p(0) = b.
30 / 215
![Page 77: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/77.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: conjugate gradient method
Taking into account that p(i)TAp(j) = 0, i 6= j we have
u =N∑
j=1
αjp(j) ‖u‖2
A =N∑
j=1
αjr(j)T r(j)
31 / 215
![Page 78: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/78.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: conjugate gradient method
Taking into account that p(i)TAp(j) = 0, i 6= j we have
u =N∑
j=1
αjp(j) ‖u‖2
A =N∑
j=1
αjr(j)T r(j)
uTAu =∑N
j=1
∑Ni=1 αjαip
(j)TAp(i)
=∑N
j=1 α2j p(j)TAp(j)
but αjp(j)TAp(j) = r(j)T r(j).
31 / 215
![Page 79: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/79.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: stopping criteria
I Classic Criterion:
IF ‖Au(k) − b‖2 ≤√ε‖b‖2 THEN STOP ,
I New Criterion:
IF ‖Au(k) − b‖A−1 ≤ η‖b‖A−1 THEN STOP ,
with η < 1 an a-priori threshold fixed by the user. The choiceof η will depend on the properties of the problem that wewant to solve, and, in the practical cases, η can be frequentlymuch larger than ε , the roundoff unit of the computer finiteprecision arithmetic.
32 / 215
![Page 80: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/80.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: stopping criteria
I Classic Criterion:
IF ‖Au(k) − b‖2 ≤√ε‖b‖2 THEN STOP ,
I New Criterion:
IF ‖Au(k) − b‖A−1 ≤ η‖b‖A−1 THEN STOP ,
with η < 1 an a-priori threshold fixed by the user.
The choiceof η will depend on the properties of the problem that wewant to solve, and, in the practical cases, η can be frequentlymuch larger than ε , the roundoff unit of the computer finiteprecision arithmetic.
32 / 215
![Page 81: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/81.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: stopping criteria
I Classic Criterion:
IF ‖Au(k) − b‖2 ≤√ε‖b‖2 THEN STOP ,
I New Criterion:
IF ‖Au(k) − b‖A−1 ≤ η‖b‖A−1 THEN STOP ,
with η < 1 an a-priori threshold fixed by the user. The choiceof η will depend on the properties of the problem that wewant to solve, and, in the practical cases, η can be frequentlymuch larger than ε , the roundoff unit of the computer finiteprecision arithmetic.
32 / 215
![Page 82: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/82.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: stopping criteria cont.
‖Au(k) − b‖A−1 ?
‖b‖A−1 ?
33 / 215
![Page 83: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/83.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: stopping criteria cont.
‖Au(k) − b‖A−1 ?
‖b‖A−1 ?
33 / 215
![Page 84: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/84.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: stopping criteria cont.
‖Au(k) − b‖A−1
I Hestenes-Stiefel rule (1952) (see Strakos and Tichy, 2002)numerically stable
I Gauss quadrature rules (Golub and Meurant, 1997)I Gauss equivalent to Hestenes-Stiefel rule (Strakos and Tichy).
The Gauss quadrature does not require any a-priori knowledgeof the smallest and the biggest eigenvalues and computes alower bound of‖Au(k) − b‖A−1 .
I Gauss-Lobatto and Gauss-Radau. They compute lower andupper bounds using the extremes eigenvalues of A.
34 / 215
![Page 85: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/85.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: stopping criteria cont.
‖Au(k) − b‖A−1
I Hestenes-Stiefel rule (1952) (see Strakos and Tichy, 2002)numerically stable
I Gauss quadrature rules (Golub and Meurant, 1997)I Gauss equivalent to Hestenes-Stiefel rule (Strakos and Tichy).
The Gauss quadrature does not require any a-priori knowledgeof the smallest and the biggest eigenvalues and computes alower bound of‖Au(k) − b‖A−1 .
I Gauss-Lobatto and Gauss-Radau. They compute lower andupper bounds using the extremes eigenvalues of A.
34 / 215
![Page 86: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/86.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: stopping criteria cont.
‖Au(k) − b‖A−1
I Hestenes-Stiefel rule (1952) (see Strakos and Tichy, 2002)numerically stable
I Gauss quadrature rules (Golub and Meurant, 1997)
I Gauss equivalent to Hestenes-Stiefel rule (Strakos and Tichy).The Gauss quadrature does not require any a-priori knowledgeof the smallest and the biggest eigenvalues and computes alower bound of‖Au(k) − b‖A−1 .
I Gauss-Lobatto and Gauss-Radau. They compute lower andupper bounds using the extremes eigenvalues of A.
34 / 215
![Page 87: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/87.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: stopping criteria cont.
‖Au(k) − b‖A−1
I Hestenes-Stiefel rule (1952) (see Strakos and Tichy, 2002)numerically stable
I Gauss quadrature rules (Golub and Meurant, 1997)I Gauss equivalent to Hestenes-Stiefel rule (Strakos and Tichy).
The Gauss quadrature does not require any a-priori knowledgeof the smallest and the biggest eigenvalues and computes alower bound of‖Au(k) − b‖A−1 .
I Gauss-Lobatto and Gauss-Radau. They compute lower andupper bounds using the extremes eigenvalues of A.
34 / 215
![Page 88: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/88.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: stopping criteria cont.
‖Au(k) − b‖A−1
I Hestenes-Stiefel rule (1952) (see Strakos and Tichy, 2002)numerically stable
I Gauss quadrature rules (Golub and Meurant, 1997)I Gauss equivalent to Hestenes-Stiefel rule (Strakos and Tichy).
The Gauss quadrature does not require any a-priori knowledgeof the smallest and the biggest eigenvalues and computes alower bound of‖Au(k) − b‖A−1 .
I Gauss-Lobatto and Gauss-Radau. They compute lower andupper bounds using the extremes eigenvalues of A.
34 / 215
![Page 89: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/89.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: Hestenes-Stiefel rule
During the conjugate gradient iterates, we compute the scalar αk
and the conjugate vectors p(k) (p(j)TAp(i) = 0, j 6= i) and theresiduals r(k). Thus,
u =N∑j=1
αjp(j)
and
‖δu(k)‖2A = ‖Au(k) − b‖2
A−1 = e2A =
N∑j=k+1
αj r(j)T r(j)
35 / 215
![Page 90: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/90.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: Hestenes-Stiefel rule
Under the assumption that e(k+d)A << e
(k)A , where the integer d
denotes a suitable delay, the Hestenes and Stiefel estimate ξk willbe
ξk =k+d∑
j=k+1
αj r(j)T r(j).
The choice of a value for d depends on preconditioner andill-conditioning.
36 / 215
![Page 91: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/91.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The symmetric case: Hestenes-Stiefel rule
Under the assumption that e(k+d)A << e
(k)A , where the integer d
denotes a suitable delay, the Hestenes and Stiefel estimate ξk willbe
ξk =k+d∑
j=k+1
αj r(j)T r(j).
The choice of a value for d depends on preconditioner andill-conditioning.
36 / 215
![Page 92: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/92.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
bT A−1b
Fromr(k)Tv = 0, ∀v ∈ Kk ,
we prove
bTA−1b = uTAu ≥k∑
j=1
αj r(j)T r(j),
(the right-hand side will converge monotonically to ‖u‖2A).
Therefore, we use the following stopping criterion
IF ξk ≤ η2k∑
j=1
αj r(j)T r(j) THEN STOP .
37 / 215
![Page 93: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/93.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Preconditioning
Let U a non singular matrix: the symmetric preconditioned systemis
U−TAU−1y = U−Tb(y = Uu
)y(k) = y(k−1) + αk−1p(k−1) αk−1 =
r(k−1)T r(k−1)
p(k−1)TU−TAU−1p(k−1),
r(k) = r(k−1) − αk−1U−TAU−1p(k−1)
p(k) = r(k) + βk−1p(k−1), βk−1 =r(k)T r(k)
r(k−1)T r(k−1),
where y(0) = 0 and r(0) = p(0) = b. In exact arithmetic we have
r(k) = U−Tb−U−TAU−1y(k).
38 / 215
![Page 94: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/94.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Preconditioning
Let U a non singular matrix: the symmetric preconditioned systemis
U−TAU−1y = U−Tb(y = Uu
)y(k) = y(k−1) + αk−1p(k−1) αk−1 =
r(k−1)T r(k−1)
p(k−1)TU−TAU−1p(k−1),
r(k) = r(k−1) − αk−1U−TAU−1p(k−1)
p(k) = r(k) + βk−1p(k−1), βk−1 =r(k)T r(k)
r(k−1)T r(k−1),
where y(0) = 0 and r(0) = p(0) = b. In exact arithmetic we have
r(k) = U−Tb−U−TAU−1y(k).
Defining u(k) = U−1y(k) we have r(k) = U−T r(k). Then
‖r(k)‖2(U−T AU−1)−1 = r(k)TUA−1UT r(k) = ‖r(k)‖2
A−1
38 / 215
![Page 95: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/95.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Preconditioning
The dual norm of the preconditioned residual is equal to the dualnorm of the original residual.
39 / 215
![Page 96: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/96.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
PCG algorithm
M. Arioli
where y(0) = 0 and r (0) = p(0) = U−T b. Moreover, in exact arithmetic,we have that r (k) = U−T b − U−T AU−1y(k), and, therefore, defining u(k) =U−1y(k), we have that
r (k) = U−T (b − Au(k)) = U−T r(k).
Then, we have that
∥r (k)∥2(U−T AU−1)−1 = r (k)T UA−1UT r(k) = ∥r∥2
A−1 .
Finally, if we define p(k) = U−1p(k), and M = UT U , we obtain the variantof the preconditioned conjugate gradient algorithm, which incorporates theproposed stopping criterion with a suitable choice of d described in Figure 1.
Remark 2 The value of the exponent t in formula (10) can be higher than theone suggested by (12) which depends on the regularity of the solution. Thisis the case when super-convergence in the nodes of the mesh occurs and weknow that the values in the mesh nodes are very accurate [23,8].
Preconditioned Conjugate Gradient Algorithm (PCG)Given an initial guess u(0), compute r(0) = b − Au(0), and solve Mz(0) = r(0). Setp(0) = z(0), β0 = 0, α−1 = 1, ρ0 = bT u(0), and ξ0 = ∞.
k = 0while = ξk > η2(ρ0 + r(0)T u(k)) do
k = k + 1;χk = r(k−1)T z(k−1) ;
αk−1 = r(k−1)T z(k−1)
p(k−1)T Ap(k−1);
ψk = αk−1χk ;u(k) = u(k−1) + αk−1p
(k−1);r(k) = r(k−1) − αk−1Ap(k−1);Solve Mz(k) = r(k);
βk = r(k)T z(k)
r(k−1)T z(k−1);
pk = zk + βkp(k−1);
if = k > d then
ξk =k!
j=k−d+1
ψj ;
elseξk = ξk−1;
endifend while.
Fig. 1. Preconditioned Conjugate Gradient Algorithm (PCG)
40 / 215
![Page 97: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/97.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Continuous problem
a(u, v) =
∫Ωk(x)∇u · ∇vdx, ∀u, v ∈ H1
0 (Ω)
∀u, v ∈ H10 (Ω), ∃γ ∈ IR+ and ∃M ∈ IR+ such that
γ||u||21,Ω ≤ a(u, u)
a(u, v) ≤ M||u||1,Ω||v ||1,Ω ,
L(v) =∫
Ω fvdx, L(v) ∈ H−1(Ω).
(P)
Find u ∈ H1
0 (Ω) such thata(u, v) = L(v), ∀v ∈ H1
0 (Ω),has a unique solution.
41 / 215
![Page 98: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/98.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Finite-element approximation
I Weak formulation
Find uh ∈ Hh such thatah(uh, vh) = Lh(vh),∀vh ∈ Hh,
Finite element methods choose Hh to be a space of functions vhdefined on a subdivision Ωh of Ω into simplices T of diameter hT ;
h denotes a piecewise constant function defined on Ωh via
h|T = hT .
I Existence and uniqueness: Hh ⊂ H = H10 (Ω).
I Error Estimate: ‖u − uh‖H ≤ C (h)See Claes Johnson Numerical Solutions Of Partial Differential Equations By The Finite Element Method
2009
42 / 215
![Page 99: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/99.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Finite-element approximation
I Weak formulationFind uh ∈ Hh such thatah(uh, vh) = Lh(vh),∀vh ∈ Hh,
Finite element methods choose Hh to be a space of functions vhdefined on a subdivision Ωh of Ω into simplices T of diameter hT ;
h denotes a piecewise constant function defined on Ωh via
h|T = hT .
I Existence and uniqueness: Hh ⊂ H = H10 (Ω).
I Error Estimate: ‖u − uh‖H ≤ C (h)See Claes Johnson Numerical Solutions Of Partial Differential Equations By The Finite Element Method
2009
42 / 215
![Page 100: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/100.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Finite-element approximation
I Weak formulationFind uh ∈ Hh such thatah(uh, vh) = Lh(vh),∀vh ∈ Hh,
Finite element methods choose Hh to be a space of functions vhdefined on a subdivision Ωh of Ω into simplices T of diameter hT ;
h denotes a piecewise constant function defined on Ωh via
h|T = hT .
I Existence and uniqueness: Hh ⊂ H = H10 (Ω).
I Error Estimate: ‖u − uh‖H ≤ C (h)See Claes Johnson Numerical Solutions Of Partial Differential Equations By The Finite Element Method
2009
42 / 215
![Page 101: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/101.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Finite-element framework
SolveAuh = b
given
supw∈IRN\0
supv∈IRN\0
wTAv
‖v‖H‖w‖H≤ C1 (sup-sup)
infw∈IRN\0
supv∈IRN\0
wTAv
‖v‖H‖w‖H≥ C2 (inf-sup)
Note: ‖vh‖Hh= ‖v‖H.
43 / 215
![Page 102: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/102.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Example: Mesh
An example of mesh for the unitary square in IR2
44 / 215
![Page 103: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/103.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Finite-element framework
Finally, assuming h < 1 and t > 0, and choosing η = O(h), wehave
‖u − u(k)h ‖H ≤ C ∗(ht)‖u‖H + 2‖u − uh‖H ≤ C (h).
where
I u(x) is the exact solution of the variational problem,
I uh(x) is the exact solution of the approximate problem,
I u(k)h (x) =
∑Ni=1 u
(k)h φi (x) is the approximate solution at step
k. (φi (x) are the basis functions)
45 / 215
![Page 104: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/104.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Test problemsProblem 1
k(x) =
1 x ∈ Ω \ Ω1 ∪ Ω2 ∪ Ω3,10−6 x ∈ Ω1,
10−4 x ∈ Ω2,
10−2 x ∈ Ω3.
Problem 2
k(x) =
1 x ∈ Ω \ Ω1 ∪ Ω2 ∪ Ω3,106 x ∈ Ω1,
104 x ∈ Ω2,
102 x ∈ Ω3.
M. Arioli
4 Numerical experiments
We generated two test problem classes using FEMLAB c⃝ under Matlab c⃝.The first test problem class is define on a L-shape domain ! of R2. Thesecond test problem class is defined on the cube [0, 1] × [0, 1] × [0, 1]. Inboth the classes, we chose boundary condition zero and, in the conjugategradient algorithm, the staring point u(0) = 0. Finally, in all the figures, theestimate of the energy norm stops d steps before the final iteration becauseof the choice of our stopping criterion, and the values in the legends of thefigures are:
– ||Au(k) − b||2/||b||2, the value of the residual at step k is computed usingA;
– ||δu||A/||u||A = ||u − u(k)||A/||u||A energy norm of the algebraic error;– ||δu||a/||u||a = ||u − u(k)
h ||a/||u||a =!a(u, u) − bT u(k)
"1/2 error inenergy between the solution of (4) and current solution at step k.
4.1 L-shape test problems
In Fig. 2, we plot the geometry of the domain !. In problem (4), we choosethe functional L(v) =
#!
10vdx, ∀v ∈ H 10 (!), and in the bilinear form (1),
the function K(x) ∈ L∞(!) takes different values in each subdomain. In thefirst test problem within this class we have:
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Ω
Ω3
Ω1 Ω
2
Γ
Fig. 2. Geometry of the domain !
L(v) =∫
Ω10vdx, ∀v ∈ H1
0 (Ω)46 / 215
![Page 105: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/105.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Preconditioners: estimates for κ(M−1A)
M Problem 1 Problem 2
I 3.6 108 1.8 1010
Jacobi 2.4 104 1.5 109
Inc. Cholesky(0) 7.2 103 4.3 108
η2 = 3.44.30510−5 and N = 29619.
The condition numbers of the preconditioned matrices M−1A forthe second problem are are still very high, and only the incompleteCholesky preconditioner with drop tolerance 10−2 is an effectivechoice.
47 / 215
![Page 106: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/106.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Example: Problem 1
M. Arioli
and Problem 2. Therefore, by this approximate value E(u), we estimated theerror at step k:
||δu||a||u||a
=!
1 −a(u(k)
h , u(k)h )
E(u)
"1/2
≈ ||u(k)h (x) − u(x)||a/||u(x)||a.
In the experiments, u(k) is the computed value at iteration k of the conjugategradient algorithm.
We compare the behaviour of
∥u − u(k)∥A
∥u∥A
= ||Au(k) − b||A−1
||b||A−1,
with the corresponding estimate ξk/(bT u(0) + r(0)T u(k)), and the value of∥Au(k)−b∥2/∥b∥2. Moreover, we plot the values at each step k of ||δu||a/||u||a .
The stopping criteria normally used are based on the values of ∥Au(k) −b∥2/∥b∥2 [2]. In the practice, the conjugate gradient algorithm is stoppedwhen ∥Au(k) − b∥2/∥b∥2 ≤
√ε.
4.1.1 Problem 1. In Fig. 3 and Fig. 4, respectively for the Jacobi and theincomplete Cholesky decomposition preconditioners and for d = 5, wepresent the history of convergence for Problem 1. During the initial
0 5 10 15 20 25 30 35 40 4510
3
102
101
100
101
102
Iteration
Res
idua
l Nor
ms
L Shape 2D Jacobi Preconditioner (d = 5)
|| δ u ||A / ||u||
AH S estimate || δ u ||
A / ||u||
A|| u u(k)||
a/|| u||
a||A u b||
2 / ||b||
2
Fig. 3. Behaviour of the norms of the residual for the Jacobi preconditioner in Problem 1
Behaviour of the norms of the residual for the Jacobipreconditioner.
48 / 215
![Page 107: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/107.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Example: Problem 1A stopping criterion for the conjugate gradient algorithm
0 5 10 15 20 25 3010
3
10 2
10 1
100
101
Iteration
Res
idua
l Nor
ms
L Shape 2D Cholesky(0) Preconditioner (d = 5)
|| δ u ||A / ||u||
AH S estimate || δ u ||
A / ||u||
A|| u u(k)||
a/|| u||
a||A u b||
2 / ||b||
2
Fig. 4. Behaviour of the norms of the residual for the incomplete Cholesky preconditionerin Problem 1
iterations of the conjugate gradient algorithm, the ratio between ||u(k)||A and||u||A is relatively large, as can be seen from Fig. 5 relatively to the Jacobipreconditioner. Nevertheless, the ratio value quickly stabilises itself close to1. We obtained similar plots for the incomplete Cholesky preconditioner.
4.1.2 Problem 2. Problem 2 is harder to solve. Both Jacobi and Incom-plete Cholesky without fill-in failed for small values of d . In Fig. 6, we plotthe estimates relative to several values of d for the Jacobi preconditioner.Only when d ≥ 90, the oscillations were smaller than η and, then, the algo-rithm stopped with an accurate solution. In Fig. 7 and Fig. 8, we present theconvergence history for the cases relative to the incomplete Cholesky withdrop tolerance 10−2 and d = 10 and d = 20 respectively. In these cases,the good preconditioner allows to choose a small value for d. Nonetheless,the convergence is not particularly fast and we can see in Fig. 9 that theratio between the lower bound (19) and ||u||2A stagnates. Finally, in Fig. 10,we forced the large value of d = 160 when using the incomplete Choleskypreconditioner with drop tolerance 10−2. We point out that in this case the∥Au(k) − b∥2/∥b∥2 does not go under the value
√ε. Therefore, in this case,
the criterion based on the Euclidean norm of the residual gives a misleadinginformation about the iterative process.
Behaviour of the norms of the residual for the incomplete Choleskypreconditioner.
49 / 215
![Page 108: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/108.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Example: Problem 2
M. Arioli
0 20 40 60 80 100 1200
0.2
0.4
0.6
0.8
1
1.2
Iteration
Rat
io
LShape 2D Cholesky(1e2) Preconditioner (d = 10)
bT u(k) / ||u||A2
Fig. 9. Ratio bT u(k)/||u||2A for the incomplete Cholesky preconditioner with drop toler-ance 10−2 and d = 10 in Problem 2
0 50 100 150 200 250 30010
10
10 8
10 6
10 4
10 2
100
102
Iteration
Res
idua
l Nor
ms
L Shape 2D Cholesky(1e 2) Preconditioner (d = 160)
|| δ u ||A / ||u||
AH S estimate || δ u ||
A / ||u||
A|| u u(k)||
a/|| u||
a||A u b||
2 / ||b||
2
Fig. 10. Behaviour of the norms of the residual for the incomplete Cholesky preconditionerwith drop tolerance 10−2 and d = 160 in Problem 2
50 / 215
![Page 109: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/109.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Example: Problem 2 M. Arioli
0 20 40 60 80 100 1200
0.2
0.4
0.6
0.8
1
1.2
Iteration
Rat
io
LShape 2D Cholesky(1e2) Preconditioner (d = 10)
bT u(k) / ||u||A2
Fig. 9. Ratio bT u(k)/||u||2A for the incomplete Cholesky preconditioner with drop toler-ance 10−2 and d = 10 in Problem 2
0 50 100 150 200 250 30010
10
10 8
10 6
10 4
10 2
100
102
Iteration
Res
idua
l Nor
ms
L Shape 2D Cholesky(1e 2) Preconditioner (d = 160)
|| δ u ||A / ||u||
AH S estimate || δ u ||
A / ||u||
A|| u u(k)||
a/|| u||
a||A u b||
2 / ||b||
2
Fig. 10. Behaviour of the norms of the residual for the incomplete Cholesky preconditionerwith drop tolerance 10−2 and d = 160 in Problem 2
51 / 215
![Page 110: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/110.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Example: Problem 2
M. Arioli
0 5 10 15 20 25 30 35 40 450
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Iteration
Rat
io
LShape 2D Jacobi Preconditioner (d = 5)
bT u(k) / ||u||A2
Fig. 5. Ratio bT u(k)/||u||2A for the Jacobi preconditioner in Problem 1
0 200 400 600 800 1000 120010
5
104
103
102
101
100
101
102
103
104
Iteration
Err
or e
stim
ates
L Shape 2D Jacobi Preconditioner
||A u b||2 / ||b||
2|| u u(k)||
a/|| u||
ad = 10d = 70d = 90d = 130
Fig. 6. Comparison of several estimates of the energy error for d = 10, 70, 90, 130 inProblem 2
52 / 215
![Page 111: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/111.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Lecture 3
53 / 215
![Page 112: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/112.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
The non symmetric positive definite problem
I a(u, v) 6= a(v , u)
I A asymmetric but positive definite
I H = 12 (AT + A) SPD
I A = 12 (AT + A) + 1
2 (AT − A) = H−N
How to calculate ‖r(k)‖H−1?
I Solve preconditioned system
H−1/2AH−1/2u = H−1/2b
I ‖r(k)‖l2 = ‖r(k)‖H−1
I 3-term recurrence
I Approximate it from Krylov subspace information.
See A., Login, and WathenNumer. Math. (2004) (DOI) 10.1007/s00211-004-0568-z
A. and Loghin Electronic Transactions on Numerical Analysis. 29,
(2008).54 / 215
![Page 113: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/113.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup framework
Solve
Au = f
given
supw∈IRn\0
supv∈IRn\0
wTAv
‖v‖H‖w‖H≤ C1 (sup-sup)
infw∈IRn\0
supv∈IRn\0
wTAv
‖v‖H‖w‖H≥ C2 (inf-sup)
Note: ‖vh‖Hh= ‖v‖H defines the spd matrix H.
55 / 215
![Page 114: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/114.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
3-term recurrence Algorithm
Kk = span
H−1f,H−1Nf, . . . ,(H−1N
)k−1f
56 / 215
![Page 115: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/115.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
3-term recurrence Algorithm
Kk = span
H−1f,H−1Nf, . . . ,(H−1N
)k−1f
We compute the Lanczos vectors v(j) by a 3-term recurrence:
αjv(j+1) = H−1Nv(j) − γjv(j) − βjv(j−1), j ≥ 0
with v(−1) = 0 and v(0) = H−1Nf The coefficients αj , γj , and βjare chosen such that
v(i)THv(j) = δij
i.e. they are H orthogonal. Widlund SINUM,15, 1978
56 / 215
![Page 116: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/116.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
3-term recurrence Algorithm
Kk = span
H−1f,H−1Nf, . . . ,(H−1N
)k−1f
We compute the Lanczos vectors v(j) by a 3-term recurrence:
αjv(j+1) = H−1Nv(j) − γjv(j) − βjv(j−1), j ≥ 0
with v(−1) = 0 and v(0) = H−1Nf The coefficients αj , γj , and βjare chosen such that
v(i)THv(j) = δij
i.e. they are H orthogonal. Widlund SINUM,15, 1978 This ispossible only in this case for the peculiar preconditioning and theSkew-Symmetry of N. In general, we cannot have 3-term recurrentformulae for non-symmetric matrices (see Faber-ManteuffelSINUM, 21, 1984)
56 / 215
![Page 117: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/117.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
One crime
Replace‖u − uh‖Hh
≤ C (h)
with‖u − u
(k)h ‖Hh
≤ C (h)
Sufficient condition
‖u − uh‖Hh+ ‖uh − u
(k)h ‖Hh
∼ O(C (h))
⇓‖uh − u
(k)h ‖Hh
∼ O(C (h))
57 / 215
![Page 118: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/118.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
One crime
Replace‖u − uh‖Hh
≤ C (h)
with‖u − u
(k)h ‖Hh
≤ C (h)
Sufficient condition
‖u − uh‖Hh+ ‖uh − u
(k)h ‖Hh
∼ O(C (h))
⇓‖uh − u
(k)h ‖Hh
∼ O(C (h))
57 / 215
![Page 119: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/119.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Stopping criteria
A general stopping criterion:
‖uh − u(k)h ‖Hh
= ‖u− u(k)‖H ≤ C (h)
Residual equationr(k) = A(u− u(k))
⇓
‖u− u(k)‖H = ‖A−1r(k)‖H = ‖r(k)‖A−T HA−1 ≤ C (h)
58 / 215
![Page 120: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/120.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Stopping criteria
A general stopping criterion:
‖uh − u(k)h ‖Hh
= ‖u− u(k)‖H ≤ C (h)
Residual equationr(k) = A(u− u(k))
⇓
‖u− u(k)‖H = ‖A−1r(k)‖H = ‖r(k)‖A−T HA−1 ≤ C (h)
58 / 215
![Page 121: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/121.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Stopping criteria
Lemma Let (inf-sup) hold. Then
‖r(k)‖A−T HA−1 ≤ C−12 ‖r(k)‖H−1 .
New stopping criterion
‖r(k)‖H−1 ≤ C2C (h)‖u(k)‖H.
59 / 215
![Page 122: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/122.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Stopping criteria
Lemma Let (inf-sup) hold. Then
‖r(k)‖A−T HA−1 ≤ C−12 ‖r(k)‖H−1 .
New stopping criterion
‖r(k)‖H−1 ≤ C2C (h)‖u(k)‖H.
59 / 215
![Page 123: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/123.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Examples
Elliptic problems in IR2 (Ω unit square)
−∇ · (a(x)∇u) + b(x) · ∇u + c(x)u = f in Ω
u = 0 on Γ.
where(a)ij , (b)i , c ∈ L∞(Ω), i , j = 1, 2,
k2(x) |ξξξ|2 ≤ ξξξTa(x)ξξξ ≤ k1(x) |ξξξ|2 ,
c(x)− 1
2∇ · b(x) ≥ 0 ∀x ∈ Ω.
60 / 215
![Page 124: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/124.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Examples
a(w , v) = (a · ∇w ,∇v) + (b · ∇w , v) + (cw , v),
is continuous and coercive with
C1 = ‖k1‖L∞(Ω) + ‖b‖L∞(Ω) + C (Ω)‖c‖L∞(Ω),
C2 = minx∈Ω
k2(x),
wrt ‖ · ‖H = | · |H10 (Ω) := | · |1.
61 / 215
![Page 125: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/125.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Examples
Error estimate:
|u − uh|1 ≤ Chs−1‖u‖s , 1 ≤ s ≤ 2.
Issues
I What is h?
I How to approximate ‖u‖s?
62 / 215
![Page 126: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/126.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Examples
Error estimate:
|u − uh|1 ≤ Chs−1‖u‖s , 1 ≤ s ≤ 2.
Issues
I What is h?
I How to approximate ‖u‖s?
62 / 215
![Page 127: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/127.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments
I Discretization:linear elements on uniform & adaptive meshes
I Estimation of parameters
h ∼ ‖uk‖M
‖uk‖l2, ‖u‖s ∼ ‖Auk‖l2
63 / 215
![Page 128: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/128.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments
I Discretization:linear elements on uniform & adaptive meshes
I Estimation of parameters
h ∼ ‖uk‖M
‖uk‖l2, ‖u‖s ∼ ‖Auk‖l2
63 / 215
![Page 129: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/129.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments
Stopping criteria and estimates
I Residual dual norm: ‖rk‖H−1
I Energy estimate ‖uk − uk−1‖H ≤ C2h2‖Auk‖l2
64 / 215
![Page 130: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/130.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments
Stopping criteria and estimates
I Residual dual norm: ‖rk‖H−1
I Energy estimate ‖uk − uk−1‖H ≤ C2h2‖Auk‖l2
64 / 215
![Page 131: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/131.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Advection-diffusion problem
−ε∆u + b · ∇u = f in Ω
u = g on Γ.
b = (2y(1− x2),−2x(1− y 2)),
u(x , y) = x
(1− e
y−1ε
1− e−2ε
),
‖vh‖2Hh
= ε|vh|21 +∑T∈T h
δT‖b · ∇vh‖20,T
65 / 215
![Page 132: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/132.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Advection-diffusion problem
−1
0
1
−1−0.5
00.5
1−1
−0.5
0
0.5
1
ε = 10−2
66 / 215
![Page 133: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/133.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Advection-diffusion problem
0 20 40 60 80 100 120 14010−12
10−10
10−8
10−6
10−4
10−2
100
true errorresidual dual norm2−norminterp errorfinal errorfinal interp errorenergy estimateerror estimate
Uniform mesh; ε = 1
67 / 215
![Page 134: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/134.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Advection-diffusion problem
0 50 100 150 200 250 30010−12
10−10
10−8
10−6
10−4
10−2
100
true errorresidual dual norm2−norminterp errorfinal errorfinal interp errorenergy estimateerror estimate
Uniform mesh; ε = 10−1
68 / 215
![Page 135: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/135.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Advection-diffusion problem
0 50 100 150 200 250 300 35010−12
10−10
10−8
10−6
10−4
10−2
100
true errorresidual dual norm2−norminterp errorfinal errorfinal interp errorenergy estimateerror estimate
Uniform mesh; ε = 10−2
69 / 215
![Page 136: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/136.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
How to calculate ‖rk‖H−1?
I Solve preconditioned system
H−1/2AH−1/2u = H−1/2f
I ‖rk‖l2 = ‖rk‖H−1
I 3-term recurrence.
I Approximate it from Krylov subspace information.
70 / 215
![Page 137: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/137.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
How to calculate ‖rk‖H−1?
I Concus & Golub, Widlund: 3-term recurrences fornonsymmetric problems
I work in H-inner productI do not minimize the residual norm.
RecallKk(r0,A) = span
r0,Ar0, . . . ,Ak−1r0
Arnoldi process
VTk AVk = Hk
where VTk Vk = Ik and Hk= Hessenberg.
71 / 215
![Page 138: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/138.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
How to calculate ‖rk‖H−1?
Lemma Arnoldi applied to
Kk(r0, A) ≡ Kk(H−1/2r0,H−1/2AH−1/2)
and Arnoldi in the H-inner product applied to
Kk(r0, A) ≡ Kk(H−1r0,H−1A)
produce the same Hk . Moreover,
(Hk)ij = 0, |i − j | > 1.
72 / 215
![Page 139: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/139.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
CG Conclusions
FINAL MESSAGE: DO NOT ACCURATELY COMPUTE THESOLUTION OF AN INACCURATE PROBLEM
73 / 215
![Page 140: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/140.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operators
Let M ∈ IRm×m and N ∈ IRn×n be symmetric positive definitematrices, and let A ∈ IRm×n be a full rank matrix.
M = v ∈ IRm; ‖u‖2M = vTMv, N = q ∈ IRn; ‖q‖2
N = qTNq
M? = w ∈ IRm; ‖w‖2M−1 = wTM−1w,
N? = y ∈ IRn; ‖y‖2N−1 = yTN−1y
74 / 215
![Page 141: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/141.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operators
Let M ∈ IRm×m and N ∈ IRn×n be symmetric positive definitematrices, and let A ∈ IRm×n be a full rank matrix.
M = v ∈ IRm; ‖u‖2M = vTMv, N = q ∈ IRn; ‖q‖2
N = qTNq
M? = w ∈ IRm; ‖w‖2M−1 = wTM−1w,
N? = y ∈ IRn; ‖y‖2N−1 = yTN−1y
〈v,Aq〉M,M? = vTAq, Aq ∈ L(M) ∀q ∈ N.
74 / 215
![Page 142: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/142.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear operators
Let M ∈ IRm×m and N ∈ IRn×n be symmetric positive definitematrices, and let A ∈ IRm×n be a full rank matrix.
M = v ∈ IRm; ‖u‖2M = vTMv, N = q ∈ IRn; ‖q‖2
N = qTNq
M? = w ∈ IRm; ‖w‖2M−1 = wTM−1w,
N? = y ∈ IRn; ‖y‖2N−1 = yTN−1y
〈v,Aq〉M,M? = vTAq, Aq ∈ L(M) ∀q ∈ N.
The adjoint operator AF of A can be defined as
〈AFg, f〉N?,N = fTATg, ATg ∈ L(N) ∀g ∈M.
74 / 215
![Page 143: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/143.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Elliptic SVD
Given q ∈M and v ∈ N, the critical points for the functional
vTAq
‖q‖N ‖v‖M
are the “elliptic singular values and singular vectors’’ of A.
75 / 215
![Page 144: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/144.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Elliptic SVD
Given q ∈M and v ∈ N, the critical points for the functional
vTAq
‖q‖N ‖v‖M
are the “elliptic singular values and singular vectors’’ of A.The saddle-point conditions are
Aqi = σiMvi vTi Mvj = δijATvi = σiNqi qT
i Nqj = δij
σ1 ≥ σ2 ≥ · · · ≥ σn > 0
75 / 215
![Page 145: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/145.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Elliptic SVD
Given q ∈M and v ∈ N, the critical points for the functional
vTAq
‖q‖N ‖v‖M
are the “elliptic singular values and singular vectors’’ of A.The saddle-point conditions are
Aqi = σiMvi vTi Mvj = δijATvi = σiNqi qT
i Nqj = δij
σ1 ≥ σ2 ≥ · · · ≥ σn > 0
The elliptic singular values are the standard singular values of
A = M−1/2AN−1/2. The elliptic singular vectors qi and vi , i = 1, . . . , n
are the transformation by M−1/2 and N−1/2 respectively of the left and
right standard singular vector of A.
75 / 215
![Page 146: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/146.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Quadratic programming
The general problem
minAT w=r
1
2wTWw − gTw
where the matrix W is positive semidefinite andker(W) ∩ ker(AT ) = 0 can be reformulated by choosing
M = W + νAN−1AT
u = w −M−1gb = r − ATM−1g.
as a projection problem
minAT u=b
‖u‖2M
If W is non singular then we can choose ν = 0.
76 / 215
![Page 147: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/147.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Augmented system
The augmented system that gives the optimality conditions for theprojection problem:[
M AAT 0
] [up
]=
[0b
].
77 / 215
![Page 148: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/148.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized Golub-Kahan bidiagonalization
In Golub Kahan (1965), Paige Saunders (1982), several algorithmsfor the bidiagonalization of a m × n matrix are presented. All ofthem can be theoretically applied to A and their generalization toA is straightforward as shown by Bembow (1999). Here, we wantspecifically to analyse one of the variants known as the”Craig”-variant (see Paige Saunders (1982), Saunders(1995,1997)).
78 / 215
![Page 149: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/149.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized Golub-Kahan bidiagonalization
AQ = MV
[B0
]VTMV = Im
AT V = NQ[BT ; 0
]QTNQ = In
where
B =
α1 0 0 · · · 0
β2 α2 0. . . 0
.... . .
. . .. . .
. . .
0 · · · βn−1 αn−1 0
0 · · · 0 βn αn
0 · · · 0 0 βn+1
.
78 / 215
![Page 150: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/150.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized Golub-Kahan bidiagonalization
AQ = MV
[B0
]VTMV = Im
ATV = NQ[BT ; 0
]QTNQ = In
where
B =
α1 β1 0 · · · 0
0 α2 β2. . . 0
.... . .
. . .. . .
. . .
0 · · · 0 αn−1 βn−1
0 · · · 0 0 αn
.
78 / 215
![Page 151: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/151.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Algorithm
The augmented system that gives the optimality conditions forminAT u=b ‖u‖2
M [M AAT 0
] [up
]=
[0b
]can be transformed by the change of variables
u = Vzp = Qy
79 / 215
![Page 152: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/152.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Algorithm
In 0 B0 Im−n 0
BT 0 0
z1
z2
y
=
00
QTb
.
79 / 215
![Page 153: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/153.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Algorithm
[In B
BT 0
] [z1
y
]=
[0
QTb
].
79 / 215
![Page 154: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/154.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Algorithm
[In B
BT 0
] [z1
y
]=
[0
QTb
].
QTb = e1‖b‖N
the value of z1 will correspond to the first column of the inverse ofB multiplied by ‖b‖N.
79 / 215
![Page 155: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/155.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Algorithm
Thus, we can compute the first column of B and of V:α1Mv1 = Aq1, such as
w = M−1Aq1
α1 = wTMw = wAq1
v1 = w/√α1.
80 / 215
![Page 156: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/156.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Algorithm
Thus, we can compute the first column of B and of V:α1Mv1 = Aq1, such as
w = M−1Aq1
α1 = wTMw = wAq1
v1 = w/√α1.
Finally, knowing q1 and v1 we can start the recursive relations
gi+1 = N−1(ATvi − αiNqi
)βi+1 = gTNg
qi+1 = g√βi+1
w = M−1 (Aqi+1 − βi+1Mvi )αi+1 = wTMwvi+1 = w/
√αi+1.
80 / 215
![Page 157: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/157.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
u
Thus, the value of u can be approximated when we have computedthe first k columns of V by
u(k) = Vkzk =k∑
j=1
ζjvj .
81 / 215
![Page 158: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/158.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
u
Thus, the value of u can be approximated when we have computedthe first k columns of V by
u(k) = Vkzk =k∑
j=1
ζjvj .
The entries ζj of zk can be easily computed recursively startingwith
ζ1 = −‖b‖N
α1
as
ζi+1 = − βiαi+1
ζi i = 1, . . . , n
81 / 215
![Page 159: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/159.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
p
Approximating p = Qy by p(k) = Qkyk =∑k
j=1 ψjqj , we have that
yk = −B−1k zk .
82 / 215
![Page 160: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/160.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
p
Approximating p = Qy by p(k) = Qkyk =∑k
j=1 ψjqj , we have that
yk = −B−1k zk .
Following an observation made by Paige and Saunders, we caneasily transform the previous relation into a recursive one whereonly one extra vector is required.
82 / 215
![Page 161: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/161.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
p
Approximating p = Qy by p(k) = Qkyk =∑k
j=1 ψjqj , we have that
yk = −B−1k zk .
From p(k) = −QkB−1k zk = −
(B−Tk QT
k
)Tzk and Dk = B−Tk QT
k
di =qi − βidi−1
αii = 1, . . . , n
(d0 = 0
)where dj are the columns of D.Starting with p(1) = −ζ1d1 and u(1) = ζ1v1
u(i+1) = u(i) + ζi+1vi+1
p(i+1) = p(i) − ζi+1di+1
i = 1, . . . , n
82 / 215
![Page 162: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/162.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Stopping criteria
‖u− u(k)‖2M = ‖e(k)‖2
M =n∑
j=k+1
ζ2j =
∣∣∣∣∣∣z− [ zk0
] ∣∣∣∣∣∣22.
83 / 215
![Page 163: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/163.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Stopping criteria
‖u− u(k)‖2M = ‖e(k)‖2
M =n∑
j=k+1
ζ2j =
∣∣∣∣∣∣z− [ zk0
] ∣∣∣∣∣∣22.
‖ATu(k) − b‖N−1 = |βk+1 ζk | ≤ σ1|ζk | = ‖A‖2|ζk |.
83 / 215
![Page 164: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/164.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Stopping criteria
‖u− u(k)‖2M = ‖e(k)‖2
M =n∑
j=k+1
ζ2j =
∣∣∣∣∣∣z− [ zk0
] ∣∣∣∣∣∣22.
‖ATu(k) − b‖N−1 = |βk+1 ζk | ≤ σ1|ζk | = ‖A‖2|ζk |.
‖p− p(k)‖N =∣∣∣∣∣∣QB−1
(z−
[zk0
]) ∣∣∣∣∣∣N≤ ‖e
(k)‖M
σn.
83 / 215
![Page 165: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/165.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Error bound
Lower bound We can estimate ‖e(k)‖2M by the lower bound
ξ2k,d =
k+d+1∑j=k+1
ζ2j < ‖e(k)‖2
M.
Given a threshold τ < 1 and an integer d , we canstop the iterations when
ξ2k,d ≤ τ
k+d+1∑j=1
ζ2j < τ‖u‖2
M.
Upper bound Despite being very inexpensive, the previousestimator is still a lower bound of the error. We canuse an approach inspired by the Gauss-Radauquadrature algorithm and similar to the onedescribed in Golub-Meurant (2010).
84 / 215
![Page 166: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/166.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Error bound
Lower bound We can estimate ‖e(k)‖2M by the lower bound
ξ2k,d =
k+d+1∑j=k+1
ζ2j < ‖e(k)‖2
M.
Given a threshold τ < 1 and an integer d , we canstop the iterations when
ξ2k,d ≤ τ
k+d+1∑j=1
ζ2j < τ‖u‖2
M.
Upper bound Despite being very inexpensive, the previousestimator is still a lower bound of the error. We canuse an approach inspired by the Gauss-Radauquadrature algorithm and similar to the onedescribed in Golub-Meurant (2010).
84 / 215
![Page 167: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/167.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup
Let H and P be two Hilbert spaces, and H? and P? thecorresponding dual spaces. Let
a(u, v) : H× H→ IR b(u, q) : H×P→ IR|a(u, v)| ≤ ‖a‖ ‖u‖H ‖u‖H ∀u ∈ H,∀v ∈ H|b(u, q)| ≤ ‖b‖ ‖v‖H ‖q‖P ∀u ∈ H, ∀q ∈ P
be continuous bilinear forms with ‖a‖ and ‖b‖ the correspondingnorms. Given f ∈ H? and g ∈ P?, we seek the solutions u ∈ H andp ∈ P of the system
a(u, v) + b(v , p) = 〈f , v〉H?,H ∀v ∈ Hb(u, q) = 〈g , q〉P?,P ∀q ∈ P.
(2)
85 / 215
![Page 168: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/168.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-supWe can introduce the operators M , A and its adjoint A F
M : H→ H?, 〈M u, v〉H?×H = a(u, v) ∀u ∈ H,∀v ∈ HA F : H→ P?, 〈A Fu, q〉P?×P = b(u, q) ∀u ∈ H,∀q ∈ PA : P→ H?, 〈v ,A p〉H×H? = b(v , p) ∀v ∈ H, ∀p ∈ P
and we have
〈A Fu, q〉P?×P = 〈u,A q〉H×H? = b(u, q).
In order to make the following discussion simpler, we assume thata(u, v) is symmetric and coercive on H
0 < χ1‖u‖H ≤ a(u, u).
However, Brezzi:1991 the coercivity on the kernel of A F,Ker(A F) is sufficient. We will also assume that ∃χ0 > 0 suchthat
supv∈H
b(v , q)
‖v‖H≥ χ0‖q‖P\Ker(A ) = χ0
[inf
q0∈Ker(A )‖q + q0‖P
].
85 / 215
![Page 169: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/169.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup
Under these hypotheses, and for any f ∈ H? and g ∈ Im(A F)then there exists (u, p) solution of saddle problem: u is unique andp is definite up to an element of Ker(A ).
85 / 215
![Page 170: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/170.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup and Mixed finite-element method
Let now Hh → H and Ph → P be two finite dimensional subspacesof H and P. As for the problem (2), we can introduce theoperators Ah : Ph → H?h and Mh;Hh → H?h. We also assume that
Ker(Ah) ⊂ Ker(A )
supvh∈Hh
b(vh, qh)
‖vh‖H≥ χn‖qh‖P\Ker(Ah)
χn ≥ χ0 > 0.
86 / 215
![Page 171: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/171.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup and Mixed finite-element method
Under the hypotheses of inf-sup, we have that∃(uh, ph) ∈ Hh ×Ph solution of
a(uh, vh) + b(vh, ph) = 〈f , vh〉H?h ,Hh∀vh ∈ Hh
b(uh, qh) = 〈g , qh〉P?h ,Ph∀qh ∈ Ph.
and
‖u − uh‖H + ‖p − ph‖P\Ker(A) ≤
κ
(inf
vh∈Hh
‖u − vh‖H + infqh∈Ph
‖p − qh‖P),
where κ = κ(‖a‖, ‖b‖, χ0, χ1) is independent of h.
86 / 215
![Page 172: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/172.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup and Mixed finite-element method
Let φii=1,...,m be a basis for Hh and ψjj=1,...,n be a basis forPh. Then, the matrices M and N are the Grammian matrices ofthe operators M and A . In order to use the latter theory, we needto weaken the hypothesis, made in the introduction, that A be fullrank. In this case, we have that
I s elliptic singular values will be zero;
I however, the G-K bidiagonalization method will still work and,if Aq1 6= 0, it will compute a matrix B of rank less than orequal to n − s.
86 / 215
![Page 173: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/173.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup and Mixed finite-element method
On the basis of the latter observations, the error ‖e(k)‖M can bestill computed. Finally, we point out that for h ↓ 0 the ellipticsingular values of all A ∈ IRmh×nh will be bounded with upper andlower bounds independent of h, i.e.
χ0 ≤ σnh ≤ · · · ≤ σ1 ≤ ‖a‖.
86 / 215
![Page 174: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/174.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup and Mixed finite-element method
TheoremUnder previous hypotheses, and denoting by u∗ one of the iteratesof Algorithm Craig for which ‖e(k)‖M < τ , we have
‖u − u∗‖H + ‖p − p∗‖P\Ker(A ) ≤
κ
(inf
vh∈Hh
‖u − vh‖H + infqh∈Ph
‖p − qh‖P + τ
),(2)
where u∗ =∑nh
i=1 φiu∗i ∈ Hh, p∗ =
∑nhj=1 φip
∗j ∈ Ph and κ a
constant independent of h.
86 / 215
![Page 175: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/175.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Two examples
StokesThe Stokes problems have been generated using the softwareprovided by ifiss3.0 package (Elman, Ramage, and Silvester). Weuse the default geometry of “Step case” and the Q2-Q1approximation described in ifiss3.0 manual and in Elman,Silvester, and Wathen (2005).
name m n nnz(M) nnz(A)
Step1 418 61 2126 1603Step2 1538 209 10190 7140Step3 5890 769 44236 30483Step4 23042 2945 184158 126799Step5 91138 11521 751256 518897
(nnz(M) is only for the symmetric part)
87 / 215
![Page 176: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/176.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Two examples
name # Iter.s ‖e(k)‖2 ‖ATu(k) − b‖2 ‖p− p(k)‖2 κ(B)
Step1 30 6.8e-16 5.1e-16 1.1e-13 7.6Step2 32 5.4e-14 5.4e-14 5.0e-12 7.7Step3 34 3.8e-14 2.7e-14 1.0e-11 7.8Step4 34 5.0e-13 1.3e-13 1.4e-10 7.8Step5 35 1.8e-13 3.1e-14 1.7e-10 7.8
Stokes (Step) problems results (d = 5, τ = 10−8).
87 / 215
![Page 177: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/177.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Two examplesPoisson with mixed b.c. Problems The Poisson problem is castedin its dual form as a Darcy’s problem:
Find w ∈ H = ~q | ~q ∈ Hdiv (Ω), ~q · n = 0 on ∂N(Ω) , u ∈ L2(Ω) s.t.∫Ω ~w · ~q +
∫Ω) div(~q)u =
∫∂D(Ω) uD~q · n ∀~q ∈ H∫
Ω div(~w)v =∫
Ω fv ∀v ∈ L2(Ω).
We approximated the spaces H and L2(Ω) by RT0 and bypiecewise constant functions respectively The matrix N is the massmatrix for the piecewise constant functions and it is a diagonalmatrix with diagonal entries equal to the area of the correspondingtriangle. The matrix M has been chosen such that eachapproximation Hh of H is
Hh =
q ∈ IRm ‖q‖2Hh
= qTMq.
Therefore, denoting by W the mass matrix for Hh, we have
M = W + AN−1AT .
87 / 215
![Page 178: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/178.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Two examples
Poisson with mixed b.c. Problems
h = 2−k m n nnz(M) nnz(A)
2−6 12288 8192 36608 244482−7 49152 32768 146944 980482−8 196608 131072 588800 3927042−9 786432 524288 2357248 1571840
(nnz(M) is only for the symmetric part)
With the chosen boundary conditions, it is easy to verify that thecontinuous solution u is u(x , y) = x .We point out that the pattern of W is structurally equal to thepattern AN−1AT .
87 / 215
![Page 179: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/179.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Two examples
name # Iter.s ‖e(k)‖2 ‖ATu(k) − b‖2 ‖p− p(k)‖2 κ(B)
h = 2−6 10 2.8e-12 2.9e-16 4.1e-11 1.05h = 2−7 10 9.7e-12 3.0e-16 2.6e-10 1.05h = 2−8 10 2.5e-11 3.0e-16 7.9e-10 1.05h = 2−9 10 2.9e-10 2.8e-16 1.3e-08 1.05
Poisson with mixed b.c. data and RT0 problem results (d = 5,τ = 10−8).
87 / 215
![Page 180: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/180.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Lecture on inf-sup
88 / 215
![Page 181: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/181.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup
Let H1 and H2 be two Hilbert space, and H?1 and H?2 thecorresponding dual spaces. Let
a(u, v) : H1 × H2 → IR
supu∈H1supv∈H2
|a(u, v)|‖u‖H1 ‖v‖H2
≤ C1 ∀u,∈ H1, ∀v ∈ H2
infu∈H1 supv∈H2
|a(u, v)|‖u‖H1 ‖v‖H2
≥ C2 ∀u ∈ H1,∀v ∈ H2
be continuous bilinear forms with ‖a‖ the corresponding norms.Given f ∈ H?2 , we seek the solutions u ∈ H1 of
a(u, v) = 〈f , v〉H?2 ,H2 ∀v ∈ H2 (3)
89 / 215
![Page 182: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/182.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup
Theorem. The inf-sup condition is equivalent to
∀v ∈ H2∃u ∈ H1 s.t.a(u, v) ≥ c1||v ||2H2
and ||u||H1 ≤ c2||v ||H2 .
IF H1 = H2 THEN the inf-sup is the coercivity condition
89 / 215
![Page 183: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/183.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup
Solve ( we assume that we have approximate the Hilbert spaceswith finite dimensional ones)
Au = f
given
maxw∈IRn\0
maxv∈IRn\0
wTAv
‖v‖H‖w‖H≤ C1 (sup-sup)
minw∈IRn\0
maxv∈IRn\0
wTAv
‖v‖H‖w‖H≥ C2 (inf-sup)
Note: ‖vh‖Hh= ‖v‖H defines the spd matrix H.
89 / 215
![Page 184: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/184.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup
Let H and P be two Hilbert spaces, and H? and P? thecorresponding dual spaces. Let
a(u, v) : H× H→ IR b(u, q) : H×P→ IR|a(u, v)| ≤ ‖a‖ ‖u‖H ‖v‖H ∀u ∈ H,∀v ∈ H|b(u, q)| ≤ ‖b‖ ‖u‖H ‖q‖P ∀u ∈ H,∀q ∈ P
be continuous bilinear forms with ‖a‖ and ‖b‖ the correspondingnorms. Given f ∈ H? and g ∈ P?, we seek the solutions u ∈ H andp ∈ P of the system
a(u, v) + b(v , p) = 〈f , v〉H?,H ∀v ∈ Hb(u, q) = 〈g , q〉P?,P ∀q ∈ P.
(3)
90 / 215
![Page 185: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/185.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-supWe can introduce the operators M , A and its adjoint A F
M : H→ H?, 〈M u, v〉H?×H = a(u, v) ∀u ∈ H,∀v ∈ HA F : H→ P?, 〈A Fu, q〉P?×P = b(u, q) ∀u ∈ H,∀q ∈ PA : P→ H?, 〈v ,A p〉H×H? = b(v , p) ∀v ∈ H, ∀p ∈ P
and we have
〈A Fu, q〉P?×P = 〈u,A q〉H×H? = b(u, q).
In order to make the following discussion simpler, we assume thata(u, v) is symmetric and coercive on H
0 < χ1‖u‖H ≤ a(u, u).
However, Brezzi:1991 the coercivity on the kernel of A F,Ker(A F) is sufficient. We will also assume that ∃χ0 > 0 suchthat
supv∈H
b(v , q)
‖v‖H≥ χ0‖q‖P\Ker(A ) = χ0
[inf
q0∈Ker(A )‖q + q0‖P
].
90 / 215
![Page 186: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/186.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup
Under these hypotheses, and for any f ∈ H? and g ∈ Im(A F)then there exists (u, p) solution of saddle problem: u is unique andp is definite up to an element of Ker(A ).
90 / 215
![Page 187: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/187.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup
Remember that
〈A Fu, q〉P?×P and 〈v ,A p〉H×H? = b(v , p)
Then, we solve [M AA F
] [up
]=
[fg
]
90 / 215
![Page 188: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/188.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup and Mixed finite-element method
Let now Hh → H and Ph → P be two finite dimensional subspacesof H and P. As for the problem (2), we can introduce theoperators Ah : Ph → H?h and Mh;Hh → H?h. We also assume that
Ker(Ah) ⊂ Ker(A )
supvh∈Hh
b(vh, qh)
‖vh‖Hh
≥ χn‖qh‖Ph\Ker(Ah)
χn ≥ χ0 > 0.
91 / 215
![Page 189: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/189.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup and Mixed finite-element method
Under the hypotheses of inf-sup, we have that∃(uh, ph) ∈ Hh ×Ph solution of
a(uh, vh) + b(vh, ph) = 〈f , vh〉H?h ,Hh∀vh ∈ Hh
b(uh, qh) = 〈g , qh〉P?h ,Ph∀qh ∈ Ph.
and
‖u − uh‖H + ‖p − ph‖P\Ker(A) ≤
κ
(inf
vh∈Hh
‖u − vh‖H + infqh∈Ph
‖p − qh‖P),
where κ = κ(‖a‖, ‖b‖, χ0, χ1) is independent of h.
91 / 215
![Page 190: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/190.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup and Mixed finite-element method
Let φii=1,...,m be a basis for Hh and ψjj=1,...,n be a basis forPh. Then, the matrices M and N are the Grammian matrices ofthe operators M and A . In order to use the latter theory, we needto weaken the hypothesis, made in the introduction, that A be fullrank. In this case, we have that
I s elliptic singular values will be zero;
I however, the G-K bidiagonalization method will still work and,if Aq1 6= 0, it will compute a matrix B of rank less than orequal to n − s.
91 / 215
![Page 191: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/191.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
inf-sup and Mixed finite-element method
On the basis of the latter observations, the error ‖e(k)‖M can bestill computed. Finally, we point out that for h ↓ 0 the ellipticsingular values of all A ∈ IRmh×nh will be bounded with upper andlower bounds independent of h, i.e.
χ0 ≤ σnh ≤ · · · ≤ σ1 ≤ ‖a‖.
91 / 215
![Page 192: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/192.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized SVD
Given q ∈M and v ∈ N, the critical points for the functional
vTAq
‖q‖N ‖v‖M
are the “elliptic singular values and singular vectors’’ of A.
92 / 215
![Page 193: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/193.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized SVD
Given q ∈M and v ∈ N, the critical points for the functional
vTAq
‖q‖N ‖v‖M
are the “elliptic singular values and singular vectors’’ of A.The saddle-point conditions are
Aqi = σiMvi vTi Mvj = δijATvi = σiNqi qT
i Nqj = δij
σ1 ≥ σ2 ≥ · · · ≥ σn > 0
92 / 215
![Page 194: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/194.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized SVD
Given q ∈M and v ∈ N, the critical points for the functional
vTAq
‖q‖N ‖v‖M
are the “elliptic singular values and singular vectors’’ of A.The saddle-point conditions are
Aqi = σiMvi vTi Mvj = δijATvi = σiNqi qT
i Nqj = δij
σ1 ≥ σ2 ≥ · · · ≥ σn > 0
The elliptic singular values are the standard singular values of
A = M−1/2AN−1/2. The elliptic singular vectors qi and vi , i = 1, . . . , n
are the transformation by M−1/2 and N−1/2 respectively of the left and
right standard singular vector of A.
92 / 215
![Page 195: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/195.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Quadratic programming
The general problem
minAT w=r
1
2wTWw − gTw
where the matrix W is positive semidefinite andker(W) ∩ ker(AT ) = 0 can be reformulated by choosing
M = W + νAN−1AT
u = w −M−1gb = r − ATM−1g.
as a projection problem
minAT u=b
‖u‖2M
If W is non singular then we can choose ν = 0.
93 / 215
![Page 196: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/196.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Augmented system
The augmented system that gives the optimality conditions for theprojection problem:[
M AAT 0
] [up
]=
[0b
].
94 / 215
![Page 197: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/197.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Lecture on Golub-Kahan
95 / 215
![Page 198: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/198.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Elliptic SVD
Given q ∈M and v ∈ N, the critical points for the functional
vTAq
‖q‖N ‖v‖M
are the “ELLIPTIC singular values and singular vectors’’ of A.
96 / 215
![Page 199: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/199.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Elliptic SVD
Given q ∈M and v ∈ N, the critical points for the functional
vTAq
‖q‖N ‖v‖M
are the “ELLIPTIC singular values and singular vectors’’ of A.The saddle-point conditions are
Aqi = σiMvi vTi Mvj = δijATvi = σiNqi qT
i Nqj = δij
σ1 ≥ σ2 ≥ · · · ≥ σn > 0
96 / 215
![Page 200: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/200.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Elliptic SVD
Given q ∈M and v ∈ N, the critical points for the functional
vTAq
‖q‖N ‖v‖M
are the “ELLIPTIC singular values and singular vectors’’ of A.The saddle-point conditions are
Aqi = σiMvi vTi Mvj = δijATvi = σiNqi qT
i Nqj = δij
σ1 ≥ σ2 ≥ · · · ≥ σn > 0
The elliptic singular values are the standard singular values of
A = M−1/2AN−1/2. The elliptic singular vectors qi and vi , i = 1, . . . , n
are the transformation by M−1/2 and N−1/2 respectively of the left and
right standard singular vector of A.
96 / 215
![Page 201: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/201.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Quadratic programming
The general problem
minAT w=r
1
2wTWw − gTw
where the matrix W is positive semidefinite andker(W) ∩ ker(AT ) = 0 can be reformulated by choosing
M = W + νAN−1AT
u = w −M−1gb = r − ATM−1g.
as a projection problem
minAT u=b
‖u‖2M
If W is non singular then we can choose ν = 0.
97 / 215
![Page 202: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/202.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Augmented system
The augmented system that gives the optimality conditions for theprojection problem:[
M AAT 0
] [up
]=
[0b
].
98 / 215
![Page 203: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/203.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized Golub-Kahan bidiagonalization
In Golub Kahan (1965), Paige Saunders (1982), several algorithmsfor the bidiagonalization of a m × n matrix are presented. All ofthem can be theoretically applied to A and their generalization toA is straightforward as shown by Bembow (1999). Here, we wantspecifically to analyse one of the variants known as the”Craig”-variant (see Paige Saunders (1982), Saunders(1995,1997)).
99 / 215
![Page 204: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/204.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized Golub-Kahan bidiagonalization
AQ = MV
[B0
]VTMV = Im
AT V = NQ[BT ; 0
]QTNQ = In
where
B =
α1 0 0 · · · 0
β2 α2 0. . . 0
.... . .
. . .. . .
. . .
0 · · · βn−1 αn−1 0
0 · · · 0 βn αn
0 · · · 0 0 βn+1
.
99 / 215
![Page 205: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/205.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized Golub-Kahan bidiagonalization
AQ = MV
[B0
]VTMV = Im
ATV = NQ[BT ; 0
]QTNQ = In
where
B =
α1 β1 0 · · · 0
0 α2 β2. . . 0
.... . .
. . .. . .
. . .
0 · · · 0 αn−1 βn−1
0 · · · 0 0 αn
.
99 / 215
![Page 206: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/206.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Algorithm
The augmented system that gives the optimality conditions forminAT u=b ‖u‖2
M [M AAT 0
] [up
]=
[0b
]can be transformed by the change of variables
u = Vzp = Qy
100 / 215
![Page 207: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/207.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Algorithm
In 0 B0 Im−n 0
BT 0 0
z1
z2
y
=
00
QTb
.
100 / 215
![Page 208: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/208.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Algorithm
[In B
BT 0
] [z1
y
]=
[0
QTb
].
100 / 215
![Page 209: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/209.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Algorithm
[In B
BT 0
] [z1
y
]=
[0
QTb
].
QTb = e1‖b‖N
the value of z1 will correspond to the first column of the inverse ofB multiplied by ‖b‖N.
100 / 215
![Page 210: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/210.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Algorithm
Thus, we can compute the first column of B and of V:α1Mv1 = Aq1, such as
w = M−1Aq1
α1 = wTMw = wAq1
v1 = w/√α1.
101 / 215
![Page 211: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/211.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Algorithm
Thus, we can compute the first column of B and of V:α1Mv1 = Aq1, such as
w = M−1Aq1
α1 = wTMw = wAq1
v1 = w/√α1.
Finally, knowing q1 and v1 we can start the recursive relations
gi+1 = N−1(ATvi − αiNqi
)βi+1 = gTNg
qi+1 = g√βi+1
w = M−1 (Aqi+1 − βi+1Mvi )αi+1 = wTMwvi+1 = w/
√αi+1.
101 / 215
![Page 212: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/212.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
u
Thus, the value of u can be approximated when we have computedthe first k columns of V by
u(k) = Vkzk =k∑
j=1
ζjvj .
102 / 215
![Page 213: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/213.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
u
Thus, the value of u can be approximated when we have computedthe first k columns of V by
u(k) = Vkzk =k∑
j=1
ζjvj .
The entries ζj of zk can be easily computed recursively startingwith
ζ1 = −‖b‖N
α1
as
ζi+1 = − βiαi+1
ζi i = 1, . . . , n
102 / 215
![Page 214: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/214.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
p
Approximating p = Qy by p(k) = Qkyk =∑k
j=1 ψjqj , we have that
yk = −B−1k zk .
103 / 215
![Page 215: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/215.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
p
Approximating p = Qy by p(k) = Qkyk =∑k
j=1 ψjqj , we have that
yk = −B−1k zk .
Following an observation made by Paige and Saunders, we caneasily transform the previous relation into a recursive one whereonly one extra vector is required.
103 / 215
![Page 216: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/216.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
p
Approximating p = Qy by p(k) = Qkyk =∑k
j=1 ψjqj , we have that
yk = −B−1k zk .
From p(k) = −QkB−1k zk = −
(B−Tk QT
k
)Tzk and Dk = B−Tk QT
k
di =qi − βidi−1
αii = 1, . . . , n
(d0 = 0
)where dj are the columns of D.Starting with p(1) = −ζ1d1 and u(1) = ζ1v1
u(i+1) = u(i) + ζi+1vi+1
p(i+1) = p(i) − ζi+1di+1
i = 1, . . . , n
103 / 215
![Page 217: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/217.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Stopping criteria
‖u− u(k)‖2M = ‖e(k)‖2
M =n∑
j=k+1
ζ2j =
∣∣∣∣∣∣z− [ zk0
] ∣∣∣∣∣∣22.
104 / 215
![Page 218: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/218.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Stopping criteria
‖u− u(k)‖2M = ‖e(k)‖2
M =n∑
j=k+1
ζ2j =
∣∣∣∣∣∣z− [ zk0
] ∣∣∣∣∣∣22.
‖ATu(k) − b‖N−1 = |βk+1 ζk | ≤ σ1|ζk | = ‖A‖2|ζk |.
104 / 215
![Page 219: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/219.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Stopping criteria
‖u− u(k)‖2M = ‖e(k)‖2
M =n∑
j=k+1
ζ2j =
∣∣∣∣∣∣z− [ zk0
] ∣∣∣∣∣∣22.
‖ATu(k) − b‖N−1 = |βk+1 ζk | ≤ σ1|ζk | = ‖A‖2|ζk |.
‖p− p(k)‖N =∣∣∣∣∣∣QB−1
(z−
[zk0
]) ∣∣∣∣∣∣N≤ ‖e
(k)‖M
σn.
104 / 215
![Page 220: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/220.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Error bound
Lower bound We can estimate ‖e(k)‖2M by the lower bound
ξ2k,d =
k+d+1∑j=k+1
ζ2j < ‖e(k)‖2
M.
Given a threshold τ < 1 and an integer d , we canstop the iterations when
ξ2k,d ≤ τ
k+d+1∑j=1
ζ2j < τ‖u‖2
M.
Upper bound Despite being very inexpensive, the previousestimator is still a lower bound of the error. We canuse an approach inspired by the Gauss-Radauquadrature algorithm and similar to the onedescribed in Golub-Meurant (2010).
105 / 215
![Page 221: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/221.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Error bound
Lower bound We can estimate ‖e(k)‖2M by the lower bound
ξ2k,d =
k+d+1∑j=k+1
ζ2j < ‖e(k)‖2
M.
Given a threshold τ < 1 and an integer d , we canstop the iterations when
ξ2k,d ≤ τ
k+d+1∑j=1
ζ2j < τ‖u‖2
M.
Upper bound Despite being very inexpensive, the previousestimator is still a lower bound of the error. We canuse an approach inspired by the Gauss-Radauquadrature algorithm and similar to the onedescribed in Golub-Meurant (2010).
105 / 215
![Page 222: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/222.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Error bound
TheoremUnder previous hypotheses, and denoting by u∗ one of the iteratesof Algorithm Craig for which ‖e(k)‖M < τ , we have
‖u − u∗‖H + ‖p − p∗‖P\Ker(A ) ≤
κ
(inf
vh∈Hh
‖u − vh‖H + infqh∈Ph
‖p − qh‖P + τ
),(3)
where u∗ =∑nh
i=1 φiu∗i ∈ Hh, p∗ =
∑nhj=1 φip
∗j ∈ Ph and κ a
constant independent of h.
106 / 215
![Page 223: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/223.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Lecture on SQD
107 / 215
![Page 224: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/224.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Symmetric Quasi-Definite Systems
[M AAT −N
] [xy
]=
[fg
]where M = MT 0, N = NT 0.
I Interior-point methods for LP, QP, NLP, SOCP, SDP, . . .
I Regularized/stabilized PDE problems
I Regularized least squares
I How to best take advantage of the structure?
108 / 215
![Page 225: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/225.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Main Property
Theorem (Vanderbei, 1995)If K is SQD, it is strongly factorizable, i.e., for any permutationmatrix P, there exists a unit lower triangular L and a diagonal Dsuch that PTKP = LDLT .
I Cholesky-factorizable
I Used to speed up factorization in regularized least-squares(Saunders) and interior-point methods (Friedlander and O.)
I Stability analysis by Gill, Saunders, Shinnerl (1996).
109 / 215
![Page 226: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/226.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Centered preconditioning
[M−
12
N−12
] [M AAT −N
] [M−
12
N−12
] [xy
]=
[M−
12 f
N−12 g
]
which is equivalent to
C︷ ︸︸ ︷[Im M−
12 AN−
12
N−12 ATM−
12 −In
][xy
]=
[M−
12 f
N−12 g
]
Theorem (Saunders (1995))
Suppose A = M−12 AN−
12 has rank p ≤ m with nonzero singular
values σ1, . . . , σp. The eigenvalues of C are +1, −1 and±√1 + σk , k = 1, . . . , p.
110 / 215
![Page 227: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/227.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Centered preconditioning
[M−
12
N−12
] [M AAT −N
] [M−
12
N−12
] [xy
]=
[M−
12 f
N−12 g
]
which is equivalent to
C︷ ︸︸ ︷[Im M−
12 AN−
12
N−12 ATM−
12 −In
][xy
]=
[M−
12 f
N−12 g
]
Theorem (Saunders (1995))
Suppose A = M−12 AN−
12 has rank p ≤ m with nonzero singular
values σ1, . . . , σp. The eigenvalues of C are +1, −1 and±√1 + σk , k = 1, . . . , p.
110 / 215
![Page 228: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/228.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Symmetric spectrum and Iterative methods
A symmetric matrix with a symmetric spectrum can be transformpreserving the symmetry of the spectrum in a SQD one.Moreover, Fischer (Theorem 6.9.9 in “Polynomial based iterationmethods for symmetric linear systems”) Freund (1983), FreundGolub Nachtigal (1992), and Ramage Silvester Wathen (1995) givedifferent poofs that MINRES and CG perform redundant iterations.
111 / 215
![Page 229: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/229.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Iterative Methods I
Facts: SQD systems are symmetric, non-singular, square andindefinite.
I MINRES
I SYMMLQ
I (F)GMRES??
I QMRS????
Fact: . . . none exploits the SQD structure and they are doingredundant iterations
112 / 215
![Page 230: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/230.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Iterative Methods I
Facts: SQD systems are symmetric, non-singular, square andindefinite.
I MINRES
I SYMMLQ
I (F)GMRES??
I QMRS????
Fact: . . . none exploits the SQD structure and they are doingredundant iterations
112 / 215
![Page 231: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/231.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Iterative Methods I
Facts: SQD systems are symmetric, non-singular, square andindefinite.
I MINRES
I SYMMLQ
I (F)GMRES??
I QMRS????
Fact: . . . none exploits the SQD structure and they are doingredundant iterations
112 / 215
![Page 232: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/232.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Related Problems: an example
[M AAT −N
] [xy
]=
[b0
]
113 / 215
![Page 233: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/233.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Related Problems: an example
[M AAT −N
] [xy
]=
[b0
]are the optimality conditions of
miny∈IRm
12
∥∥∥∥[AI
]y −
[b0
]∥∥∥∥2
E−1+
≡ miny∈IRm
12
∥∥∥∥∥[
M−12 0
0 N12
]([AI
]y −
[b0
])∥∥∥∥∥2
2
113 / 215
![Page 234: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/234.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Related Problems: an example
[M AAT −N
] [xy
]=
[b0
]are the optimality conditions of
miny∈IRm
12
∥∥∥∥[AI
]y −
[b0
]∥∥∥∥2
E−1+
≡ miny∈IRm
12
∥∥∥∥∥[
M−12 0
0 N12
]([AI
]y −
[b0
])∥∥∥∥∥2
2
or of
minimizex,y
12 (‖x‖2
M + ‖y‖2N) subject to Mx + Ay = b.
113 / 215
![Page 235: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/235.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some properties of SQD matrices
Let us denote the Cholesky factors of M and N by R and U (uppertriangular matrices).
H =
[M
N
]=
[RTR
UTU
]= RT R
We observe that
C =
[M AAT −N
]=
[RT 00 UT
] [Im A
AT −In
] [R 00 U
]= RT CR,
114 / 215
![Page 236: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/236.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some properties of SQD matrices
Let us denote the Cholesky factors of M and N by R and U (uppertriangular matrices).
H =
[M
N
]=
[RTR
UTU
]= RT R
We observe that
C =
[M AAT −N
]=
[RT 00 UT
] [Im A
AT −In
] [R 00 U
]= RT CR,
H−1C = R−1CR
114 / 215
![Page 237: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/237.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some properties of SQD matrices
By direct computation it is easy to prove that
C2 =
[Im + AAT
In + AT A
]=
[D1
D2
]= D.
115 / 215
![Page 238: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/238.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some properties of SQD matrices
By direct computation it is easy to prove that
C2 =
[Im + AAT
In + AT A
]=
[D1
D2
]= D.
C−1 = D−1C = CD−1;
CD = C3 = DC;
CH−1C = RT DR = D =
[M + AN−1AT
N + ATM−1A
].
115 / 215
![Page 239: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/239.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some properties of SQD matricesBy direct computation it is easy to prove that
C2 =
[Im + AAT
In + AT A
]=
[D1
D2
]= D.
C−1 = D−1C = CD−1;
CD = C3 = DC;
CH−1C = RT DR = D =
[M + AN−1AT
N + ATM−1A
].
(H−1C
)2= R−1C2R = R−1DR = H−1D,(
H−1C)3
= R−1C3R = H−1CH−1D = H−1DH−1C
C−1 = D−1CH−1 = H−1CD−1.
115 / 215
![Page 240: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/240.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Some properties of SQD matrices
D and C commute.Both matrices can be simultaneously diagonalized by thegeneralized eigenvalues of
Cz = λjHz,
where the λj , j = 1, . . . , p = rank(A) are the same eigenvalues of
C
116 / 215
![Page 241: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/241.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Krylov subspaces
Hereafter we will denote by
Ki (C, z) = Range
z, Cz, C2z, . . . , Ci−1z, Ciz
the Krylov subspace generated by C and a vector z. We point outthat Ki (C, z) are also the Krylov subspaces used to define theLanczos algorithm applied to C symmetrically preconditioned by R.
Ki (H−1C,w) = R−1Ki (C, z), where w = Rz.
117 / 215
![Page 242: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/242.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Krylov subspaces
C2k = Dk
C2k+1 = CDk = Dk C
.
Therefore,
Kk(C, z) = Kbk/2c(D, z) + Kdk/2e−1(D, Cz)
= Kbk/2c(D, z) + CKdk/2e−1(D, z).
118 / 215
![Page 243: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/243.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Krylov subspaces
Finally, denoting by D1 and D2 the diagonal blocks of D, i.e. wehave:
Ki (D,
[z1
z2
]) =
[Ki (D1, z1)
0
]⊕[
0
Ki (D2, z2)
]and
CKi (D,
[z1
z2
]) =
[Ki (D1, z1)
ATKi (D1, z1)
]⊕[
AKi (D2, z2)
−Ki (D2, z2)
]
=
[Ki (D1, z1)
Ki (D2, ATz1)
]⊕[
Ki (D1, Az2)
−Ki (D2, z2)
].
119 / 215
![Page 244: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/244.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized Golub-Kahan bidiagonalization
TWO VARIANTS
120 / 215
![Page 245: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/245.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized Golub-Kahan bidiagonalization
AQ = MV
[B0
]VTMV = Im
AT V = NQ[BT ; 0
]QTNQ = In
where
B =
α1 0 0 · · · 0
β2 α2 0. . . 0
.... . .
. . .. . .
. . .
0 · · · βn−1 αn−1 0
0 · · · 0 βn αn
0 · · · 0 0 βn+1
.
120 / 215
![Page 246: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/246.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized Golub-Kahan bidiagonalization
AQ = MV
[B0
]VTMV = Im
ATV = NQ[BT ; 0
]QTNQ = In
where
B =
α1 β1 0 · · · 0
0 α2 β2. . . 0
.... . .
. . .. . .
. . .
0 · · · 0 αn−1 βn−1
0 · · · 0 0 αn
.
120 / 215
![Page 247: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/247.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized Least Squares
Normal equations: (ATM−1A + N)y = ATM−1b.
At k-th iteration, seek y ≈ yk := Vk yk :
(BTk Bk + I)yk = BT
k β1e1
i.e.:
miny∈IRk
12
∥∥∥∥[Bk
I
]y −
[β1e1
0
]∥∥∥∥2
2
or: [I Bk
BTk −I
] [xkyk
]=
[β1e1
0
].
121 / 215
![Page 248: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/248.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized Least Squares
Normal equations: (ATM−1A + N)y = ATM−1b.
At k-th iteration, seek y ≈ yk := Vk yk :
(BTk Bk + I)yk = BT
k β1e1
i.e.:
miny∈IRk
12
∥∥∥∥[Bk
I
]y −
[β1e1
0
]∥∥∥∥2
2
or: [I Bk
BTk −I
] [xkyk
]=
[β1e1
0
].
121 / 215
![Page 249: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/249.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized Least Squares
Normal equations: (ATM−1A + N)y = ATM−1b.
At k-th iteration, seek y ≈ yk := Vk yk :
(BTk Bk + I)yk = BT
k β1e1
i.e.:
miny∈IRk
12
∥∥∥∥[Bk
I
]y −
[β1e1
0
]∥∥∥∥2
2
or: [I Bk
BTk −I
] [xkyk
]=
[β1e1
0
].
121 / 215
![Page 250: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/250.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized Least Squares
Normal equations: (ATM−1A + N)y = ATM−1b.
At k-th iteration, seek y ≈ yk := Vk yk :
(BTk Bk + I)yk = BT
k β1e1
i.e.:
miny∈IRk
12
∥∥∥∥[Bk
I
]y −
[β1e1
0
]∥∥∥∥2
2
or: [I Bk
BTk −I
] [xkyk
]=
[β1e1
0
].
121 / 215
![Page 251: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/251.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized LSQRSolve
miny∈IRk
12
∥∥∥∥[Bk
I
]y −
[β1e1
0
]∥∥∥∥2
2
by specialized Givens Rotations (Eliminate I first and Rk will beupper bidiagonal)
miny∈IRk
12
∥∥∥∥[Rk
0
]y −
[φk0
]∥∥∥∥2
2
.
As in Paige-Saunders ’82 we can build recursive expressions of yk
yk+1 = yk + dkφk
(Dk = Vk R−1
k
)and we have that
||y||2N+AT M−1A =m∑j=1
φ2j and ||y − yk ||2N+AT M−1A =
m∑j=k+1
φ2j
122 / 215
![Page 252: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/252.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized LSQRSolve
miny∈IRk
12
∥∥∥∥[Bk
I
]y −
[β1e1
0
]∥∥∥∥2
2
by specialized Givens Rotations (Eliminate I first and Rk will beupper bidiagonal)
miny∈IRk
12
∥∥∥∥[Rk
0
]y −
[φk0
]∥∥∥∥2
2
.
As in Paige-Saunders ’82 we can build recursive expressions of yk
yk+1 = yk + dkφk
(Dk = Vk R−1
k
)and we have that
||y||2N+AT M−1A =m∑j=1
φ2j and ||y − yk ||2N+AT M−1A =
m∑j=k+1
φ2j
122 / 215
![Page 253: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/253.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized LSQRSolve
miny∈IRk
12
∥∥∥∥[Bk
I
]y −
[β1e1
0
]∥∥∥∥2
2
by specialized Givens Rotations (Eliminate I first and Rk will beupper bidiagonal)
miny∈IRk
12
∥∥∥∥[Rk
0
]y −
[φk0
]∥∥∥∥2
2
.
As in Paige-Saunders ’82 we can build recursive expressions of yk
yk+1 = yk + dkφk
(Dk = Vk R−1
k
)and we have that
||y||2N+AT M−1A =m∑j=1
φ2j and ||y − yk ||2N+AT M−1A =
m∑j=k+1
φ2j
122 / 215
![Page 254: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/254.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Error bound
Lower bound We can estimate ||y − yk ||2N+AT M−1Aby the lower
bound
ξ2k,d =
k+d+1∑j=k+1
φ2j < ||y − yk ||2N+AT M−1A.
and ||y||2N+AT M−1A
by the lower bound∑k
j=1 φ2j .
Given a threshold τ < 1 and an integer d , we canstop the iterations when
ξ2k,d ≤ τ
k+d+1∑j=1
φ2j < τ
k∑j=1
φ2j < τ ||y||2N+AT M−1A.
Upper bound Despite being very inexpensive, the previousestimator is still a lower bound of the error. We canuse an approach inspired by the Gauss-Radauquadrature algorithm and similar to the onedescribed in Golub-Meurant (2010).
123 / 215
![Page 255: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/255.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Error bound
Lower bound We can estimate ||y − yk ||2N+AT M−1Aby the lower
bound
ξ2k,d =
k+d+1∑j=k+1
φ2j < ||y − yk ||2N+AT M−1A.
and ||y||2N+AT M−1A
by the lower bound∑k
j=1 φ2j .
Given a threshold τ < 1 and an integer d , we canstop the iterations when
ξ2k,d ≤ τ
k+d+1∑j=1
φ2j < τ
k∑j=1
φ2j < τ ||y||2N+AT M−1A.
Upper bound Despite being very inexpensive, the previousestimator is still a lower bound of the error. We canuse an approach inspired by the Gauss-Radauquadrature algorithm and similar to the onedescribed in Golub-Meurant (2010).
123 / 215
![Page 256: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/256.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized CRAIG
miny,x
12 (‖y‖2
N + ‖x‖2M) s.t. Ay + Mx = b.
At step k of GK bidiagonalization, we seek
x ≈ xk := Uk xk , and y ≈ yk := Vk yk .
miny,x
12 (‖y‖2 + ‖x‖2) s.t. Bk yk + xk = β1e1
or:
miny∈IRk
12
∥∥∥∥[Bk
I
]y −
[β1e1
0
]∥∥∥∥2
2
.
124 / 215
![Page 257: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/257.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized CRAIG
miny,x
12 (‖y‖2
N + ‖x‖2M) s.t. Ay + Mx = b.
At step k of GK bidiagonalization, we seek
x ≈ xk := Uk xk , and y ≈ yk := Vk yk .
miny,x
12 (‖y‖2 + ‖x‖2) s.t. Bk yk + xk = β1e1
or:
miny∈IRk
12
∥∥∥∥[Bk
I
]y −
[β1e1
0
]∥∥∥∥2
2
.
124 / 215
![Page 258: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/258.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized CRAIG
miny,x
12 (‖y‖2
N + ‖x‖2M) s.t. Ay + Mx = b.
At step k of GK bidiagonalization, we seek
x ≈ xk := Uk xk , and y ≈ yk := Vk yk .
miny,x
12 (‖y‖2 + ‖x‖2) s.t. Bk yk + xk = β1e1
or:
miny∈IRk
12
∥∥∥∥[Bk
I
]y −
[β1e1
0
]∥∥∥∥2
2
.
124 / 215
![Page 259: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/259.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized CRAIG
miny,x
12 (‖y‖2
N + ‖x‖2M) s.t. Ay + Mx = b.
At step k of GK bidiagonalization, we seek
x ≈ xk := Uk xk , and y ≈ yk := Vk yk .
miny,x
12 (‖y‖2 + ‖x‖2) s.t. Bk yk + xk = β1e1
or:
miny∈IRk
12
∥∥∥∥[Bk
I
]y −
[β1e1
0
]∥∥∥∥2
2
.
124 / 215
![Page 260: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/260.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized CRAIG
By contrast with generalized LSQR, we solve the SQD subsystem[Ik Bk
BTk −Ik
] [xkyk
]=
[β1e1
0
]
125 / 215
![Page 261: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/261.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized CRAIG
By contrast with generalized LSQR, we solve the SQD subsystem[Ik Bk
BTk −Ik
] [xkyk
]=
[β1e1
0
]Following Saunders (1995) and Paige (1974), we compute an LQ
factorization to the k-by-2k matrix[Bk Ik
]by applying 2k − 1
Givens rotations that zero out the identity block.
125 / 215
![Page 262: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/262.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized CRAIG
By contrast with generalized LSQR, we solve the SQD subsystem[Ik Bk
BTk −Ik
] [xkyk
]=
[β1e1
0
][Bk Ik
]QT
k =[Bk 0
]QT
k Qk = I
where
Bk :=
α1
β2 α2
. . .. . .
βk αk
.
125 / 215
![Page 263: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/263.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized CRAIG
β1e1 = Bk yk + xk =[Bk Ik
] [ykxk
]=
[Bk 0
]Qk
[ykxk
]=[Bk 0
] [zk0
]= Bk zk ,
for some zk ∈ IRk : zk = (ζ1, . . . , ζk)
126 / 215
![Page 264: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/264.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized CRAIG
β1e1 = Bk yk + xk =[Bk Ik
] [ykxk
]=
[Bk 0
]Qk
[ykxk
]=[Bk 0
] [zk0
]= Bk zk ,
for some zk ∈ IRk : zk = (ζ1, . . . , ζk)
ζ1 = β1/α1, ζi+1 = −βi+1ζi/αi+1, (i = 1, . . . , k − 1).
126 / 215
![Page 265: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/265.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized CRAIG
Solving for xk directly, and bypassing xk , may now be done. Bydefinition,
xk = Uk xk = UkB−Tk zk .
Since B−Tk is upper bidiagonal, all components of B−Tk zk arelikely to change at every iteration. Fortunately, upon definingDk := UkB−Tk , and denoting di the i-th column of Dk , we areable to use a recursion formula for xk provided that di may befound easily. Slightly rearranging, we have
BkDTk = UT
k
and therefore it is easy to identify each di—i.e., each row ofDT
k —recursively.
127 / 215
![Page 266: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/266.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized CRAIG
Solving for xk directly, and bypassing xk , may now be done. Bydefinition,
xk = Uk xk = UkB−Tk zk .
d1 := u1/α1, di+1 := (ui+1− βi+1di )/αi+1, (i = 1, . . . , k − 1).
This yields the update
xk+1 = xk + ζk+1dk+1
for xk+1.
127 / 215
![Page 267: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/267.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized CRAIG: errors bound
Let Bk be defined as above and Dk := UkB−Tk . For k = 1, . . . , n,we have
DTk (AN−1AT + M)Dk = Ik .
In particular,
xk =k∑
j=1
ζjdj
and we have the estimates
‖xk‖2AN−1AT+M =
k∑i=1
ζ2i , (4a)
‖x∗ − xk‖2AN−1AT +M =
n∑i=k+1
ζ2i , (4b)
128 / 215
![Page 268: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/268.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized CRAIG: errors bound
As for generalized LSQR, we can estimate the error using thewindowing technique and we can give a lower bound of the error by
ξ2k,d =
k+d+1∑j=k+1
ζ2i ≤ ‖x∗ − xk‖2
AN−1AT +M
and we can estimate ‖x∗‖AN−1AT +M by the lower bound∑k
j=1 ζ2j .
129 / 215
![Page 269: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/269.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Generalized CRAIG: errors bound
As for GLSQR. If we know a lower bound of singular values we canuse an approach inspired by the Gauss-Radau quadrature algorithmand similar to the one described in Golub-Meurant (2010).
129 / 215
![Page 270: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/270.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Other variants:
Generalized LSMR
minimizey∈IRm
12‖N−
12 (ATM−1b− (ATM−1A + N)y))‖2.
Generalized Craig-MR
Error bounds similar to the ones given above exist for the MRvariants
130 / 215
![Page 271: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/271.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Other variants:
Generalized LSMR
Generalized Craig-MR
Error bounds similar to the ones given above exist for the MRvariants
130 / 215
![Page 272: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/272.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments
We will focus on optimization problems:
minimizex∈IRn
gTx + 12xTHx subject to Cx = d, x ≥ 0,
where g ∈ IRn and H = HT ∈ IRn×n is positive semi-definite, andresult in linear systems with coefficient matrix[
H + X−1Z + ρI CT
C −δI
]where ρ > 0 and δ > 0 are regularization parameters.
131 / 215
![Page 273: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/273.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments MINRES
This is a blow-up of some iterations
132 / 215
![Page 274: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/274.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments GLSQR
0 20 40 60 80 100 120101
102
103
104
105 Regularized Least-Squares Objective
G-LSQRG-LSMR
0 20 40 60 80 100 12010-7
10-5
10-3
10-1
101
103
105
107
109 Residual of Normal Equations
Figure : Problem DUAL1 (255, 171).
133 / 215
![Page 275: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/275.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments GLSQR
0 50 100 150 200101
102
103 Regularized Least-Squares Objective
G-LSQRG-LSMR
0 50 100 150 20010-8
10-7
10-6
10-5
10-4
10-3
10-2
10-1
100
101
102
103
104 Residual of Normal Equations
Figure : Problem MOSARQP1 (5700, 3200).
134 / 215
![Page 276: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/276.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
How to choose d?
problem m ndual1 255 171stcqp1 12291 10246
qpcboei1 1355 980
135 / 215
![Page 277: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/277.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments GCraig
d = 5, 15
0 50 100 15010-13
10-12
10-11
10-10
10-9
10-8
10-7
10-6
10-5
10-4
10-3
10-2
10-1
100
101 CRAIG: Direct Errors and Estimates
d=5d=15Actual
0 10 20 30 40 50 60 7010-13
10-12
10-11
10-10
10-9
10-8
10-7
10-6
10-5
10-4
10-3
10-2
10-1
100
101
102
103 CRAIG: Direct Errors and Estimates
d=5d=15Actual
Figure : Problem dual1
136 / 215
![Page 278: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/278.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
CG?
137 / 215
![Page 279: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/279.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments CG
d = 5, 15
0 50 100 150 200 250-1010-109-108-107-106-105-104-103-102-101-1000100101102103104105106107108109
10101011101210131014 D-Lanczos
d=5d=15Actual
0 20 40 60 80 100 120 140-106
-105
-104
-103
-102
-101
-1000
100
101
102
103
104
105
106
107
108 D-Lanczos
d=5d=15Actual
Figure : Problem DUAL1 and MOSARQP1 (5700, 3200).
138 / 215
![Page 280: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/280.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments CG
d = 5, 15
0 10 20 30 40 50 60-106
-105
-104
-103
-102
-101
-1000
100
101
102
103
104
105
106
107 D-Lanczos
d=5d=15Actual
0 10 20 30 40 50 60-103
-102
-101
-100
0
100
101
102
103
104 D-Lanczos
d=5d=15Actual
Figure : Problem Stokes (IFISS 3.1): colliding and cavity
139 / 215
![Page 281: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/281.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Conclusions
I Preconditioning −→ Norms i.e. different topologies!!
I Nice relation between the algebraic error and theapproximation error
I A. and Orban ”Iterative methods for symmetric quasi definitesystems” Cahier du GERAD G-2013-32
140 / 215
![Page 282: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/282.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Conclusions
I Preconditioning −→ Norms i.e. different topologies!!
I Nice relation between the algebraic error and theapproximation error
I A. and Orban ”Iterative methods for symmetric quasi definitesystems” Cahier du GERAD G-2013-32
140 / 215
![Page 283: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/283.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Conclusions
I Preconditioning −→ Norms i.e. different topologies!!
I Nice relation between the algebraic error and theapproximation error
I A. and Orban ”Iterative methods for symmetric quasi definitesystems” Cahier du GERAD G-2013-32
140 / 215
![Page 284: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/284.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Lecture on linear regression and LS
141 / 215
![Page 285: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/285.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
I QR algorithm
I Sparse least-squares problems
I Rounding error analysis
142 / 215
![Page 286: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/286.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Elementary matrices
I Givens transformation:
G =
1. . .
c s. . .
−s c. . .
1
c2 + s2 = 1
I Householder transformation
H = I− 2yyT
yTy
143 / 215
![Page 287: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/287.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Elementary matrices
I Givens transformation:
G =
1. . .
c s. . .
−s c. . .
1
c2 + s2 = 1
I Householder transformation
H = I− 2yyT
yTy
143 / 215
![Page 288: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/288.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Elementary matrices
I Givens transformation:
G =
1. . .
c s. . .
−s c. . .
1
c2 + s2 = 1
I Householder transformation
H = I− 2yyT
yTy
143 / 215
![Page 289: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/289.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Givens transformation
u ∈ IRn find n − 1, Gi such that
Gn−1 . . .G1u = ||u||2e1
If n = 2 [c s−s c
] [xy
]=√
x2 + y 2
[10
]
c =x√
x2 + y 2s =
y√x2 + y 2
144 / 215
![Page 290: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/290.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Givens transformation
u ∈ IRn find n − 1, Gi such that
Gn−1 . . .G1u = ||u||2e1
If n = 2 [c s−s c
] [xy
]=√
x2 + y 2
[10
]
c =x√
x2 + y 2s =
y√x2 + y 2
144 / 215
![Page 291: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/291.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Givens transformation
u ∈ IRn find n − 1, Gi such that
Gn−1 . . .G1u = ||u||2e1
If n = 2 [c s−s c
] [xy
]=√
x2 + y 2
[10
]
c =x√
x2 + y 2s =
y√x2 + y 2
144 / 215
![Page 292: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/292.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Householder transformation
(I− 2yyT
yTy)u = ±||u||2e1
y = u± ||u||2e1
y = u + sign(u1)||u||2e1
to avoid cancellation
145 / 215
![Page 293: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/293.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Householder transformation
(I− 2yyT
yTy)u = ±||u||2e1
y = u± ||u||2e1
y = u + sign(u1)||u||2e1
to avoid cancellation
145 / 215
![Page 294: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/294.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Householder transformation
(I− 2yyT
yTy)u = ±||u||2e1
y = u± ||u||2e1
y = u + sign(u1)||u||2e1
to avoid cancellation
145 / 215
![Page 295: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/295.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Householder transformation:example
H = Hm . . .H2H1A
HA =
[R0
]HTH = HHT = I
146 / 215
![Page 296: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/296.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Product of Householder transformations
Let H1 and H2 be two Householder matrices
H1 = I− yyT H2 = I−wwT
‖y‖2 =√
2 ‖w‖2 =√
2
H1H2 = (I− yyT )(I−wwT )
= I− yyT −wwT + yyTwwT
= I−[
y w] [ 1 −yTw
0 1
] [yT
wT
]= I− YTYT
147 / 215
![Page 297: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/297.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Product of Householder transformations
Let H1 and H2 be two products of Householder matrices
H1 = I− YT1YT H2 = I−WT2WT
H1H2 = (I− YT1YT )(H2 = I−WT2WT )
= I− YT1YT −WT2WT + YT1YTWT2WT
= I−[
Y W] [ T1 −T1YTWT2
0 T2
] [YT
WT
]= I− Y3T3YT
3
BLAS-3 Operations in applying I− Y3T3YT3 to a matrix.
148 / 215
![Page 298: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/298.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Least Squares
minx||Ax− b||2
|| • ||2 invariant for orthonormal transformation
minx||Ax− b||2 = min
x||H(Ax− b)||2
= minx||[
R0
]x−
[b1
b2
]||2
= minx||[
Rx− b1
−b2
]||2
x = R−1
b1
149 / 215
![Page 299: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/299.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Error Analysis in mixed precision arithmetic
||fl(Hi )−Hi ||F ≤ ε+O(ε2)
Let C ∈ IRm×n. We first compute B = HiC and let B = fl(HiC)
||B− B||F = ||[fl(fl(Hi )C)− fl(Hi )C] + (fl(Hi )C−HiC)||F≤ ||fl(fl(Hi )C)− fl(Hi )C||F + ||(fl(Hi )C−HiC)||F= ||E||F + ||fl(Hi )−Hi ||F ||C||F
||E||F ≤ ε||C||F +O(ε2)
||B− B||F ≤ c1ε||C||F +O(ε2)
150 / 215
![Page 300: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/300.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Error Analysis in mixed precision arithmetic
A1 = A Ai+1 = HiAi i = 1, . . . ,m
Hi produces zeros in positions i + 1 through m of column i of HiAi
The computed quantities will be
A1 = A Ai+1 = fl(Hi Ai ) i = 1, . . . ,m
Hi = fl(Hi ) where Hi (orthonormal) would have produced zeros inpositions i + 1 through m of column i of Hi Ai
||Hi Ai − fl(Hi Ai )||F ≤ c2ε||Ai+1||F +O(ε2)
||Am −Hm . . .H1A||F ≤ c1mε||A||F +O(ε2)
Am =
[R0
]151 / 215
![Page 301: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/301.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Error Analysis in mixed precision arithmetic
Exists an orthonormal matrix Q = Hm . . .H1 and a matrix E suchthat
A + E = Q
[R0
]||E||F ≤ c2m||A||F ε+O(ε2)
The computed solution x of the least-squares problem is the exactsolution of the problem
minx ||(A + E1)x− (b + g)||2 = ||(A + E1)x− (b + g)||2||E1||F ≤ c3m||A||F ε+O(ε2)
||g||2 ≤ c4m||b||2ε+O(ε2)
152 / 215
![Page 302: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/302.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear regression
For any random vector z, we denote by E [z] its mean and byV [z] = E [(z− E [z])(z− E [z])T ] its covariance matrix.The notation z ∼ N
(z ,C
)means that z follows a Gaussian
distribution with mean z and covariance matrix C.
153 / 215
![Page 303: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/303.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear regression
Let A ∈ IRm×n, m ≥ n, with Rank(A) = n. We consider the linearregression model
y = AX + e,
where E [e] = 0 and V [e] = σ2Im. We point out that A defines agiven model and X is an unknown deterministic value.
154 / 215
![Page 304: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/304.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear regression: Gauss-Markov Theorem
The minimum-variance unbiased (MVU) estimator of X is relatedto y by the Gauss-Markov theorem.
155 / 215
![Page 305: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/305.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear regression: Gauss-Markov TheoremFor the linear model the minimum-variance unbiased estimator of X
is given byx∗ = (ATA)−1ATy.
V [x∗] satisfies V [x∗] = σ2(ATA)−1. If in addition,e ∼ N
(0, σ2Im
), m > n, and if we set
s2 =1
m − n||r||22,
where r = y − Ax∗, we have for our estimator of X
x∗ ∼ N(X, σ2(ATA)−1
),
and for s2, our estimator for σ2,
s2 ∼ σ2
m − nχ2(m − n).
155 / 215
![Page 306: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/306.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear regression: Gauss-Markov Theorem
Moreover, the predicted value y = Ax∗ and the residual r areindependently distributed as
y ∼ N(AX, σ2A(ATA)−1AT
)and
r ∼ N(0, σ2(I− A(ATA)−1AT )
).
155 / 215
![Page 307: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/307.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear regression
Let δy be a stochastic variable such that
δy ∼ N(0, τ2A(ATA)−1AT
).
Under the Hypotheses of Gauss-Markov and assuming that y andδy are independently distributed, we have
y + δy ∼ N(AX, (τ2 + σ2)A(ATA)−1AT
).
Moreover, we have that
||δy||22 ∼ τ2χ2(n).
156 / 215
![Page 308: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/308.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear regression: a perturbation TheoremLet δy be a stochastic variable such that
δy ∼ N(0, τ2A(ATA)−1AT
).
Under the hypotheses of Gauss-Markov Theorem and assumingthat y and δy are uncorrelated, there exist two stochastic variables
δx∗ ∼ N (0, τ2(ATA)−1),δy ∼ N (0, τ2Im),
such that
1. y + δy = A(x∗ + δx∗),
2. x∗ + δx∗ is MVU estimator of X for
y + δy = AX + e, e ∼ N (0, (σ2 + τ2)Im),
3. and
s2 =1
m − n||y + δy − A(x∗ + δx∗)||22,
is the estimator for ρ2 = σ2 + τ2 with s2 ∼ σ2+τ2
m−n χ2(m − n).
157 / 215
![Page 309: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/309.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Least squares problemThe minimum-variance unbiased (MVU) estimators of X and σ2 areclosely related to the solution of the least-squares problem (LSP),
minx||y − Ax||22
where y is a realization of y. The least-squares problem (LSP) hasthe unique solution
x∗ = (ATA)−1ATy,
and the corresponding minimum value is achieved by the square ofthe euclidean norm of
r = y − Ax∗ = (I− P)y
where the matrix I− P = I− A(ATA)−1AT is the orthogonalprojector onto Ker(AT ) and P is the orthogonal projector ontoRange(A).
158 / 215
![Page 310: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/310.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Least squares problem
We remark here that the solution of LSP is deterministic and,therefore, supplies only a realization of the MVU x∗ and of s2 thecorresponding estimator of σ2.The vector x∗ is also the solution of the normal equations, i.e. it isthe unique stationary point of ||y − Ax||22:
ATAx∗ = ATy.
We will denote its residual in the following by
R(x) = AT (y − Ax)
159 / 215
![Page 311: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/311.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Least squares problem
Given a vector x ∈ IRn, the following relations are satisfied:(I− P
) (y − Ax
)=
(I− P
)y,(
y − Ax)
=(y − Ax∗
)+ A(ATA)−1AT
(y − Ax)
=(y − Ax∗
)+ A(ATA)−1R(x),
and, then, we have
||y − Ax||22 = ||y − Ax∗||22 + ||R(x)||2(AT A)−1 ,
owing to the orthogonality between y − Ax∗ and A(ATA)−1R(x).
160 / 215
![Page 312: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/312.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Least squares problem
From the orthogonality of the projector P, the following aresatisfied
y = Py+(I− P
)y,
||y||22 = ||Py||22 + ||(I− P
)y||22,
||y||22 − ||Py||22 = ||(I− P
)y||22 = ||y − Ax∗||22.
Moreover, we have
||Py||22 = yTA(ATA
)−1ATy = x∗TATAx∗,
and, then we conclude that
||y||22 − ||x∗||2AT A = ||(I− P
)y||22 = ||y − Ax∗||22.
161 / 215
![Page 313: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/313.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Least squares problem
Finally, it is easy to verify that, given x as an approximation of x∗,
δy = −A(ATA)−1R(x)
is the minimum norm solution of
minw||w||22 such that ATAx = AT (y + w).
Moreover, using R(x) = AT (y − Ax) = ATA(x∗ − x), we have
||δy||22 = ||R(x)||2(AT A)−1 = ||x∗ − x||2AT A.
162 / 215
![Page 314: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/314.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Probabilistic tests and perturbation theory
We have that expression
||δy||22 = ||R(x)||2(AT A)−1 = ||x∗ − x||2AT A.
gives a useful key to understand our stopping criteria and theirprobabilistic nature. If y can be seen as a realization of astochastic variable
δy ∼ N(0, τ2P
)then, based on the perturbation Theorem, the values x and
r = y−Ax are realizations of the stochastic variables associated to
y + δy = AX + e, e ∼ N (0, (σ2 + τ2)Im),
163 / 215
![Page 315: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/315.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Probabilistic tests and perturbation theory
We have that expression
||δy||22 = ||R(x)||2(AT A)−1 = ||x∗ − x||2AT A.
gives a useful key to understand our stopping criteria and theirprobabilistic nature. If y can be seen as a realization of astochastic variable
δy ∼ N(0, τ2P
)then, based on the perturbation Theorem, the values x and
r = y−Ax are realizations of the stochastic variables associated to
y + δy = AX + e, e ∼ N (0, (σ2 + τ2)Im),
In practice, we can only check the plausibility of this hypothesisusing statistical tests. Fixing some probability threshold η, wecheck if there is any statistical reason for refusing the previoushypothesis, i.e. the probability we are wrong is very low (< η).
163 / 215
![Page 316: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/316.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
χ2 distribution test
δy is a projection onto Range (A). If δy is a realization of astochastic variable δy satisfying
minw||w||22 such that ATAx = AT (y + w).
then ‖δy‖22 is a realization of ‖δy‖2
2 ∼ τ2χ2(n).
164 / 215
![Page 317: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/317.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
χ2 distribution test
Therefore, we consider that δy is a sample of the stochasticvariable δy, if for some small enough η,
Probability(‖δy‖22 ≥ ‖δy‖2
2) ≥ 1− η,
where we assume that the random variable‖δy‖2
2τ2 follows a centered
χ2 distribution with n degrees of freedom.
164 / 215
![Page 318: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/318.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
χ2 distribution test
Thus, we can formulate our criterion as
pχ
(‖δy‖22
τ2, n
)≡ Probability
(‖δy‖22
τ2≤ ‖δy‖
22
τ2
)≤ η, (5)
where, since δy is a Gaussian distribution with covariance matrixA(ATA)−1AT , the value of pχ (., n) is the cumulative distributionfunction of the χ2 distribution Abramowitz-Stegun (26.4): Theprobability that X2 =
∑i X
2i with ν degrees of freedom
Xi ∼ N(0, 1), is such that X 2 ≤ χ2 is
Prob(χ2|ν) =[2ν/2Γ(
ν
2)]−1
∫ χ2
0tν2−1e
t2 dt.
164 / 215
![Page 319: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/319.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
χ2 distribution test
Moreover, the corresponding x is the exact solution of
ATAx = AT (y + δy).
Thus, we can interpret x as a realization of the stochastic variablex∗ + δx∗ and Ax as a realization of y + δy: i.e. we have, withprobability η, realizations consistent with the perturbed linearregression problem, that, if we choose τ2 σ2, is only marginallydifferent from the original.
164 / 215
![Page 320: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/320.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Stopping criteria for CGLS
If we use the conjugate gradient method in order to compute thesolution, it is quite natural to have a stopping criterion which takesadvantage of the minimization property of this method and of thestochastic properties of the underpinning problem
165 / 215
![Page 321: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/321.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Stopping criteria for CGLS
PCGLS algorithm Given an initial guess x(0), compute r (0) =(y−Ax(0)), R(0) = AT r (0), and solve Mz(0) = R(0).
Set q(0) = z(0), β0 = 0, ν0 = 0, χ1 = R(0)T z(0), and ξ−d =∞.
k = 0
while z(ξk−d , ‖r (0)‖2, νk , τ2, σ2)) > η do
k = k + 1;
p = Aq(k−1);
αk−1 = χk/||p||22;ψk = αk−1χk ; νk = νk−1 + ψk ;
x(k) = x(k−1) + αk−1q(k−1);
R(k) = R(k−1) − αk−1AT q(k−1);
Solve Mz(k) = R(k);
χk+1 = R(k)T z(k) ;βk = χk+1/χk ;
q(k) = z(k) + βkq(k−1);
if k > d then
ξk−d =k∑
j=k−d+1
ψj ;
elseξk−d =∞;
endifend while.
165 / 215
![Page 322: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/322.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
χ2 stopping criteria for PCGLS
To detect the convergence as early as possible and avoidover-solving in the LSP, we consider a δyk with minimumEuclidean norm such that x(k) exactly solves a LS problem. Usingthe estimations we have
IF pχ
(ξkτ2, n
)≤ η THEN STOP .
In order to have perturbations of y that not distort excessively thestatistical properties of the original linear regression, we assumethat τ2 σ2.
166 / 215
![Page 323: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/323.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
χ2 stopping criteria for PCGLS
We re-iterate that χ2 test is a measure of the probability that thenumerical values computed at step k will be statistically equivalentto those obtained solving an LSP related to a perturbed linearregression model exactly where the statistical errorse ∼ N
(0, (σ2 + τ2)Im
), i.e. small value of η will indicate that the
probability of stopping at the wrong place is small.
167 / 215
![Page 324: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/324.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
χ2 stopping criteria for PCGLS
We re-iterate that χ2 test is a measure of the probability that thenumerical values computed at step k will be statistically equivalentto those obtained solving an LSP related to a perturbed linearregression model exactly where the statistical errorse ∼ N
(0, (σ2 + τ2)Im
), i.e. small value of η will indicate that the
probability of stopping at the wrong place is small. In PCGLS,we can choose
z(ξk , ‖r (0)‖2, νk , τ2, σ2) = pχ
(ξk(m − n)
τ2, n
).
167 / 215
![Page 325: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/325.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Choice of η and τ 2 for the χ2 and F-test stopping criteria
We seek choices of η and τ2 that will depend on the properties ofthe problem that we want to solve and, in the practical cases, wewould like η and τ2 to be much larger than ε , the roundoff unit ofthe computer finite precision arithmetic.
168 / 215
![Page 326: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/326.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Choice of η and τ 2 for the χ2 and F-test stopping criteria
The choice of η is related to the probability the user subjectivelyfeels as adequate, i.e. he/she accepts that the probability ofchoosing the wrong iterate is less than η. In our experiments, wechose η = 10−8 which is quite conservative. This value is close tothe probability of winning the lotto.
168 / 215
![Page 327: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/327.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Choice of η and τ 2 for the χ2 and F-test stopping criteria
The choice of τ2 is also related to a priori knowledge of thestatistical properties of the linear regression problem and inparticular to the user knowledge of a reliable value of σ2 or of aninterval where σ2 lies. We experimented with several values of τ2.The numerical results suggest that the choice τ2 = σ2 givesreliable answers in the majority of our tests and they are alwaysconsistent with the results of Theorem on perturbations. When σ2
is approximated by its upper bound (‖y‖22 − νk)/(m − n) and the
dynamical choices are used τ2k = (‖y‖2
2 − νk)/(m− n) we can havean early stop because (‖y‖2
2 − νk)/(m− n) is a poor approximationof the true standard deviation. However, smaller values of τ2
k
(τ2k = 0.1(‖y‖2
2 − νk)/(m − n) or τ2k = 0.01(‖y‖2
2 − νk)/(m − n))proved more robust and reliable. In these cases it would be usefulto have lower bound approximations of σ2. Unfortunately, tocompute a lower bound of (‖y‖2
2 − νk)/(m − n) can be costly.
168 / 215
![Page 328: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/328.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Choice of η and τ 2 for the χ2 and F-test stopping criteria
The values of ξk and νk are lower bounds respectively for the trueenergy norm of the errors and the energy norm of the solution,which are both independent of the preconditioner used. However, agood preconditioner will help to reduce the delay factor d . i.e. wewill have better approximations at a cheaper computational cost.
168 / 215
![Page 329: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/329.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Data assimilation test
Data assimilation problems constitute an important class ofregression problems. Their purpose is to reconstruct the initialconditions at t = 0 of a dynamical system based on knowledge ofthe system’s evolution laws and on observations of the state attimes ti . More precisely, consider a linear dynamical systemdescribed by the equation u = f (t, u) whose solution operator isgiven by u(t) = M(t)u0. Assume that the system state is observed(possibly only in parts) at times tiNi=0, yielding observationvectors yiNi=0, whose model is given by yi = Hu(ti ) + ε, where εis a noise with covariance matrix Ri = σ2I .
169 / 215
![Page 330: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/330.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Data assimilation test
We are then interested in finding u0 which minimizes
1
2
N∑i=0
‖HM(ti )u0 − yi‖2R−1i.
We consider here the case where the dynamical system is the linearheat equation in a two-dimensional domain, defined onS2 = [0, 1]× [0, 1] by
∂u
∂t= −∆u in S2, u = 0 on ∂S2, u(., 0) = u0 in S2.
170 / 215
![Page 331: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/331.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Data assimilation test
The system is integrated with timestep dt, using an implicit Eulerscheme. In the physical domain, a regular finite difference schemeis taken for the Laplace operator, with same spacing h in the twospatial dimensions. The data of our problem is computed byimposing a solution u0(x , y , 0) computing the exact systemtrajectory and observing Hu at every point in the spatial domainand at every time step. In our application, m = 8100,n = 900 = 302, dt = 1, h = 1/31, N = 8 andH = diag(11.5, 21.5, . . . , n1.5). The observation vector y is obtainedby imposing u0(x , y , 0) = 1
4 sin( 14 x)(x − 1) sin(5y)(y − 1), and by
adding a random measurement error with Gaussian distributionwith zero mean and covariance matrix Ri = σ2In, where σ = 10−3.In our numerical experiments, we use PCGLS withoutpreconditioner.
171 / 215
![Page 332: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/332.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Data Assimilation test: results
Our choice of not using a preconditioner is not optimal, however,the choice of d = 5 in this problem gives reliable answers andstable behaviour of the stopping criteria.
172 / 215
![Page 333: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/333.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Data Assimilation test: results
(a) (b)
(c) (d)172 / 215
![Page 334: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/334.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Lecture on LDLT multifrontal and GMRES and FGMRES
173 / 215
![Page 335: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/335.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Outline
I GMRES and Flexible GMRES
I Multifrontal
I Static pivoting
I Roundoff error analysis
I Mixed precision
I Test problems
I Numerical experiments
174 / 215
![Page 336: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/336.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
GMRES and FGMRES
Let r0 = b− Ax0 and Kk(A, r0) be the usual Krylov spaceGMRES
minx∈x0+Kk (A,r0)
||r0 − Ax||2 r0 − Axk⊥AKk(A, r0)
175 / 215
![Page 337: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/337.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
GMRES and FGMRES
Let r0 = b− Ax0 and Kk(A, r0) be the usual Krylov spaceGMRES
minx∈x0+Kk (A,r0)
||r0 − Ax||2 r0 − Axk⊥AKk(A, r0)
GMRES Left preconditioning
L−1Ax = L−1b
(L−1A,L−1b) −→ (A,b)Kk(L−1A,L−1r0) −→ Kk(A, r0)
changes the norm.
175 / 215
![Page 338: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/338.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
GMRES and FGMRES
Let r0 = b− Ax0 and Kk(A, r0) be the usual Krylov spaceGMRES
minx∈x0+Kk (A,r0)
||r0 − Ax||2 r0 − Axk⊥AKk(A, r0)
GMRES Right preconditioning
AM−1y = b
(AM−1, r0) −→ (A, r0)Kk(AM−1, r0) −→ Kk(A, r0)xk = M−1ykAM−1Vk = Vk+1Hk
175 / 215
![Page 339: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/339.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
GMRES and FGMRESLet r0 = b− Ax0 and Kk(A, r0) be the usual Krylov spaceGMRES
minx∈x0+Kk (A,r0)
||r0 − Ax||2 r0 − Axk⊥AKk(A, r0)
GMRES Right preconditioning
AM−1y = b
(AM−1, r0) −→ (A, r0)Kk(AM−1, r0) −→ Kk(A, r0)xk = M−1ykAM−1Vk = Vk+1Hk
Flexible GMRES Right preconditioning
Zk −→ Kk(A, r0), xk = x0 + Zkyk AZk = Vk+1Hk
Zk = span(r0,AM−11 r0, . . . ,
k−1∏j=0
AM−1j
r0)
175 / 215
![Page 340: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/340.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear system
We wish to solve large sparse systems
Ax = b
where A ∈ IRN×N is symmetric indefinite
176 / 215
![Page 341: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/341.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Linear system
A particular and important case arises in saddle-point problemswhere the coefficient matrix is of the form[
H AAT 0
]Since we want accurate solutions and norm-wise backward stability,we will use as preconditioners fast factorizations of A computedusing static pivoting or mixed precision arithmetic.
177 / 215
![Page 342: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/342.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Multifrontal method
ASSEMBLY TREE
AT EACH NODE
F F
F F
11 12
2212
T
F22 ← F22 − FT12F−1
11 F12
178 / 215
![Page 343: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/343.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Multifrontal method
ASSEMBLY TREE AT EACH NODE
F F
F F
11 12
2212
T
F22 ← F22 − FT12F−1
11 F12
178 / 215
![Page 344: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/344.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Multifrontal method
ASSEMBLY TREE AT EACH NODE
F F
F F
11 12
2212
T
F22 ← F22 − FT12F−1
11 F12
178 / 215
![Page 345: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/345.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Multifrontal method
I From children to parent
179 / 215
![Page 346: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/346.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Multifrontal method
I From children to parent
I ASSEMBLYGather/Scatter operations(indirect addressing)
179 / 215
![Page 347: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/347.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Multifrontal method
I From children to parent
I ASSEMBLYGather/Scatter operations(indirect addressing)
I ELIMINATION FullGaussian elimination,Level 3 BLAS (TRSM,GEMM)
179 / 215
![Page 348: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/348.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Multifrontal method
I From children to parent
I ASSEMBLYGather/Scatter operations(indirect addressing)
I ELIMINATION FullGaussian elimination,Level 3 BLAS (TRSM,GEMM)
179 / 215
![Page 349: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/349.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Multifrontal method
F F
F F
11 12
2212
T
Pivot can only be chosen fromF11 block since F22 is NOT fullysummed.
180 / 215
![Page 350: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/350.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Multifrontal method
F F
F F
11 12
2212
T
0
0
Situation wrt rest of matrix
181 / 215
![Page 351: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/351.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Pivoting (1× 1)
x
y
Choose x as 1× 1 pivot if |x | > u|y |where |y | is the largest in column.
182 / 215
![Page 352: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/352.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Pivoting (2× 2)
x2x2x1x3
yz
For the indefinite case, we can choose 2× 2 pivot where werequire ∣∣∣∣∣
[x1 x2
x2 x3
]−1∣∣∣∣∣[|y ||z |
]≤[
1u1u
]where again |y | and |z | are the largest in their columns.
183 / 215
![Page 353: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/353.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Pivoting
x
y
k
k
If we assume that k − 1 pivots are chosen but |xk | < u|y |:
I we can either take the RISK and use it or
I DELAY the pivot and then send to the parent a larger Schurcomplement.
184 / 215
![Page 354: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/354.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Pivoting
x
y
k
k
If we assume that k − 1 pivots are chosen but |xk | < u|y |:I we can either take the RISK and use it or
I DELAY the pivot and then send to the parent a larger Schurcomplement.
184 / 215
![Page 355: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/355.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Pivoting
x
y
k
k
If we assume that k − 1 pivots are chosen but |xk | < u|y |:I we can either take the RISK and use it or
I DELAY the pivot and then send to the parent a larger Schurcomplement.
184 / 215
![Page 356: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/356.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Pivoting
x
y
k
k
If we assume that k − 1 pivots are chosen but |xk | < u|y |:I we can either take the RISK and use it or
I DELAY the pivot and then send to the parent a larger Schurcomplement.
This can cause more work and storage
184 / 215
![Page 357: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/357.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Static Pivoting
An ALTERNATIVE is to use Static Pivoting, by replacing xk by
xk + τ
and CONTINUE.
This is even more important in the case of parallel implementationwhere static data structures are often preferred
185 / 215
![Page 358: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/358.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Static Pivoting
An ALTERNATIVE is to use Static Pivoting, by replacing xk by
xk + τ
and CONTINUE.
This is even more important in the case of parallel implementationwhere static data structures are often preferred
185 / 215
![Page 359: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/359.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Static Pivoting
Several codes use (or have an option for) this device:
I SuperLU (Demmel and Li)
I PARDISO (Gartner and Schenk)
I MA57 (Duff and Pralet)
186 / 215
![Page 360: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/360.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Static Pivoting
We thus have factorized
A + E = LDLT = M
where |E| ≤ τ I
The three codes then have an Iterative Refinement option.IR will converge if ρ(M−1E) < 1
187 / 215
![Page 361: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/361.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Static Pivoting
We thus have factorized
A + E = LDLT = M
where |E| ≤ τ I
The three codes then have an Iterative Refinement option.IR will converge if ρ(M−1E) < 1
187 / 215
![Page 362: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/362.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Static Pivoting
Choosing τ
Increase τ =⇒ increase stability of decomposition
Decrease τ =⇒ better approximation of the original matrix,reduces ||E||Trade-off
I ≈ ε =⇒ big growth in preconditioning matrix M
I ≈ 1 =⇒ huge error ||E||.
Conventional wisdom is to choose
τ = O(√ε )
In real life ρ(M−1E ) > 1
188 / 215
![Page 363: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/363.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Static Pivoting
Choosing τ
Increase τ =⇒ increase stability of decomposition
Decrease τ =⇒ better approximation of the original matrix,reduces ||E||Trade-off
I ≈ ε =⇒ big growth in preconditioning matrix M
I ≈ 1 =⇒ huge error ||E||.
Conventional wisdom is to choose
τ = O(√ε )
In real life ρ(M−1E ) > 1
188 / 215
![Page 364: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/364.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Static Pivoting
Choosing τ
Increase τ =⇒ increase stability of decomposition
Decrease τ =⇒ better approximation of the original matrix,reduces ||E||
Trade-off
I ≈ ε =⇒ big growth in preconditioning matrix M
I ≈ 1 =⇒ huge error ||E||.
Conventional wisdom is to choose
τ = O(√ε )
In real life ρ(M−1E ) > 1
188 / 215
![Page 365: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/365.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Static Pivoting
Choosing τ
Increase τ =⇒ increase stability of decomposition
Decrease τ =⇒ better approximation of the original matrix,reduces ||E||Trade-off
I ≈ ε =⇒ big growth in preconditioning matrix M
I ≈ 1 =⇒ huge error ||E||.
Conventional wisdom is to choose
τ = O(√ε )
In real life ρ(M−1E ) > 1
188 / 215
![Page 366: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/366.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Static Pivoting
Choosing τ
Increase τ =⇒ increase stability of decomposition
Decrease τ =⇒ better approximation of the original matrix,reduces ||E||Trade-off
I ≈ ε =⇒ big growth in preconditioning matrix M
I ≈ 1 =⇒ huge error ||E||.
Conventional wisdom is to choose
τ = O(√ε )
In real life ρ(M−1E ) > 1
188 / 215
![Page 367: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/367.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Static Pivoting
Choosing τ
Increase τ =⇒ increase stability of decomposition
Decrease τ =⇒ better approximation of the original matrix,reduces ||E||Trade-off
I ≈ ε =⇒ big growth in preconditioning matrix M
I ≈ 1 =⇒ huge error ||E||.
Conventional wisdom is to choose
τ = O(√ε )
In real life ρ(M−1E ) > 1
188 / 215
![Page 368: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/368.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Right preconditioned GMRES
procedure [x] = right Prec GMRES(A,M,b)x0 = M−1b, r0 = b− Ax0 and β = ||r0||v1 = r0/β; k = 0;while ||rk || > µ(||b||+ ||A|| ||xk ||)
k = k + 1;zk = M−1vk ; w = Azk ;for i = 1, . . . , k do
hi ,k = vTi w ;w = w − hi ,kvi ;
end for;hk+1,k = ||w||;vk+1 = w/hk+1,k ;Vk = [v1, . . . , vk ]; Hk = hi ,j1≤i≤j+1;1≤j≤k ;yk = arg miny ||βe1 −Hky||;xk = x0 + M−1Vkyk and rk = b− Axk ;
end while ;end procedure.189 / 215
![Page 369: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/369.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Right preconditioned Flexible GMRES
procedure [x] =FGMRES(A,Mi ,b)
x0 = M−10 b, r0 = b− Ax0 and β = ||r0||
v1 = r0/β; k = 0;while ||rk || > µ(||b||+ ||A|| ||xk ||)
k = k + 1;
zk = M−1k vk ; w = Azk ;
for i = 1, . . . , k dohi ,k = vTi w ;w = w − hi ,kvi ;
end for;hk+1,k = ||w||; vk+1 = w/hk+1,k ;Zk = [z1, . . . , zk ]; Vk = [v1, . . . , vk ];Hk = hi ,j1≤i≤j+1;1≤j≤k ;yk = arg miny ||βe1 −Hky||;xk = x0 + Zkyk and rk = b− Axk ;
end while ;end procedure.190 / 215
![Page 370: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/370.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error 1
The computed L and D in floating-point arithmetic satisfy
A + δA + τE = M
||δA|| ≤ c(n)ε || |L| |D| |LT | ||||E|| ≤ 1.
The perturbation δA must have a norm smaller than τ , in order tonot dominate the global error.
A sufficient condition for this is n ε || |L| |D| |LT | || ≤ τ
|| |L| |D| |LT | || ≈ nτ =⇒ ε ≤ τ2
n2
Moreover, we assume that max||M−1||, ||Zk || ≤ cτ .
191 / 215
![Page 371: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/371.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error 1
The computed L and D in floating-point arithmetic satisfy
A + δA + τE = M
||δA|| ≤ c(n)ε || |L| |D| |LT | ||||E|| ≤ 1.
The perturbation δA must have a norm smaller than τ , in order tonot dominate the global error.
A sufficient condition for this is n ε || |L| |D| |LT | || ≤ τ
|| |L| |D| |LT | || ≈ nτ =⇒ ε ≤ τ2
n2
Moreover, we assume that max||M−1||, ||Zk || ≤ cτ .
191 / 215
![Page 372: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/372.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error 1
The computed L and D in floating-point arithmetic satisfy
A + δA + τE = M
||δA|| ≤ c(n)ε || |L| |D| |LT | ||||E|| ≤ 1.
The perturbation δA must have a norm smaller than τ , in order tonot dominate the global error.
A sufficient condition for this is n ε || |L| |D| |LT | || ≤ τ
|| |L| |D| |LT | || ≈ nτ =⇒ ε ≤ τ2
n2
Moreover, we assume that max||M−1||, ||Zk || ≤ cτ .
191 / 215
![Page 373: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/373.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error 1
The computed L and D in floating-point arithmetic satisfy
A + δA + τE = M
||δA|| ≤ c(n)ε || |L| |D| |LT | ||||E|| ≤ 1.
The perturbation δA must have a norm smaller than τ , in order tonot dominate the global error.
A sufficient condition for this is n ε || |L| |D| |LT | || ≤ τ
|| |L| |D| |LT | || ≈ nτ =⇒ ε ≤ τ2
n2
Moreover, we assume that max||M−1||, ||Zk || ≤ cτ .
191 / 215
![Page 374: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/374.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error
The roundoff error analysis of both FGMRES and GMRES can bemade in four stages:
1. Error analysis of the Arnoldi-Krylov process (Giraud andLangou, Bjorck and Paige, and Paige, Rozloznık, andStrakos).
2. Error analysis of the Givens process used on the upperHessenberg matrix Hk in order to reduce it to uppertriangular form.
3. Error analysis of the computation of xk in FGMRES andGMRES.
4. Use of the static pivoting properties and A + E = LDLT inorder to have the final expressions.
The first two stages of the roundoff error analysis are the same forboth FGMRES and GMRES. The last stage is specific to each oneof the two algorithms.
192 / 215
![Page 375: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/375.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff errorThe roundoff error analysis of both FGMRES and GMRES can bemade in four stages:
1. Error analysis of the Arnoldi-Krylov process (Giraud andLangou, Bjorck and Paige, and Paige, Rozloznık, andStrakos).MGS applied to
z1 = M−11 r0/||r0||, zj = M−1
j vjC = (z1,Az1,Az2, . . . ) = Vk+1Rk
Rk =
||r0|| H1,1 . . . H1,k
0 H2,1 . . . H2,k
0 0 . . . H3,k...
......
...0 0 0 Hk+1,k
2. Error analysis of the Givens process used on the upperHessenberg matrix Hk in order to reduce it to uppertriangular form.
3. Error analysis of the computation of xk in FGMRES andGMRES.
4. Use of the static pivoting properties and A + E = LDLT inorder to have the final expressions.
The first two stages of the roundoff error analysis are the same forboth FGMRES and GMRES. The last stage is specific to each oneof the two algorithms.
192 / 215
![Page 376: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/376.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error
The roundoff error analysis of both FGMRES and GMRES can bemade in four stages:
1. Error analysis of the Arnoldi-Krylov process (Giraud andLangou, Bjorck and Paige, and Paige, Rozloznık, andStrakos).
2. Error analysis of the Givens process used on the upperHessenberg matrix Hk in order to reduce it to uppertriangular form.
3. Error analysis of the computation of xk in FGMRES andGMRES.
4. Use of the static pivoting properties and A + E = LDLT inorder to have the final expressions.
The first two stages of the roundoff error analysis are the same forboth FGMRES and GMRES. The last stage is specific to each oneof the two algorithms.
192 / 215
![Page 377: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/377.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error
The roundoff error analysis of both FGMRES and GMRES can bemade in four stages:
1. Error analysis of the Arnoldi-Krylov process (Giraud andLangou, Bjorck and Paige, and Paige, Rozloznık, andStrakos).
2. Error analysis of the Givens process used on the upperHessenberg matrix Hk in order to reduce it to uppertriangular form.
3. Error analysis of the computation of xk in FGMRES andGMRES.
4. Use of the static pivoting properties and A + E = LDLT inorder to have the final expressions.
The first two stages of the roundoff error analysis are the same forboth FGMRES and GMRES. The last stage is specific to each oneof the two algorithms.
192 / 215
![Page 378: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/378.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error
The roundoff error analysis of both FGMRES and GMRES can bemade in four stages:
1. Error analysis of the Arnoldi-Krylov process (Giraud andLangou, Bjorck and Paige, and Paige, Rozloznık, andStrakos).
2. Error analysis of the Givens process used on the upperHessenberg matrix Hk in order to reduce it to uppertriangular form.
3. Error analysis of the computation of xk in FGMRES andGMRES.
4. Use of the static pivoting properties and A + E = LDLT inorder to have the final expressions.
The first two stages of the roundoff error analysis are the same forboth FGMRES and GMRES. The last stage is specific to each oneof the two algorithms.
192 / 215
![Page 379: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/379.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error FGMRESTheorem 1.
σmin(Hk) > c7(k , 1)ε ||Hk ||+O(ε 2) ∀k ,
|sk | < 1− ε , ∀k ,
(where sk are the sines computed during the Givens algorithm)and
2.12(n + 1)ε < 0.01 and 18.53ε n32κ(C(k)) < 0.1 ∀k
∃k, k ≤ n
such that, ∀k ≥ k , we have
||b− Axk || ≤ c1(n, k)ε(||b||+ ||A|| ||x0||+ ||A|| ||Zk || ||yk ||
)+O(ε 2).
193 / 215
![Page 380: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/380.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error FGMRESMoreover, if Mi = M,∀i ,
ρ = 1.3 ||Wk ||+ c2(k , 1)ε ||M|| ||Zk || < 1 ∀k < k ,
where
Wk = [Mz1 − v1, . . . ,Mzk − vk ] ,
we have:
||b− Axk || ≤
c(n, k)γε (||b||+ ||A|| ||x0||+ ||A|| ||Zk || ||M(xk − x0)||) +O(ε 2)
γ =1.3
1− ρ.
194 / 215
![Page 381: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/381.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error FGMRESTheorem 2Under the Hypotheses of Theorem 1, and
c(n)ε || |L| |D| |LT | || < τ
c(n, k)γε ||A|| ||Zk || < 1 ∀k < k
max||M−1||, ||Zk || ≤ cτ
we have
||b− Axk || ≤ 2µε (||b||+ ||A|| (||x0||+ ||xk ||)) +O(ε 2).
µ =c(n, k)
1− c(n, k)ε ||A|| ||Zk ||
195 / 215
![Page 382: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/382.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error FGMRESTheorem 2Under the Hypotheses of Theorem 1, and
c(n)ε || |L| |D| |LT | || < τ
c(n, k)γε ||A|| ||Zk || < 1 ∀k < k
max||M−1||, ||Zk || ≤ cτ
we have
||b− Axk || ≤ 2µε (||b||+ ||A|| (||x0||+ ||xk ||)) +O(ε 2).
µ =c(n, k)
1− c(n, k)ε ||A|| ||Zk ||195 / 215
![Page 383: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/383.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error right preconditionedGMRES
Theorem 3We assume of applying Iterative Refinement for solvingM(xk − x0) = Vk yk at last step.
Under the Hypotheses of Theorem 1 and c(n)ε κ(M) < 1
∃k, k ≤ n
such that, ∀k ≥ k , we have
||b− Axk || ≤ c1(n, k)ε||b||+ ||A|| ||x0||+
||A|| ||Zk || ||M(xk − x0)||+||AM−1|| ||M|| ||xk − x0||+||AM−1|| || |L| |D| |LT | || ||M(xk − x0)||
+O(ε 2).
196 / 215
![Page 384: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/384.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error right preconditionedGMRES
As we did for FGMRES, if
c(n)ε || |L| |D| |LT | || < τ
we can prove that ∃k∗ s.t ∀k ≥ k∗ the right preconditionedGMRES computes a xk s.t.
||b− Axk || ≤ c(n, k) ε[||b||+ ||A|| ||x0||+
||A|| ||Zk || ||M(xk − x0)||+|| |L| |D| |LT | || ||M (xk − x0)||
]+O(ε 2).
197 / 215
![Page 385: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/385.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Roundoff error right preconditionedGMRES
As we did for FGMRES, if
c(n)ε || |L| |D| |LT | || < τ
we can prove that ∃k∗ s.t ∀k ≥ k∗ the right preconditionedGMRES computes a xk s.t.
||b− Axk || ≤ c(n, k) ε[||b||+ ||A|| ||x0||+
||A|| ||Zk || ||M(xk − x0)||+|| |L| |D| |LT | || ||M (xk − x0)||
]+O(ε 2).
197 / 215
![Page 386: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/386.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Mixed precision arithmetic
I Very fast 32-bit arithmetic unitM is the fl(LU) of A and ||M− A|| ≤ c(N)
√ε ||A||
(ε = 2.2× 10−16)
I We use 32-bit arithmetic for factorization and triangular solves
I If κ(A)√ε > 1 then Iterative Refinement may not converge.
FGMRES does
I ||Wk || ≤√ε c(N)||A|| < 1 and
||M(xk − x0)|| ≤ ||b− Axk ||+O(√ε ) ⇒ FGMRES backward
stable
I GMRES is not backward stable
198 / 215
![Page 387: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/387.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Mixed precision arithmetic
I Very fast 32-bit arithmetic unitM is the fl(LU) of A and ||M− A|| ≤ c(N)
√ε ||A||
(ε = 2.2× 10−16)
I We use 32-bit arithmetic for factorization and triangular solves
I If κ(A)√ε > 1 then Iterative Refinement may not converge.
FGMRES does
I ||Wk || ≤√ε c(N)||A|| < 1 and
||M(xk − x0)|| ≤ ||b− Axk ||+O(√ε ) ⇒ FGMRES backward
stable
I GMRES is not backward stable
198 / 215
![Page 388: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/388.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Mixed precision arithmetic
I Very fast 32-bit arithmetic unitM is the fl(LU) of A and ||M− A|| ≤ c(N)
√ε ||A||
(ε = 2.2× 10−16)
I We use 32-bit arithmetic for factorization and triangular solves
I If κ(A)√ε > 1 then Iterative Refinement may not converge.
FGMRES does
I ||Wk || ≤√ε c(N)||A|| < 1 and
||M(xk − x0)|| ≤ ||b− Axk ||+O(√ε ) ⇒ FGMRES backward
stable
I GMRES is not backward stable
198 / 215
![Page 389: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/389.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Mixed precision arithmetic
I Very fast 32-bit arithmetic unitM is the fl(LU) of A and ||M− A|| ≤ c(N)
√ε ||A||
(ε = 2.2× 10−16)
I We use 32-bit arithmetic for factorization and triangular solves
I If κ(A)√ε > 1 then Iterative Refinement may not converge.
FGMRES does
I ||Wk || ≤√ε c(N)||A|| < 1 and
||M(xk − x0)|| ≤ ||b− Axk ||+O(√ε ) ⇒ FGMRES backward
stable
I GMRES is not backward stable
198 / 215
![Page 390: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/390.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Mixed precision arithmetic
I Very fast 32-bit arithmetic unitM is the fl(LU) of A and ||M− A|| ≤ c(N)
√ε ||A||
(ε = 2.2× 10−16)
I We use 32-bit arithmetic for factorization and triangular solves
I If κ(A)√ε > 1 then Iterative Refinement may not converge.
FGMRES does
I ||Wk || ≤√ε c(N)||A|| < 1 and
||M(xk − x0)|| ≤ ||b− Axk ||+O(√ε ) ⇒ FGMRES backward
stable
I GMRES is not backward stable
198 / 215
![Page 391: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/391.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Improved error analysis for FGMRESIf we apply Flex-GMRES to solve the system, using finite-precisionarithmetic conforming to IEEE standard with relative precision εand under the following hypotheses:
2.12(n + 1)ε < 0.01 and c0(n)ε κ(C(k)) < 0.1 ∀k
wherec0(n) = 18.53n
32
and|sk | < 1− ε , ∀k ,
where sk are the sines computed during the Givens algorithmapplied to Hk in order to compute yk , then there exists k , k ≤ nsuch that, ∀k ≥ k , we have
||b− Axk || ≤ c1(n, k)ε(||b||+ ||A|| ||x0||+
||A|| || |Zk | |yk | ||+ ||AZk || ||yk ||)
+O(ε 2).
199 / 215
![Page 392: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/392.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Test Problems
n nnz Description
CONT 201 80595 239596 KKT matrix Convex QP (M2)
CONT 300 180895 562496 KKT matrix Convex QP (M2)
TUMA 1 22967 76199 Mixed-Hybrid finite-element
Test problems
200 / 215
![Page 393: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/393.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
MA57 tests
n nnz(L)+nnz(D) Factorization time
CONT 201 80595 9106766 9.0 sec
CONT 300 180895 22535492 28.8 sec
MA57 without static pivot
nnz(L)+nnz(D)+ Factorization time # static pivotsFGMRES (#it)
CONT 201 5563735 (6) 3.1 sec 27867
CONT 300 12752337 (8) 8.9 sec 60585
MA57 with static pivot τ = 10−8
201 / 215
![Page 394: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/394.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
MA57 tests
n nnz(L)+nnz(D) Factorization time
CONT 201 80595 9106766 9.0 sec
CONT 300 180895 22535492 28.8 sec
MA57 without static pivot
nnz(L)+nnz(D)+ Factorization time # static pivotsFGMRES (#it)
CONT 201 5563735 (6) 3.1 sec 27867
CONT 300 12752337 (8) 8.9 sec 60585
MA57 with static pivot τ = 10−8
201 / 215
![Page 395: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/395.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
|| |L| |D| |LT | || vs 1/τ
102 104 106 108 1010 1012 1014100
105
1010
1015
1020
1 / o
|| |L
| |D
| |LT ||
|| '
|| |L| |D| |LT|| ||'
vs 1/o
TUMA1CONT201CONT300y = 1/o
202 / 215
![Page 396: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/396.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Test Problems: TUMA 1
0 0.5 1 1.5 2x 104
0
0.5
1
1.5
2
x 104
nz = 87760
TUMA 1
203 / 215
![Page 397: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/397.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Test Problems: CONT-201
204 / 215
![Page 398: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/398.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments: TUMA 1
||b − Axk ||||b||+ ||A||||xk ||
||M(xk − x0)||τ IR GMRES FGMRES ||Zk || GMRES FGMRES || |L| |D| |LT | ||
1.0e-03 3.0e-03 1.0e-14 7.2e-17 1.2e+02 3.5e-03 3.5e-03 4.4e+041.0e-04 5.3e-17 1.8e-16 3.1e-17 4.7e+01 4.4e-04 4.4e-04 1.8e+051.0e-05 5.1e-17 1.3e-16 1.9e-17 4.4e+01 4.5e-05 4.5e-05 1.8e+061.0e-06 1.5e-16 1.3e-16 1.9e-17 4.4e+01 4.5e-06 4.5e-06 1.8e+071.0e-07 1.8e-17 1.2e-16 2.0e-17 4.3e+01 4.5e-07 4.5e-07 1.8e+081.0e-08 1.7e-17 1.3e-16 1.8e-17 4.3e+01 4.5e-08 4.5e-08 1.8e+091.0e-09 1.8e-17 2.8e-15 1.8e-17 2.6e+01 4.0e-08 4.0e-08 1.8e+101.0e-10 1.7e-17 4.2e-13 1.8e-17 8.8e+00 4.0e-07 4.0e-07 1.8e+111.0e-11 6.7e-17 1.0e-10 6.2e-17 6.8e+00 4.0e-06 4.0e-06 1.8e+121.0e-12 2.1e-17 1.0e-08 2.2e-17 3.2e+01 4.3e-05 4.3e-05 1.8e+131.0e-13 2.0e-17 2.4e-07 1.9e-17 1.3e+02 3.9e-04 3.9e-04 1.8e+141.0e-14 8.6e-17 8.6e-06 2.1e-17 1.8e+02 4.3e-03 4.3e-03 1.8e+15
TUMA 1 results
205 / 215
![Page 399: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/399.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments: CONT 201
||b − Axk ||||b||+ ||A||||xk ||
||M(xk − x0)||τ IR GMRES FGMRES ||Zk || GMRES FGMRES || |L| |D| |LT | ||
1.0e-03 4.0e-04 1.8e-05 9.8e-06 * 7.1e-04 1.5e-04 8.3e+071.0e-04 4.0e-05 2.0e-07 2.0e-07 * 1.5e-05 1.9e-05 1.8e+081.0e-05 3.5e-06 1.8e-12 1.1e-16 4.1e+05 5.9e-06 1.3e-05 4.4e+091.0e-06 3.5e-07 1.1e-11 2.1e-16 2.7e+06 7.8e-07 7.8e-07 1.8e+101.0e-07 4.0e-08 4.8e-11 1.8e-16 1.4e+08 8.7e-08 8.7e-08 1.9e+121.0e-08 3.8e-13 2.7e-10 5.8e-17 2.1e+07 1.3e-06 1.3e-06 1.8e+131.0e-09 5.5e-17 1.8e-09 4.5e-17 1.1e+07 1.3e-06 1.3e-06 1.5e+131.0e-10 7.7e-17 3.2e-09 7.2e-17 3.4e+05 9.2e-06 9.2e-06 1.5e+141.0e-11 4.6e-17 2.1e-09 4.5e-17 1.9e+03 2.8e-04 2.8e-04 2.6e+151.0e-12 5.2e-17 4.5e-07 3.8e-17 2.0e+02 9.5e-04 9.5e-04 1.6e+161.0e-13 1.3e-16 1.3e-04 2.6e-16 1.6e+02 1.1e-02 1.1e-02 4.1e+171.0e-14 1.2e-03 2.3e-01 2.5e-14 4.3e+02 1.9e-02 1.0e-02 9.2e+18
CONT 201 results
206 / 215
![Page 400: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/400.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments: CONT 300
||b − Axk ||||b||+ ||A||||xk ||
||M(xk − x0)||τ IR GMRES FGMRES ||Zk || GMRES FGMRES || |L| |D| |LT | ||
1.0e-03 3.8e-04 3.6e-05 2.5e-05 * 8.7e-04 1.3e-04 2.5e+081.0e-04 3.6e-05 5.5e-07 5.5e-07 * 6.5e-05 2.8e-05 4.3e+091.0e-05 4.3e-06 8.7e-09 8.7e-09 * 3.7e-06 6.1e-06 1.4e+111.0e-06 3.7e-07 6.9e-11 1.4e-16 3.0e+06 5.7e-07 9.8e-07 6.2e+111.0e-07 6.8e-08 2.1e-10 8.2e-17 7.6e+06 2.3e-07 2.3e-07 2.0e+121.0e-08 2.1e-09 1.4e-08 1.2e-16 7.5e+07 1.8e-06 1.8e-06 4.1e+131.0e-09 1.1e-16 1.6e-05 8.8e-17 3.7e+07 2.8e-04 2.8e-04 3.7e+151.0e-10 3.9e-17 6.8e-07 4.1e-17 3.8e+05 3.6e-04 3.6e-04 9.6e+151.0e-11 4.0e-17 1.6e-06 8.7e-17 1.4e+03 5.3e-03 5.3e-03 1.0e+171.0e-12 7.3e-17 1.1e-06 2.7e-16 1.5e+02 1.0e-02 1.0e-02 1.9e+171.0e-13 1.8e-16 3.4e-03 9.2e-16 1.3e+02 1.9e-01 1.9e-01 1.3e+191.0e-14 1.1e-15 1.4e-01 1.8e-14 2.1e+02 4.7e-02 4.7e-02 6.6e+19
CONT 300 results
207 / 215
![Page 401: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/401.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments
0 5 10 15 20 25 30 3510−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
Number of iterations
Norm
of t
he re
sidua
l sca
led
by ||
A ||
||x|
| + ||
b||
CONT201 Test example
FGMRES o = 10−3
FGMRES o = 10−4
FGMRES o = 10−5
FGMRES o = 10−6
FGMRES o = 10−7
FGMRES o = 10−8
FGMRES o = 10−9
FGMRES o = 10−10
FGMRES o = 10−11
FGMRES o = 10−12
FGMRES o = 10−13
FGMRES o = 10−14
FGMRES on CONT-201 test example
208 / 215
![Page 402: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/402.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments
0 5 10 15 20 25 30 3510−12
10−10
10−8
10−6
10−4
10−2
100
Number of iterations
Norm
of t
he re
sidua
l sca
led
by ||
A ||
||x|
| + ||
b||
CONT201 Test example
RGMRES o = 10−3
RGMRES o = 10−4
RGMRES o = 10−5
RGMRES o = 10−6
RGMRES o = 10−7
RGMRES o = 10−8
RGMRES o = 10−9
RGMRES o = 10−10
RGMRES o = 10−11
RGMRES o = 10−12
RGMRES o = 10−13
RGMRES o = 10−14
GMRES on CONT-201 test example
209 / 215
![Page 403: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/403.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments
0 5 10 15 20 25 30 3510−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
Number of iterations
Norm
of t
he re
sidua
l sca
led
by ||
A ||
||x|
| + ||
b|| '
CONT201 Test example: o = 10−6, 10−8, 10−10
FGMRES o = 10−6
GMRES o = 10−6
FGMRES o = 10−8
GMRES o = 10−8
FGMRES o = 10−10
GMRES o = 10−10
GMRES vs. FGMRES on CONT-201 test example:τ = 10−6, 10−8, 10−10
210 / 215
![Page 404: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/404.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments
0 5 10 15 20 25 30 3510−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
Number of iterations
Norm
of t
he re
sidua
l sca
led
by ||
A ||
||x|
| + ||
b|| '
CONT300 Test example: o = 10−6, 10−8, 10−10
FGMRES o = 10−6
GMRES o = 10−6
FGMRES o = 10−8
GMRES o = 10−8
FGMRES o = 10−10
GMRES o = 10−10
GMRES vs. FGMRES on CONT-300 test example:τ = 10−6, 10−8, 10−10
211 / 215
![Page 405: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/405.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments
0 5 10 15 20 2510−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
ItRefGMRES fullGMRES10 + ItRefGMRES restart=5GMRES restart=3GMRES restart=2GMRES restart=1flexible GMRES
Restarted GMRES vs. FGMRES on CONT-201 test example:τ = 10−8
212 / 215
![Page 406: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/406.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Numerical experiments
0 5 10 15 20 2510−14
10−12
10−10
10−8
10−6
10−4
10−2
100
ItRefGMRES fullGMRES10 + ItRefGMRES restart=5GMRES restart=3GMRES restart=2GMRES restart=1
Restarted GMRES on CONT-201 test example: τ = 10−6
213 / 215
![Page 407: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/407.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
IR vs FGMRES
Iterative refinement FGMRES
Matrix Total it RR Total / inner RR ‖AZk‖ ‖ |Zk | |yk | ‖bcsstk20
30 2.1e-152 / 2 1.4e-11 1.7e+00 4.6e+02
n = 485 4 / 2 3.4e-14 1.6e+00 3.8e-01κ(A) ≈ 4× 1012 6 / 2 7.2e-17 1.6e+00 5.6e-04
bcsstm27
22 1.6e-15
2 / 2 5.8e-11 1.7e+00 2.7e+01n = 1224 4 / 2 1.8e-11 6.3e-01 1.3e+00
κ(A) ≈ 5× 109 6 / 2 6.0e-13 2.0e+00 7.6e-028 / 2 1.5e-13 1.7e+00 1.0e-02
10 / 2 1.2e-14 1.7e+00 1.9e-0312 / 2 2.6e-15 1.8e+00 1.7e-0414 / 2 1.8e-16 1.6e+00 4.3e-05
s3rmq4m1
16 2.2e-15
2 / 2 3.5e-11 1.0e+00 8.6e+01n = 5489 4 / 2 2.1e-13 1.1e+00 3.2e-01
κ(A) ≈ 4× 109 6 / 2 4.5e-15 1.7e+00 6.4e-038 / 2 1.1e-16 1.6e+00 1.3e-04
s3dkq4m253 1.1e-10 10 / 10 6.3e-17 1.2e+00 1.2e+03n = 90449
κ(A) ≈ 7× 1010
214 / 215
![Page 408: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/408.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Summary
I IR with static pivoting is very sensitive to τ and not robust
I GMRES is also sensitive and not robust
I FGMRES is robust and less sensitive (see roundoff analysis)
I Gains from restarting. Makes GMRES more robust, savesstorage in FGMRES ( but not really needed)
I Understanding of why τ ≈ √ε is best.
215 / 215
![Page 409: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/409.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Summary
I IR with static pivoting is very sensitive to τ and not robust
I GMRES is also sensitive and not robust
I FGMRES is robust and less sensitive (see roundoff analysis)
I Gains from restarting. Makes GMRES more robust, savesstorage in FGMRES ( but not really needed)
I Understanding of why τ ≈ √ε is best.
215 / 215
![Page 410: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/410.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Summary
I IR with static pivoting is very sensitive to τ and not robust
I GMRES is also sensitive and not robust
I FGMRES is robust and less sensitive (see roundoff analysis)
I Gains from restarting. Makes GMRES more robust, savesstorage in FGMRES ( but not really needed)
I Understanding of why τ ≈ √ε is best.
215 / 215
![Page 411: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/411.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Summary
I IR with static pivoting is very sensitive to τ and not robust
I GMRES is also sensitive and not robust
I FGMRES is robust and less sensitive (see roundoff analysis)
I Gains from restarting. Makes GMRES more robust, savesstorage in FGMRES ( but not really needed)
I Understanding of why τ ≈ √ε is best.
215 / 215
![Page 412: TU Berlin … · Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli Programme Short introduction to the Finite Element method (FEM) and adaptive](https://reader035.vdocument.in/reader035/viewer/2022063011/5fc5c36ad951d42aad3d1c2e/html5/thumbnails/412.jpg)
Matrices, Graphs, and PDEs: A journey of unexpected relations, BMS, Berlin 2014 Mario Arioli
Summary
I IR with static pivoting is very sensitive to τ and not robust
I GMRES is also sensitive and not robust
I FGMRES is robust and less sensitive (see roundoff analysis)
I Gains from restarting. Makes GMRES more robust, savesstorage in FGMRES ( but not really needed)
I Understanding of why τ ≈ √ε is best.
215 / 215