numerical behavior of inexact saddle point solversnumerical behavior of inexact saddle point solvers...
TRANSCRIPT
![Page 1: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/1.jpg)
Numerical behavior of inexact saddle point solvers
Pavel Jiranek1,2, Miroslav Rozloznık1,2
Faculty of Mechatronics and Interdisciplinary Engineering Studies,Technical University of Liberec,
Czech Republic1
and
Institute of Computer Science,Czech Academy of Sciences, Prague,
Czech Republic2
9th IMACS International Symposiumon Iterative Methods in Scientific Computings
March 17–20, 2008, Lille, France
1 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 2: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/2.jpg)
Saddle point problems
We consider a saddle point problem with the symmetric 2× 2 block form„A BBT 0
«„xy
«=
„f0
«.
A is a square n× n nonsingular (symmetric positive definite) matrix,
B is a rectangular n×m matrix of (full column) rank m.
Applications: mixed finite element approximations, weighted least squares,constrained optimization etc. [Benzi, Golub, and Liesen, 2005].
2 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 3: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/3.jpg)
inexact solutions of inner systems + rounding errors→ inexact saddle point solver
3 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 4: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/4.jpg)
Schur complement reduction method
Compute y as a solution of the Schur complement system
BTA−1By = BTA−1f,
compute x as a solution of
Ax = f −By.
Systems with A are solved inexactly, the computed solution u of Au = b isinterpreted an exact solution of a perturbed system
(A+ ∆A)u = b+ ∆b, ‖∆A‖ ≤ τ‖A‖, ‖∆b‖ ≤ τ‖b‖, τκ(A)� 1.
4 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 5: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/5.jpg)
Iterative solution of the Schur complement system
choose y0, solve Ax0 = f −By0
compute αk and p(y)k
yk+1 = yk + αkp(y)k˛
˛˛˛
solve Ap(x)k = −Bp(y)
k
back-substitution:
A: xk+1 = xk + αkp(x)k ,
B: solve Axk+1 = f −Byk+1,
C: solve Auk = f −Axk −Byk+1,
xk+1 = xk + uk.
9>>>>>>>>>=>>>>>>>>>;inneriteration
r(y)k+1 = r
(y)k − αkB
T p(x)k
9>>>>>>>>>>>>>>>>>>>>>=>>>>>>>>>>>>>>>>>>>>>;
outeriteration
5 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 6: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/6.jpg)
Measure of the limiting accuracy
The limiting (maximum attainable) accuracy is measured by the ultimate(asymptotic) values of:
1 the Schur complement residual: BTA−1f −BTA−1Byk;
2 the residuals in the saddle point system: f −Axk −Byk and −BTxk;
3 the forward errors: x− xk and y − yk.
Numerical example:
A = tridiag(1, 4, 1) ∈ R100×100, B = rand(100, 20), f = rand(100, 1),
κ(A) = ‖A‖ · ‖A−1‖ = 7.1695 · 0.4603 ≈ 3.3001,
κ(B) = ‖B‖ · ‖B†‖ = 5.9990 · 0.4998 ≈ 2.9983.
6 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 7: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/7.jpg)
Accuracy in the outer iteration process
BT (A+ ∆A)−1By = BT (A+ ∆A)−1f,
‖BTA−1f −BTA−1By‖ ≤ τκ(A)
1− τκ(A)‖A−1‖‖B‖2‖y‖.
‖ −BTA−1f +BTA−1Byk − r(y)k ‖ ≤
O(τ)κ(A)
1− τκ(A)‖A−1‖‖B‖(‖f‖+ ‖B‖Yk).
0 50 100 150 200 250 30010
−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
τ = O(u)
τ = 10−2
τ = 10−6
τ = 10−10
iteration number k
rela
tive
resi
dual
nor
ms
||BTA
−1 f−
BTA
−1 B
y k||/||B
TA
−1 f−
BTA
−1 B
y 0||, ||
r(y)
k||/
||r(y
)0
||
7 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 8: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/8.jpg)
Accuracy in the saddle point system
−BTA−1f +BTA−1Byk = −BTxk −BTA−1(f −Axk −Byk)
‖f −Axk −Byk‖ ≤O(α1)κ(A)
1− τκ(A)(‖f‖+ ‖B‖Yk),
‖ −BTxk − r(y)k ‖ ≤
O(α2)κ(A)
1− τκ(A)‖A−1‖‖B‖(‖f‖+ ‖B‖Yk),
Yk ≡ max{‖yi‖ | i = 0, 1, . . . , k}.
Back-substitution scheme α1 α2
A: Generic update
xk+1 = xk + αkp(x)k
τ u
B: Direct substitutionxk+1 = A−1(f −Byk+1)
τ τ
C: Corrected dir. subst.xk+1 = xk +A−1(f −Axk −Byk+1)
u τ
}additionalsystem with A
8 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 9: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/9.jpg)
Generic update: xk+1 = xk + αkp(x)k
0 50 100 150 200 250 30010
−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
τ = O(u)
τ = 10−2
τ = 10−6
τ = 10−10
iteration number k
resi
dual
nor
m ||
f−A
x k−B
y k||
0 50 100 150 200 250 30010
−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
τ = O(u), τ = 10−10, τ=10−6, τ=10−2
iteration number k
rela
tive
resi
dual
nor
ms
||−B
Tx k||/
||−B
Tx 0||,
||r k(y
) ||/||r
0(y) ||
9 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 10: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/10.jpg)
Direct substitution: xk+1 = A−1(f −Byk+1)
0 50 100 150 200 250 30010
−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
τ = O(u)
τ = 10−2
τ = 10−6
τ = 10−10
iteration number k
resi
dual
nor
m ||
f−A
x k−B
y k||
0 50 100 150 200 250 30010
−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
τ = O(u)
τ = 10−2
τ = 10−6
τ = 10−10
iteration number k
rela
tive
resi
dual
nor
ms
||−B
Tx k||/
||−B
Tx 0||,
||r k(y
) ||/||r
0(y) ||
10 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 11: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/11.jpg)
Corrected direct substitution: xk+1 = xk +A−1(f −Axk −Byk+1)
0 50 100 150 200 250 30010
−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
τ = O(u), τ = 10−10, τ=10−6, τ=10−2
iteration number k
resi
dual
nor
m ||
f−A
x k−B
y k||
0 50 100 150 200 250 30010
−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
τ = O(u)
τ = 10−2
τ = 10−6
τ = 10−10
iteration number k
rela
tive
resi
dual
nor
ms
||−B
Tx k||/
||−B
Tx 0||,
||r k(y
) ||/||r
0(y) ||
11 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 12: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/12.jpg)
Forward error of computed approximate solution
‖x− xk‖ ≤ γ1‖f −Axk −Byk‖+ γ2‖ −BTxk‖,
‖y − yk‖ ≤ γ2‖f −Axk −Byk‖+ γ3‖ −BTxk‖,
γ1 = σ−1min(A), γ2 = σ−1
min(B), γ3 = σ−1min(BTA−1B).
0 50 100 150 200 250 30010
−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
τ = O(u)
τ = 10−2
τ = 10−6
τ = 10−10
iteration number k
rela
tive
erro
r no
rms
||x−
x k|| A/||
x−x 0|| A
, ||y
−y k|| B
TA
−1 B
/||y−
y 0|| BTA
−1 B
12 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 13: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/13.jpg)
Conclusions
All bounds of the limiting accuracy depend on the maximum norm ofcomputed iterates, cf. [Greenbaum, 1997].
The accuracy measured by the residuals of the saddle point problemdepends on the choice of the back-substitution scheme [J, R, 2008].
Care must be taken when solving nonsymmetric systems [J, R, 2008b].
0 50 100 150 200 250 300 350 400 450
10−15
10−10
10−5
100
105
iteration number k
rela
tive
resi
dual
nor
ms
||BTA
−1 f−
BTA
−1 B
y k||/||B
TA
−1 f||
and
||r(y
)k
||/||B
TA
−1 f||
The residuals in the outer iteration process and the forward errors ofcomputed approximations are proportional to the backward error insolution of inner systems.
13 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 14: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/14.jpg)
Thank you for your attention.
http://www.cs.cas.cz/∼miro
P. Jiranek and M. Rozloznık. Maximum attainable accuracy of inexact saddlepoint solvers. SIAM J. Matrix Anal. Appl., 29(4):1297–1321, 2008.
P. Jiranek and M. Rozloznık. Limiting accuracy of segregated solution methodsfor nonsymmetric saddle point problems. J. Comput. Appl. Math., 215:28–37,2008.
14 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 15: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/15.jpg)
Null-space projection method
compute x ∈ N(BT ) as a solution of the projected system
(I −Π)A(I −Π)x = (I −Π)f,
compute y as a solution of the least squares problem
By ≈ f −Ax,
Π is the orthogonal projector onto R(B).
The least squares with B are solved inexactly, i.e. the computed solution v ofBv ≈ c is an exact solution of a perturbed least squares problem
(B + ∆B)v ≈ c+ ∆c, ‖∆B‖ ≤ τ‖B‖, ‖∆c‖ ≤ τ‖c‖, τκ(B)� 1.
15 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 16: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/16.jpg)
Iterative solution of the null-space projected system
choose x0, solve By0 ≈ f −Ax0
compute αk and p(x)k ∈ N(BT )
xk+1 = xk + αkp(x)k˛
˛˛˛
solve Bp(y)k ≈ r(x)
k − αkAp(x)k
back-substitution:
A: yk+1 = yk + p(y)k ,
B: solve Byk+1 ≈ f −Axk+1,
C: solve Bvk ≈ f −Axk+1 −Byk,
yk+1 = yk + vk.
9>>>>>>>>>=>>>>>>>>>;inneriteration
r(x)k+1 = r
(x)k − αkAp
(x)k −Bp(y)
k
9>>>>>>>>>>>>>>>>>>>>>=>>>>>>>>>>>>>>>>>>>>>;
outeriteration
16 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 17: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/17.jpg)
Accuracy in the saddle point system
‖f −Axk −Byk − r(x)k ‖ ≤
O(α3)κ(B)
1− τκ(B)(‖f‖+ ‖A‖Xk),
‖ −BTxk‖ ≤O(τ)κ(B)
1− τκ(B)‖B‖Xk,
Xk ≡ max{‖xi‖ | i = 0, 1, . . . , k}.
Back-substitution scheme α3
A: Generic update
yk+1 = yk + p(y)k
u
B: Direct substitutionyk+1 = B†(f −Axk+1)
τ
C: Corrected dir. subst.yk+1 = yk +B†(f −Axk+1 −Byk)
u
}additional leastsquare with B
17 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 18: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/18.jpg)
Generic update: yk+1 = yk + p(y)k
0 10 20 30 40 50 60 70 80 90 10010
−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
τ = O(u), τ = 10−10, τ=10−6, τ=10−2
iteration number
rela
tive
resi
dual
nor
ms
||f−
Ax k−
By k||/
||f−
Ax 0−
By 0||,
||r(x
)k
||/||r
(x)
0||
0 10 20 30 40 50 60 70 80 90 10010
−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
τ = O(u)
τ = 10−2
τ = 10−6
τ = 10−10
iteration numberre
sidu
al n
orm
||−
BTx k||
18 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 19: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/19.jpg)
Direct substitution: yk+1 = B†(f −Axk+1)
0 10 20 30 40 50 60 70 80 90 10010
−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
τ = O(u)
τ = 10−2
τ = 10−6
τ = 10−10
iteration number
rela
tive
resi
dual
nor
ms
||f−
Ax k−
By k||/
||f−
Ax 0−
By 0||,
||r(x
)k
||/||r
(x)
0||
0 10 20 30 40 50 60 70 80 90 10010
−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
τ = O(u)
τ = 10−2
τ = 10−6
τ = 10−10
iteration numberre
sidu
al n
orm
||−
BTx k||
19 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 20: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/20.jpg)
Corrected direct substitution: yk+1 = yk +B†(f −Axk+1 −Byk)
0 10 20 30 40 50 60 70 80 90 10010
−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
τ = O(u), τ = 10−10, τ=10−6, τ=10−2
iteration number
rela
tive
resi
dual
nor
ms
||f−
Ax k−
By k||/
||f−
Ax 0−
By 0||,
||r(x
)k
||/||r
(x)
0||
0 10 20 30 40 50 60 70 80 90 10010
−18
10−16
10−14
10−12
10−10
10−8
10−6
10−4
10−2
100
τ = O(u)
τ = 10−2
τ = 10−6
τ = 10−10
iteration numberre
sidu
al n
orm
||−
BTx k||
20 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers
![Page 21: Numerical behavior of inexact saddle point solversNumerical behavior of inexact saddle point solvers Pavel Jir anek 1;2, Miroslav Rozlo zn k Faculty of Mechatronics and Interdisciplinary](https://reader030.vdocument.in/reader030/viewer/2022040813/5e5784c6f29ac00ba72fa877/html5/thumbnails/21.jpg)
References
M. Benzi, G. H. Golub, and J. Liesen. Numerical solution of saddle pointproblems. Acta Numer., 14:1–137, 2005.
A. Greenbaum. Estimating the attainable accuracy of recursively computedresidual methods. SIAM J. Matrix Anal. Appl., 18(3):535–551, 1997.
P. Jiranek and M. Rozloznık. Maximum attainable accuracy of inexact saddlepoint solvers. SIAM J. Matrix Anal. Appl., 29(4):1297–1321, 2008a.
P. Jiranek and M. Rozloznık. Limiting accuracy of segregated solution methodsfor nonsymmetric saddle point problems. J. Comput. Appl. Math., 215:28–37, 2008b.
21 Pavel Jiranek, Miroslav Rozloznık Numerical behavior of inexact saddle point solvers