iterative methods for systems of linear equations
TRANSCRIPT
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 1/17
Iterative methodsfor systems of linear
equationsAstrid Xiomara
Rodriguezwww.nebrija.es/.../MetodosMatem/sistemas_lin
eales_iterativos
Iterative methodsfor systems of linear
equationsAstrid Xiomara
Rodriguezwww.nebrija.es/.../MetodosMatem/sistemas_lin
eales_iterativos
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 2/17
Iterative methods for systems
of linear equations
Iterative methods for systems
of linear equations
Introduction
Heat EquationJacobi Method
Gauss-Seidel Method
Overrelaxation methodCapacitor Problem
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 3/17
Direct methods versus
iterative methods
Direct methods versus
iterative methodsDIRECT
Ax =b
x = A\ b
Moderate size
Alter the structure
Rounding error
ITERATIVE
x = Cx + d
x(k+1) = Cx(k) + d
Big size
Zeros preservedTruncation error
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 4/17
Iterative methods compared with direct, we donot guarantee a better approach, however, aremore efficient when working with large matrices
In the resolution for numerical partial differentialequations often appear linear equation systemswith even 100 000 unknowns, in these systemsthe coefficient matrix is sparse, ie a highpercentage of matrix elements are equal to 0. If
there are any patterns in the nonzero elements(eg tridiagonal systems), then an iterativemethod can be very effective.
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 5/17
Heat Equation
Heat Equation
System Eq. linear. Associated matrix
T (T T ) / 2T (T T ) / 2
T (T T ) / 2
T (T T ) / 2
1 0 2
2 1 3
3 2 4
n n-1 n+1
!
!
!
!
/
2 -
- 2 -
- 2
-
- 2
1
1 1
¨
ª
©©©©
©©
¸
º
¹¹¹¹
¹¹
T0 T1 T2 . . . Tn Tn+1
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 6/17
J acobi methodJ acobi method System of linear equations
a x
a x
a x
a x
a x a x a x
a x a x a x
a x a x a x
a x a x a x
b
b
b
b
11 1
21 1
31 1
n1 n
12 2 13 3 1n n
22 2 23 3 2n n
32 2 33 3 3n n
n2 2 n3 3 nn n
1
2
3
n
!
!
!
!
¾
¿
±±±
À
±±±
/
.
.
.
/ / /
.
/
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 7/17
Of all the iterative methods, Jacobi's is theeasiest to implement and understand, however,
is not very efficient in terms of obtainingsolutions.Consider the system:
we can write equivalently as:
Jacobi's method is to use the above formulas asfixed point iteration.
±°
±¯
®
!
!
!
1552
2184
74
z y x
z y x
z y x
±±±
°
±±±
¯
®
!
!
!
5
2158
421
4
7
y x z
z x y
z y x
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 8/17
Fixed point equationFixed point equation
x (bx (b
x (b
x (b
a x a x a x ) / aa x a x a x ) / a
a x a x a x ) / a
a x a x a x ) / a
1 1
1 2
2 3
n n
12 2 13 3 1n n 11
21 1 23 3 2n n 22
31 1 32 2 3n n 33
n1 1 n2 2 n,n 1 n 1 nn
! !
!
!
¾
¿
±±±
À
±±±
/
.
.
.
/ / 1 /
.
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 9/17
J acobi IterationJ acobi Iterationx (b
x (b
x (b
x (b
a x a x a x ) /a
a x a x a x ) /a
a x a x a x ) /a
a x a x a x )/a
1
(k+1)
1
2
(k+1)
2
3
(k+1)
3
n
(k+1)
n
12 2
(k)
13 3
(k)
1n n
(k)
11
21 1
(k)
23 3
(k)
2n n
(k)
22
31 1
(k)
32 2
(k)
3n n
(k)
33
n1 1
(k)
n2 2
(k)
n,n 1 n 1
(k)
nn
!
!
!
!
¾
¿
À
/
.
.
.
/ / 1 /
.
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 10/17
At each step of the Jacobi iteration yields a vectorwith n coordinates
P0 = (x1 (0), x2 (0), ..., xn (0)), ..., Pk = (x1 (k), x2(k), ..., xn (k))
where the initial estimate (x1 (0), x2 (0), ..., xn (0))should be chosen. When you do not have a clue aboutthe solution is usually taken xi (0) = bi / aii
In the example above, if we take P0 = (x (0), y (0), z(0)) = (1,2,2)In the first iteration is obtained
P1 = (1.75, 3375, 3.00)
Generating the sequence of iterations of Jacobi notesthat converges to (2, 4, 3).
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 11/17
Many times the Jacobi iteration method does notwork. Here is an example of rearranging theequations above example.Example:
Now the iteration formula is
and notes that the sequence of Jacobi diverges.Note that the system matrix is not strictlydiagonal dominant.
±
°
±¯
!
!
!
74
2184
1552
z y x
z y x
z y x
±±
±
°
±±±
¯
¡
!
!
!
y x z
z y y
z y x
47
8
421
2
515
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 12/17
G auss-Seidel iterationG auss-Seidel iteration
x (b
x (b
x (b
x (b
a x a x a x )/a
a x a x a x )/a
a x a x a x )/a
a x a x a x )/a
1
(k+1)
1
2
(k+1)
2
3
(k+1)
3
n
(k+1)
n
12 2
(k)
13 3
(k)
1n n
(k)
11
21 1
(k+1)
23 3
(k)
2n n
(k)
22
31 1
(k+1)
32 2
(k+1)
3n n
(k)
33
n1 1
(k+1)
n2 2
(k+1)
n,n 1 n 1
(k+1)
nn
!
!
!
!
¾
¿
±
/
.
.
.
/ / /
.
±±
À
±±±
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 13/17
The Gauss-Seidel method is a modification of theJacobi method which accelerates the convergence of the latter.Note that the Jacobi method generates a sequence foreach unknown(X1 (k)), ..., (xn (k)). Since xi (k +1) is probablybetter approximated by xi (k) instead of xi (k) in thecalculation of xi +1 (k +1) we use xi (k +1).Apply this strategy to the example 1 and comprubesethe speed of convergence.
The Gauss-Seidel substantially cut the number of iterations to make for some precision in thesolution. Obviously the convergence criteria aresimilar to those of Jacobi.
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 14/17
Overrelaxation method
Overrelaxation method
1)+(k
i
(k)
i
1)+(k
i
(k)
i
1)+(k
ii
i
(k)
i
1)+(k
i
i(k)i
1)+(ki
xww)x(1x
xxz
2<w<0 ;wzxx
:tionIverrelaxa
zxx
:SeidelGauss
Ö
Ö
Ö
!
!
!
!
xik
zi xik+1
ik+1x
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 15/17
Overrelaxation method reducesthe number of iterations in thecalculations of solutions of linear
systems by Gauss-Seidel. It isbased on each iteration to obtain aweighted average (only for vectorelements before the position
calculation) for the solution of theJacobi method and the solution of the Gauss-Seidel.
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 16/17
P ass
Overrelaxation
P ass
Overrelaxation
x (1 )x (b
x (1 )x (b
x (1 )x (b
x (1 )x (b
a x a x a x )/a
a x a x a x ) /a
a x a x a x ) /a
1
(k+1)
1
(k)
1
2
(k+1)
2
(k)
2
3
(k+1)
3
(k)
3
n
(k+1)
n
(k)
n
12 2
(k)
13 3
(k)
1n n
(k)
11
21 1
(k+1)
23 3
(k)
2n n
(k)
22
31 1
(k+1)
32 2
(k+1)
3n n
(k)
33
!
!
!
!
[ [
[ [
[ [
[ [
/
.
.
.
/ / 1 /
.a x a x a x )/an1 1
(k+1)
n2 2
(k+1)
n,n 1 n 1
(k+1)
nn
¾
¿
±±±±
À
±
±±±
8/9/2019 Iterative Methods for Systems of Linear Equations
http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 17/17
SummarySummary Iterative methods are applied to large
and sparse matrices.
The cost per iteration is O (n2) or less if you take advantage of the dispersityAre expected to converge in less than nsteps.
The matrix has to fulfill certainconditions for the method to converge.