School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
Iterative MethodsIterative Methods
Prof. HaeProf. Hae--Jin ChoiJin Choi
[email protected]@cau.ac.kr
Part 3Part 3Chapter 12Chapter 12
1
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
Chapter ObjectivesChapter Objectives
2
l Understanding the difference between the Gauss-Seidel and Jacobi methods.
l Knowing how to assess diagonal dominance and knowing what it means.
l Recognizing how relaxation can be used to improve convergence of iterative methods.
l Understanding how to solve systems of nonlinear equations with successive substitution and Newton-Raphson.
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
GaussGauss--Seidel MethodSeidel Method
3
l The Gauss-Seidel method is the most commonly used iterative method for solving linear algebraic equations [A]{x}={b}.
l For a 3x3 system with nonzero elements along the diagonal, for example, the jth iteration values are found from the j-1th
iteration using:
�
x1j =b1 - a12x2
j-1 - a13x3j-1
a11
x2j =b2 - a21x1
j - a23x3j-1
a22
x3j =b3 - a31x1
j - a32x2j
a33
11 12 13 1 1
21 22 23 2 2
31 23 33 3 3
a a a x ba a a x ba a a x b
é ù ì ü ì üï ï ï ïê ú =í ý í ýê úï ï ï ïê úë û î þ î þ
11 1 1 12 2 13 3
22 1 2 21 1 23 3
33 1 3 31 1 32 2
a x b a x a xa x b a x a xa x b a x a x
= - -
= - -
= - -
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
Jacobi IterationJacobi Iteration
4
l The Jacobi iteration is similar to the Gauss-Seidel method, except the j-1th information is used to update all variables in the jthiteration:a) Gauss-Seidelb) Jacobi
1 11 12 2 13 3
1111 1
2 21 1 23 32
221 1
3 31 1 32 23
33
j jj
j jj
j jj
b a x a xxa
b a x a xxa
b a x a xxa
- -
- -
- -
- -=
- -=
- -=
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
ConvergenceConvergence
5
l The convergence of an iterative method can be calculated by determining the relative percent change of each element in {x}. For example, for the ith element in the jth iteration,
l The method is ended when all elements have converged to a set tolerance. �
ea,i =xij - xi
j-1
xij ´100%
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
Example 12.1Example 12.1
6
Q. Use the Gauss-Seidel Method to solve this set of equations.
4.71 10 .203.03.193.07 1.085.7 0.21.03
321
321
321
=+--=-+
=--
xxxxxxxxx
[ ]{ } 3 2.5 7Tx = -Note : True solution is
Assume that x2 and x3 are zero in the first computation.
32.01.085.7 32
1xxx ++
=7
3.01.03.19 312
xxx +--=
102.03.04.71 21
3xxx +-
=
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2 7
1 12 3
17.85 0.1 0.2
3
j jj x xx
- -+ +=
11 3
219.3 0.1 0.3
7
j jj x xx
-- - +=
1 23
71.4 0.3 0.210
j jj x xx - +
=
616667.23
)0(2.0)0(1.085.71 =
++=x
794524.27
)0(3.0)616667.2(1.03.192 -=
+--=x
005610.710
)794524.2(2.0)616667.2(3.04.713 =
-+-=x
1st iteration
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2 8
990557.23
)005610.7(2.0)794524.2(1.085.71 =
+-+=x
499625.27
)005610.7(3.0)990557.2(1.03.192 -=
+--=x
000291.710
)499625.2(2.0)990557.2(3.04.713 =
-+-=x
2nd iteration
%5.12%100990557.2
616667.2990557.21, =´
-=ea
ea,2 = 11.8%; ea,3 = 0.076%;
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
Diagonal DominanceDiagonal Dominance
9
l The Gauss-Seidel method may diverge, but if the system is diagonally dominant, it will definitely converge.
l Diagonal dominance means:
l Many engineering problems satisfy this requirement
�
aii > aijj=1j¹i
n
å
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
MATLAB ProgramMATLAB Program
10
new2
33
32new1
33
31
33
3new3
old3
22
23new1
22
21
22
2new2
old3
11
13old2
11
12
11
1new1
xaax
aa
abx
xaax
aa
abx
xaax
aa
abx
--=
--=
--=
MATLAB M-file: GaussSeidel
}]{[}{}{ xCdx -=
ïþ
ïý
ü
ïî
ïí
ì=
333
222
111
///
}{ababab
dúúú
û
ù
êêê
ë
é=
0///0///0
][
33323331
22232221
11131112
aaaaaaaaaaaa
C
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2 11
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
RelaxationRelaxation
12
l To enhance convergence, an iterative program can introduce relaxation where the value at a particular iteration is made up of a combination of the old value and the newly calculated value (update for the new one):
where λ is a weighting factor that is assigned a value between 0 and 2.
l 0< λ <1: underrelaxationl λ =1: no relaxationl 1< λ ≤2: overrelaxation
�
xinew = lxi
new + 1- l( )xiold
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
Nonlinear SystemsNonlinear Systems
13
lNonlinear systems can also be solved using the same strategy as the Gauss-Seidel method - solve each system for one of the unknowns and update each unknown using information from the previous iteration.
lThis is called successive substitution.
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
Example 12.2Example 12.2
14
Q. Use successive substitution to determine the roots ofthe following equation. A correct pair of roots is x1 = 2 and x2 = 3. Use the initial guesses of x1 = 1.5 and x2 = 3.5.
573
102212
2121
=+
=+
xxx
xxx
First iteration
2
21
110xxx -
= 2212 357 xxx -=
21429.25.3
)5.1(10 2
1 =-
=x 37516.24)5.3)(21429.2(357 22 -=-=x
Second iteration
20910.037516.24
)21429.2(10 2
1 -=--
=x 709.429)37516.24)(20910.0(357 22 =---=x
Seems to be diverging
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2 15
Use the same equation but with a different format
211 10 xxx -=1
22 3
57xxx -
=
First iteration
Second iteration
17945.2)5.3(5.1101 =-=x 86051.2)17945.2(3
5.3572 =
-=x
94053.1)86051.2(17945.2101 =-=x04955.3
)94053.1(386051.257
2 =-
=x
The approach is converging on the true values.
à The most serious shortcoming of substitution, which depends on the manner in which the equations are formulated.
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
NewtonNewton--RaphsonRaphson
16
l Nonlinear systems may also be solved using the Newton-Raphson method for multiple variables.
l For a one-variable system, the Taylor series approximation and resulting Newton-Raphson equations are:
l For a two-variable system, )()()()( 11 iiiii xfxxxfxf ¢-+= ++ 1
( )( )i
i ii
f xx xf x+ = -¢
�
f1,i+1 = f1,i + x1,i+1 - x1,i( )¶f1,i
¶x1
+ x2,i+1 - x2,i( )¶f1,i
¶x2
x1,i+1 = x1,i -f1,i
¶f2,i
¶x2
- f2,i¶f1,i
¶x2
¶f1,i
¶x1
¶f2,i
¶x2
-¶f1,i
¶x2
¶f2,i
¶x1
f2,i+1 = f2,i + x1,i+1 - x1,i( )¶f2,i
¶x1
+ x2,i+1 - x2,i( )¶f2,i
¶x2
x2,i+1 = x2,i -f2,i
¶f1,i
¶x1
- f1,i¶f2,i
¶x1
¶f1,i
¶x1
¶f2,i
¶x2
-¶f1,i
¶x2
¶f2,i
¶x1
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2 17
1[ ]{ } { } i iZ x x f+ - = -
úúúúúúúú
û
ù
êêêêêêêê
ë
é
¶
¶
¶
¶
¶
¶
¶
¶
¶
¶
¶
¶¶
¶
¶
¶
¶
¶
=
n
ininin
n
iii
n
iii
xf
xf
xf
xf
xf
xf
xf
xf
xf
Z
,
2
,
1
,
,2
2
,2
1
,2
,1
2
,1
1
,1
][
L
MMM
L
L
1, 2, ,{ }Ti i i n ix x x xé ù= ë ûL
1 1, 1 2, 1 , 1{ }Ti i i n ix x x x+ + + +é ù= ë ûL
1, 2, ,{ }T i i n if f f fé ù= ë ûL
( ) ( )
( ) ( )
1, 1,1, 1 1, 1, 1 1, 2, 1 2,
1 2
2, 2,2, 1 2, 1, 1 1, 2, 1 2,
1 2
i ii i i i i i
i ii i i i i i
f ff f x x x x
x xf f
f f x x x xx x
¶ ¶¶ ¶¶ ¶¶ ¶
+ + +
+ + +
= + - + -
= + - + -
1[ ]{ } { } [ ]{ }, where [ ] i iZ x f Z x Z Jacobian matrix+ = - + =
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2 18
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
Example 12.3 (1/2)Example 12.3 (1/2)
19
Q. Use the Newton-Raphson method to determine theroots of the equations. Use the initial guesses of x1 =1.5 and x2 = 3.5.
573
102212
2121
=+
=+
xxx
xxx
5.32)5.3)(5.1(6161 75.36)5.3(33
5.1 5.65.3)5.1(22
212
0,2222
1
0,2
12
0,121
1
0,1
=+=+=¶
¶===
¶
¶
==¶
¶=+=+=
¶
¶
xxxf
xxf
xxf
xxxf
6.5(32.5) 1.5(36.75) 156.125Jacobian = - =
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
Example 12.3 (2/2)Example 12.3 (2/2)
20
5.210)5.3(5.1)5.1( 20,1 -=-+=f
625.157)5.3)(5.1(35.3 20,2 =-+=f
The values of the functions can be evaluated at the initial guesses as
These values can be substituted to give
03603.2125.156
)5.1(625.1)5.32(5.25.11 =--
-=x
84388.2125.156
)75.36)(5.2()5.6(625.15.32 =--
-=x
The computation can be repeated until an acceptable accuracyis obtained.
School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2
MATLAB ProgramMATLAB Program
21