MEE3017 Computer Modeling Techniques in Engineering
Chapter 2 Solving Linear Equation • A linear equation represents the linear dependence of a quantity φ on a set of
variables x1 through xn and a set of constant coefficients α1 through αn; its form is φααα =+++ nnxxx ...2211 • If we replace φ by a singly subscripted quantity fi and the coefficients αj by a doubly
subscripted quantity αij , we may write a system of n such equations in the form
(2.1)
nnnnjnjnn
ininjijii
nnjj
nnjj
fxxxx
fxxxx
fxxxx
fxxxx
=+++++
=+++++
=+++++
=+++++
αααα
αααα
αααα
αααα
LL
LL
LL
LL
21211
22111
222222121
111212111
::::
::::
• The solution of such systems for the quantities xj, when the coefficients aij and the
values fi are given, pervades engineering applications and computational methods. For example, systems of linear algebraic equations are used directly in mathematical models for electrical, structural, and pipe networks, and in some computational methods for fitting curves to data. In other cases, such as in finite difference and finite element solutions of partial differential equations, they represent approximations of mathematical equations, which cannot be solved by analytical methods. 2.1 Fundamentals of linear algebra 2.1.1 Notation and Definitions • A matrix is a rectangular array of quantities.
An example of a matrix is
(2.2)
==
34333231
24232221
14131211
][aaaaaaaaaaaa
aA ij
Chapter 2 – Page 1
MEE3017 Computer Modeling Techniques in Engineering
• In Eq. (2.2), the bold-faced A is the symbol for the matrix, the literal representation on the far right shows the arrangement of the elements, and the form [aij] is a shorter abstraction of the literal form.
• The matrix has no particular meaning until we associate the elements with a given concept.
• A matrix with m rows and n columns has dimension (m × n) and is referred to as an (m × n) matrix. A column vector c and a row vector r are shown in Eq.(2.3)
(2.3) [ ]4321
3
2
1
][;][ rrrrrrccc
cc ji ==
==
2.1.2 Operations • Addition and subtraction for conformable matrices A and B are summarized by
][][)()()( ijijij
nmnmnmbacCBA ±===±
××× (2.4)
• Matrix multiplication A ⋅ B is defined only if the number of columns in A is equal to
the number of rows in B.
(2.5a) ][)()()( ijnmnrrm
pPBA ==⋅×××
(2.5b) rjirjiji
r
kkjikij babababap +++=×= ∑
=L2211
1)(
• Matrix multiplication is both distributive and associative. • If A,B, and C are matrices with appropriate dimensions to satisfy the addition/
subtraction conditions of Eq. (2.4) and the multiplication conditions of Eq.(2.5a), the distributive and associative properties are expressed in Eq.(2.6) and Eq.(2.7), respectively, by
ACABACBCABACBA ⋅+⋅=⋅+⋅+⋅=+⋅ )(;)( (2.6)
CBACBACBA ⋅⋅=⋅⋅=⋅⋅ )()( (2.7) • Multiplication of a matrix A by a scalar q to form a product matrix S equal to qA is
also a defined operation. The product in this case is given by
Chapter 2 – Page 2
MEE3017 Computer Modeling Techniques in Engineering
][][][)()( ijijij
nmnmqaaqsSAq ====
×× (2.8)
2.1.3 Square Matrices • A square matrix is one that has the same number of rows as the number of columns;
for example, a matrix with dimension (n × n). Special types of square matrices are described as follows.
• A diagonal matrix D equal to [dij] satisfies the condition
jifordij ≠= 0 (2.9) • We may replace [dij] by [di], implicitly it in the correct context so that it is not
confused with the vector notation of Eq. (2.3), and express the diagonal matrix D in the form
(2.10) D d
dd
d
dd
i i
n
n
= =
−
[ ]
1
2
1
0
0
O
O
• The identity matrix I is a diagonal matrix in which every diagonal element has a unit
value; it is the matrix of the scalar unit value. The zero matrix 0 is one whose elements are all zero; it may be either a square matrix or a general rectangular matrix.
• Multiplications involving the general diagonal matrix D, the identity matrix I, and the zero matrix 0 are summarized as follows; each of D, I and 0 is assumed to be an (n x n) matrix.
[ ] [ ]iii
nnnncduucD ===⋅
××× )1()1()(
(2.11a)
[ ] [ ]jjjnnnn
drvvDr ===⋅××× )1()()1(
(2.11b)
[ ] [ ]ijiijnnnnnn
adpPAD ===⋅××× )()()(
(2.11c)
[ ] [ ]ijjijnnnnnn
adqQDA ===⋅××× )()()(
(2.11d)
ccInnnn )1()1()( ×××
=⋅ (2.12a)
Chapter 2 – Page 3
MEE3017 Computer Modeling Techniques in Engineering
rIrnnnn )1()()1( ×××
=⋅ (2.12b)
IAAAInnnnnnnnnn )()()()()( ×××××
⋅==⋅ (2.12c)
00)1()1()( ×××
=⋅nnnn
c (2.13a)
00)1()()1( nnnn
r×××
=⋅ (2.13b)
000)()()()()( nnnnnnnnnn
AA×××××
⋅==⋅ (2.13c)
• The inverse of a square matrix A is denoted by A-1 and is itself a square matrix with
the same dimension as A such that
IAAAA =⋅=⋅ −− 11 (2.14)
The analogous scalar relation is ( ). αα α α− −= =1 1 1 • Two other types of square matrices that will be useful at a later stage are the lower
triangular and upper triangular matrices. The lower triangular form L permits nonzero elements only on and below the diagonal; it may be represented by
(2.15) L ij
n n nn
= =
[ ]l
l
l l
M O
M
M O
M
l l L L L L l
11
21 22
1 2
0
• The upper triangular form U permits nonzero elements only on and above the
diagonal; it may be represented by
(2.16) U u
u u uu u
u
ij
n
n
nn
= =
[ ]
11 12 1
22 2
0
L L L L
O M
M
O M
M
Chapter 2 – Page 4
MEE3017 Computer Modeling Techniques in Engineering
2.1.4 The Determinant of a Square Matrix The determinant of an (n × n) square matrix A is written asAand is defined by either of
∑=
=n
jijijCaA
1)( for any one value of i (2.17a)
or
∑=
=n
iijijCaA
1)( for any one value of j (2.17b)
in which Cij is known as the cofactor of the element aij. • The cofactor Cij of a (n x n) square matrix is obtained by first removing row i and
column j to form a ((n-1) x (n-1)) matrix and then by performing the operation (2.18) removed)j column and i row withA of nt(determinaC ji
ij ×−= +)1( 2.1.5 The Matrix Equation for Linear Algebraic Systems • The concept of matrix multiplication expressed in Eq.(2.5a) and (2.5b) allows us to
represent the system of linear algebraic equations given by Eq.(2.1) in the form
(2.19a)
=
⋅
n
i
2
1
n
2
1
nnnj2n1n
inij2i1i
n2j22221
n1j11211
f
f
ff
x
xx
aaaa
aaaa
aaaaaaaa
M
M
M
M
M
LLL
MM
LLL
MM
LLL
LLL
or, more compactly, in the form (2.19b) fxA =⋅ Here, A is the coefficient matrix formed by coefficients of the linear system, the original right-hand sides of the system are now the components of the column vector f. and the original unknown quantities are now the components of the column vector x. • A unique solution for x requires A to be square matrix, and it requires the system of
equations to be linearly independent. A coefficient matrix with a zero determinant is singular, a unique solution for x requires a nonsingular matrix.
Chapter 2 – Page 5
MEE3017 Computer Modeling Techniques in Engineering
• The solution of Eq.(2.19b) may be derived with the help of the associative law from Eq. (2.7), the concept of the inverse from Eq.(2.14), and operations with the identity matrix from Eq. (2.12a) as follows:
fAxAAxAAxIx ⋅=⋅⋅=⋅⋅=⋅= −−− 111 )()( (2.20) Among the additional tools used in methods for solving matrix equations are row and column exchanges. • A row exchange in the coefficient matrix A of Eq. (2.19b) is simply a position
exchange of two equations. The sequence of the components of x is preserved, but those of the right-hand side vector f must be exchanged to correspond to the new positions. A column exchange in the coefficient matrix does not alter the sequence of equations but it requires a corresponding exchange in the components of x so that the components are multiplied by the appropriate coefficients. Illustrations of these concepts are as follows.
• Exchange of rows I and k:
(2.21a)
=⋅
M
M
M
M
L
M
L
M
k
i
k
i
f
fx
a
a
1
1
• Exchange of columns j and k:
(2.21b) fx
x
aa
aa
k
j
nknj
kj
=
⋅
M
LLL
MM
LLL 11
2.1.6 Computational Techniques for Basic Operations The procedure of numerical computation: • Find a mathematical model or way of presenting the problem. • Choose a numerical method or calculation formula. • Set up the computation procedure. • Draw a program flowchart. • Write a program. • Run the computation. • Check the computation results.
Chapter 2 – Page 6
MEE3017 Computer Modeling Techniques in Engineering
Example 1: Calculation of Inverse Matrix 1) Find a mathematical model or way of presenting the problem. Problem: Find the inverse matrix from 2 by 2 matrix. The notation of a mathematical model is
=−
2221
12111
aaaa
A
1−A is the inverse matrix A.
2) Choose a suitable numerical method or formula. Using a formula of inverse matrix.
=
−
−−
=−
2221
1211
2221
1211
21122211
1 1bbbb
aaaa
aaaaA
3) Set up the computation procedure
Step 1 Data input Step 2 Compute the matrix Step 3 Error control Step 4 Compute the inverse matrix Step 5 Output the result
Chapter 2 – Page 7
MEE3017 Computer Modeling Techniques in Engineering
4) Draw a program flowchart = ≠
22211211 ,,, aaaa
21122211 aaaad −←
dabdabdab
dab
///
/
1122
2121
1212
2211
←−←−←
←
22211211 ,,, bbbb
Matrix is not Non-singular
END
RESULT, OUTPUT
d: 0
DATA INPUT
START
5) Write a program
Numerical computation program includes four parts: a. Define the variables and type b. Data input c. Computation d. Data output
Chapter 2 – Page 8
MEE3017 Computer Modeling Techniques in Engineering
Program example in C
/* < PROGRAM> */ # include <stdio.h> # include <stdlib.h> main( ) int i, j; double a[2][2]; /* 22× INPUT MATRIX */ double b[2][2]; /* 2 2× OUTPUT MATRIX */ double d; /*** STEP 1 DATA INPUT***/ printf (“ 2 INVERSE MATRIX /n *”); 2× for (i=0; i<2; i++) for (j=0; j<2; j++) printf (“a%d%d=”, i+1, j+1); scanf (“%1f”, &a[I][j]); /***STEP 2 CACULATE MATRIX***/ d=a[0][0]*a[1][1]-a[0][1]*a[1][0]; /***STEP3 ERROR CONTROL***/ if (d==0. 0) printf(“IT IS NON-SING ULAR MATRIX/n”); /***STEP 4 CALCULATE INVERSE MATRIX***/ b[0][0]=a[1][1]/d; b[0][1]=-a[0][1]/d; b[1][0]=-a[1][0]/d; b[1][1]=a[0][0]/d; /***STEP 5 RESULT OUTPUT***/ printf (“<SOLUTION>/n”); printf(“B=/n”); for (i=0; i<2; i++) for (j=0; j<2; j++) printf(“ % 1f”, b[i][j]); printf (“/n”);
Data input section
Computing section
Data output section
Variables type
Chapter 2 – Page 9
MEE3017 Computer Modeling Techniques in Engineering
6) Compute the problem
=<
====
×
B
aaaa
3221
22
22
21
12
11
-3.000000 2.000000 2.0 -1.000000
7) Check the computation results Compare with other method such as MATLAB functions. * Tip A good program for numerical computation should have following characteristics: 1) short computation time 2) accurate result 3) short program steps 4) easy to read 5) easy to move to other computers Example : Compare with MATLAB matrix inverse function by using following matrix:
=
3221
A
find ?1 =−A using MATLAB program and MATLAB inverse function.
Chapter 2 – Page 10
MEE3017 Computer Modeling Techniques in Engineering
2.1.7 Introduction to Numerical Computation Linear Equations. Preliminaries Resistive Network.
R1
VVVV
R
R
R
R
R
21
105
104
103
102
101
2
1
35
34
33
32
31
==
Ω×=
Ω×=
Ω×=
Ω×=
Ω×=
V1 R2 R3 R4 V2 i2 i3
i1
R5
Ohm’s law: V=iR (2.22)
Kirchhoff’s laws:
0=∑ I (2.23)
∑ = 0U (2.24) The implication of Kirchhoff’s and Ohm’s law are that the currents i1, i2, and i3 must satisfy the following relations:
235432314
13323212
3422421
)()(
0)(
ViRRRiRiRViRiRRiR
iRiRiRRR i
=+++−−=−++−
=−−++
mnmnmm
nn
nn
bxaxaxa
bxaxaxabxaxaxa
=+⋅⋅⋅++
=+⋅⋅⋅++=+⋅⋅⋅++
2............
211
22222121
11212111
(2.25)
Chapter 2 – Page 11
MEE3017 Computer Modeling Techniques in Engineering
The coefficients and are given real or complex numbers. Eq. (2,25) is a system if m equations with n variables, or “unknowns”, Our job is to determine their values.
ija ib.,...,, 21 nxxx
Eq. (2.25) can be written compactly in matrix form as (2.26) bAx = where ( )ijaA = is an m×n matrix of coefficients, and ( )ibb = and ( )jxx = are column vectors of dimension m and n, respectively. EXAMPLE 2 In the rectangular region shown in Figure 2.2(a), the electric potential is zero on the boundaries. The charge distribution, however, is uniform and given by
286 m×.02ε=vp
Solve Poisson’s equation to determine the potential distribution in the rectangular region. Solution To determine the potential distribution in the rectangular region, we use Poisson’s equation.
20
2
2
2
22 −=
ερ
−=∂
Φ∂+
∂Φ∂
=Φ∇ v
yx
with zero potential Φ on the boundaries. 0=
Figure 2.2 Geometry of the 6 rectangular region and the 28m× mh 2= mesh.
Chapter 2 – Page 12
MEE3017 Computer Modeling Techniques in Engineering
By establishing the rectangular grid shown in Figure 2.2(b), we realize that we have six nodes and, hence, six unknown potentials for which to solve. Replacing ∇ by its finite difference representation, we obtain
Φ2
02)4(1,1,1,,1,12 =+Φ−Φ+Φ+Φ+Φ −+−+ jijijijijih
It should be noted that although the although the mesh size was not explicitly used in solving Laplace’s equation in the previous example, h is included as a part of the matrix formation in solving Poisson’s equation. In SI system of units, h should be in meters. By applying the preceding difference equation at the various nodes in Figure 2.2(b), we obtain the following matrix equation:
−−
−−
−−
411000140100104110011401001041000114
=
ΦΦΦΦΦΦ
6
5
4
3
2
1
−−−−−−
888888
Instead of solving the resulting six equations, we may note some symmetry considerations in Figure 2.2(b). It is clear that
6521 Φ=Φ=Φ=Φ and that 43 Φ=Φ Taking these symmetry considerations into account, the number of equations reduces to two, and we obtain the following solution:
,56.41 =Φ 72.53 =Φ
Chapter 2 – Page 13
MEE3017 Computer Modeling Techniques in Engineering
2.2 Direct Methods for Linear Systems 2.2.1 Gaussian Elimination Example of Elimination Solve the system of Eqs.
(2.27) 39521744
132
321
321
321
=++=++
=++
xxxxxx
xxx
By subtracting the multiple 2 of the first equation from the second equation and the first equation from the third equation, the first “derived system” is obtained:
(2.28) 26412
132
32
32
321
=+−=+
=++
xxxxxxx
The first equation, after being multiplied by 2, becomes 2 (2.29) 624 321 =++ xxx Subtract this from the middle equation in Eq.(2.27) to see that result is 21)67()24()44( 321 −=−+−+− xxx The same rationale yields the third equation in the first derived system. Continue the computation by subtracting the multiple 2 of the second equation of the first derived system from the third equation to obtain the second derived system:
44
12132
3
32
321
=−=+
=++
xxxxxx
The derivation of this upper triangular system of equation is called the forward elimination process. From the third equation of the the system above, we immediately calculate
144
3 ==x
Then the second equation, with 1 substituted for x3, implies that
Chapter 2 – Page 14
MEE3017 Computer Modeling Techniques in Engineering
12
112 −=
−−=x
Finally, from the first equation, with their numerical values replacing x2 and x3, we see that
21
2)1(31
1 −=−−−
=x
and thereby obtain the solution, which 21
1 −=x , 12 −=x , 13 =x .
The computation of the unknowns from the upper triangular system is known as back substitution. Let us assume that in Eq.(2.25) the matrix of coefficients is nonsingular; that is, assume m = n , and that a solution exists and is unique. Then Eq.(2.25) can be rewritten as
(2.30)
1,211
1,22222121
1,11212111
2............
+
+
+
=+⋅⋅⋅++
=+⋅⋅⋅++
=+⋅⋅⋅++
nnnnnnn
nnn
nnn
axaxaxa
axaxaxaaxaxaxa
where, for later notation convenience, we have defined )1(1, niba ini ≥≤=+ .
Assume that , and subtract the multiplying 0≠na11
1
aai of the first equation from the ith
equation for i = 2, ⋅ ⋅ ⋅, n. The coefficient of x1 in the ith equation then becomes 0, and we thereby obtain the first derived system which has the form
)1(1,
)1(2
)1(2
)1(1,2
)1(22
)1(22
1,11212111
+
+
+
=+⋅⋅⋅++
=+⋅⋅⋅++=+⋅⋅⋅++
nnnnnn
nnn
nnn
axaxa
axaxaaxaxaxa
MMM (2.31)
where iji
ijij aaa
aa11
1)1( −= (2 ≤ i ≤ n, 2 ≤ j ≤ n+1) (2.32)
Assume neat that , and subtract the multiple of the second equation of (2.31) from the i
0)1(22 ≠a )1(
22)1(
2 / aaith equation (i=3, ⋅ ⋅ ⋅ , n). We thereby obtain the second derived system:
Chapter 2 – Page 15
MEE3017 Computer Modeling Techniques in Engineering
)2(1,
)2(3
)2(3
)2(1,3
)2(33
)2(33
)1(1,2
)1(23
)1(232
)1(22
1,11313212111
+
+
+
+
=+++
=++=+++=++++
nnnnnn
nnn
nnn
nnn
axaxa
axaxaaxaxaxaaxaxaxaxa
L
MMM
L
L
L
(2.33)
By repeating this process until the (n-1)st derived system has been constructed, we obtain
)1(1,
)1(
)2(1,3
)2(33
)2(33
)1(1,2
)1(23
)1(232
)1(22
1,11313212111
−+
−
+
+
+
=+
=++=+++=++++
nnnn
nnn
nnn
nnn
nnn
axa
axaxaaxaxaxaaxaxaxaxa
MMM
L
L
L
(2.34)
where the relation for obtaining the coefficients of the kth derived system from the coefficients of preceding system has the general form
)1()1(
)1(1)( −
−
−− −= k
kjkkk
kikk
ijk
ij aaa
aa (i = k+1, ⋅ ⋅ ⋅ , n; j = k+1, ⋅ ⋅ ⋅ , n+1) (2.35)
In Eq. (2.35), k ranges from 1 to n-1. The process is started by assigning (i = 1, ⋅ ⋅ ⋅ , n; j = 1, ⋅ ⋅ ⋅ , n+1) ijij aa =)0(
By inspection of Eq.(2.34), we see that the coefficient matrix of the (n-1)st derived system is in upper triangular form. The remaining step in solving this system is easy. The value of xn can be obtained from the final equation of Eq.(2.34)since nonsingularity of A implies that necessarily
. Specifically, 0)1( ≠−nnna
)1(
)1(1,
−
−+= n
nn
nnn
n aa
x
and the (backward) recursive formula for obtaining the values of the unknowns xk in terms of the previously calculated values xj (j > k) is
−= ∑+=
−−+−
n
kjj
kkj
knkk
kkk xaa
ax
1
)1()1(1,)1(
1 (k = n−1, ⋅ ⋅ ⋅ , 1) (2.36)
Gaussian elimination: Formula (2.36) is called Back Substitution. Process (2.26-2.35) is referred to as Forward Elimination.
Chapter 2 – Page 16
MEE3017 Computer Modeling Techniques in Engineering
The coefficients of the successive derived systems associated with Example 2.1 are given in Table 2.1.
TABLE 2.1 Derived System in Forward Elimination Original System A(0)
952744312
311
First Derived System A(1)
6412312
31
1−
Second Derived System A(2)
412312
41
1−
2.2.2 Gaussian Elimination with Pivoting Because of the effect of propagated rounding errors, native Gaussain elimination is sometimes unsatisfactory. Example 2.3 Assume that in the absence of round off error, the last two equations of the (n − 2)nd derived system are
3210
1
1
=+=+
−
−
nn
nn
xxxx
where the zero coefficients is obtained from previous calculations. The solution is
. As we know, resultants of arithmetic operations usually have round off errors: Therefore, the computed derived system of equations is actually
11 == −nn xx
321
1
1
=+=+
−
−
nn
nn
xxxxε
Since 0≠ε , in the computation of the (n − 1)st derived system the native elimination process is continued with the nonzero element ε, and the last two equations of the final derived system are
Chapter 2 – Page 17
MEE3017 Computer Modeling Techniques in Engineering
εε
ε23)21(
11
−=−
=+−
n
nn
x
xx
Back substitution applied to these equations results in the values
ε
ε
ε
nn
n
xx
x
−=
−
−=
−
1
21
23
1
For ε ≈ zero, ε2
−3 and ε2
− 1 ≈ε2− , xn ≈1 which is the correct value. But xn-1 will have
considerable error because xn, being very close to 1, will induce subtractive cancellation in numerator. The method of back substitution (2.36) implies that the values of all the other unknowns xn-2, ⋅ ⋅ ⋅ ,x1 obtained by the use of the erroneous value of xn-1 are also suspect. The difficulty in the system discussed above is not due simple to ε being small but rather to its being small relative to other coefficients in the same column. The element used for elimination is termed a )1( −k
kka pivot. The subtractive cancellation difficulty just observed can be remedied if we choose as the kth pivot element the coordinate having the largest magnitude among all , k ≤ i ≤ n, k ≤ j ≤ n. The
element a is then put in the diagonal position by interchanging rows i* and k and columns j* and k. That is, equations i* and k, and unknowns x
)1(**−kji
)1
a
**−kj
)1( −kkka
(i
j* and xk are interchanged. This operation is called pivoting for maximal size or, more simply, maximal pivoting, see Fig 2.2. We need to find the maximum of (n-k+1)2 numbers before computing the kth derived system. At the expense of some increase in round off error propagation, a popular alternative procedure is to perform partial pivoting, where the maximal element is chosen from only the kth column. That is, we choose the kth pivot element, to be any coordinate a that
maximizes
)1(*
−kki
)1( −kika , i ≥ k. Then the kth and i*th rows are interchanged to put the pivot
element in the diagonal position. Fig.2.2 illustrates the partial and maximal pivoting strategies.
Chapter 2 – Page 18
MEE3017 Computer Modeling Techniques in Engineering
)1()1(
)1()1(
)2(,1
)2(1,1
)1(2
)1(22
111
−−
−−
−−
−−−
knn
k
kkn
kkk
knk
kkk
n
n
aa
aaaa
aaaa
L
MM
L
L
MMO
L
L
nk
Search this portion Of kth column for pivot
Partial Pivoting
)1()1(
)1()1(
)2(,1
)2(1,1
)1(2
)1(22
111
−−
−−
−−
−−−
knn
knk
kkn
kkk
knk
kkk
n
n
aa
aaaa
aaaa
L
MM
L
L
MMO
L
L
Search this portion Of matrix for pivot
Fig.2.2 Partial and Maximal Pivoting.
2.3 Iterative Methods for Linear Systems 2.3.1 Jacobi Iteration Consider again (2.1?) with m = n. Assume that A is nonsingular and the rows have been exchanged, as necessary, so that the diagonal elements are nonzero. Eq. (2.1?) can then be rewritten so that the ith equation is explicit for xi:
( )
( )
( )nnnnnnnn
n
nn
nn
bxaxaxaa
x
bxaxaxaa
x
bxaxaxaa
x
−+++−=
−+++−=
−+++−=
−− 11,2211
2232312122
2
1131321211
1
1
1
1
L
M
L
L
(2.45)
Chapter 2 – Page 19
MEE3017 Computer Modeling Techniques in Engineering
Assume that an initial approximation of the solution has been given, or simply choose an arbitrary vector. Let denote this initial approximation. ( T
nxxX )0()0(1
)0( ,,L= )
)
Substitute it into the right-hand side of Eq.(2.45) and evaluate. The elements of the resulting vector give the next approximations of the unknowns. Let vector denote these approximations, and then substitute the new vector X
( TnxxX )1()1(
1)1( ,,L=
( )Tnx )2()2 ,,L
(1), into the right side of Eq.(2.45) to get a further approximation, , and so on. The general step is given by xX (
1)2( =
( )
( )
( )nk
nnnk
nk
nnn
kn
knn
kkk
knn
kkk
bxaxaxaa
x
bxaxaxaa
x
bxaxaxaa
x
−+++−=
−+++−=
−+++−=
−−+
+
+
)(11,
)(22
)(11
)1(
2)(
2)(
323)(
12122
)1(2
1)(
1)(
313)(
21211
)1(1
1
1
1
L
M
L
L
(2.46)
Scheme (2.46) is called the Jacobi Iteration Method. It can be proven that under certain conditions, for k → ∞, the sequence of vector x(k) converges to the exact solution of equations (2.1). One such condition is each diagonal element of the matrix of coefficient satisfy the condition:
∑≠=
⟩n
ijj
ijii aa1
, i = 1, … , n. (2.47)
If this condition is satisfied, then A is said to be diagonally dominant. The criteria for stopping the iteration process are usually either: 1. The number of iterations has exceeded some predetermined maximum K, or 2. The difference between successive values of all xi’s are less than some predetermined
tolerance, ε. One iteration stop ⇒ n division, n2 multiplication and n2 additions (or subtractions.) 2.3.2 Gauss Seidal Iteration Consider again the recursive Jacobi iteration scheme (2.46). Observe that in calculating the "new” value of , the previous value of is used on the right-hand side although the “new” value, is already known.
)1(2
+kx )(1
kx)1(
1+kx
Chapter 2 – Page 20
MEE3017 Computer Modeling Techniques in Engineering
Similarly, for obtaining the new value , the “odd” values and are used, although the new, and presumably more accurate values and of these variables are already available. A modification typically (but not always) giving faster convergence can be devised if in the calculation of (2 ≤ i ≤ n), the updated new values , … , are used in phase of the earlier values , … , in (2.46). This modification results in the Gauss Seidal iteration method, which is defined by the recursive scheme.
)1(3
+kx )(1
kx
)(1
kx
)1(2
+kx)1
)(1
kix −
)1(1
+kx (2
+kx
)1( +kix
)1(1
+kx )1(1+
−k
ix
( )
( )
( )nk
nnnk
nk
nnn
kn
knn
kkk
knn
kkk
bxaxaxaa
x
bxaxaxaa
x
bxaxaxaa
x
−+++−=
−+++−=
−+++−=
+−−
+++
++
+
)1(11,
)1(22
)1(11
)1(
2)(
2)(
323)1(
12122
)1(2
1)(
1)(
313)(
21211
)1(1
1
1
1
L
M
L
L
(2.48)
Iterative methods are particularly popular for solution of band systems arising in numerical method for partial differential equations. For such equations, in many cases the width k is relatively wide-often proportional to
- but the band itself has relatively few nonzero entries. Let p denote a bound to the effort required by elimination for solution of banded systems is proportional to k
2/1n2n, and
for iterative methods, proportional to 2pnM, where M is the number of iterations necessary for acceptable accuracy. Thus iterative methods are preferable if
2pnM < k2n, that is
pn
pkM ~2
2
<
Unfortunately, it is typically difficult to asses M until computations have already begun.
Chapter 2 – Page 21
MEE3017 Computer Modeling Techniques in Engineering
Table 2.x Solution of Linear Equations. Computation Methods Matrix Condition Characteristics
Direct Methods
*Gaussian Elimination *Gauss-Jordan Elimination *Cholesky’s Method *Lu Decomposition Method
Square Matrix Square Matrix Symmetric Matrix Symmetric Matrix
For linear systems of small or moderate size, either Gaussian Elimination or Lu Decomposition is effective and efficient. (Size < zero)
Iterative Methods
*Jacobi Iteration *Gauss-Seidal Method *Successive over- Relaxation (SOR Method)
Diagonally Dominant.
∑≠=
>n
ijj
ijii aa1
,
i = 1, … , n.
For large and high-order linear equations, for example, in solving differential equation, Iterative Methods are attractive. Fast convergence.
Chapter 2 – Page 22
MEE3017 Computer Modeling Techniques in Engineering
EXAMPLE 2 **********(will be deleted)*************** In the rectangular region shown in Figure 2.2(a), the electric potential is zero on the boundaries. The charge distribution, however, is uniform and given by
286 m×.02ε=vp
Solve Poisson’s equation to determine the potential distribution in the rectangular region. Solution To determine the potential distribution in the rectangular region, we use Poisson’s equation.
20
2
2
2
22 −=
ερ
−=∂
Φ∂+
∂Φ∂
=Φ∇ v
yx
with zero potential Φ on the boundaries. 0=
Figure 2.2 Geometry of the rectangular region and the mesh. 286 m× mh 2= By establishing the rectangular grid shown in Figure 2.2(b), we realize that we have six nodes and, hence, six unknown potentials for which to solve. Replacing ∇ by its finite difference representation, we obtain
Φ2
02)4(1,1,1,,1,12 =+Φ−Φ+Φ+Φ+Φ −+−+ jijijijijih
Chapter 2 – Page 23
MEE3017 Computer Modeling Techniques in Engineering
It should be noted that although the although the mesh size was not explicitly used in solving Laplace’s equation in the previous example, h is included as a part of the matrix formation in solving Poisson’s equation. In SI system of units, h should be in meters. By applying the preceding difference equation at the various nodes in Figure 2.2(b), we obtain the following matrix equation:
−−
−−
−−
411000140100104110011401001041000114
=
ΦΦΦΦΦΦ
6
5
4
3
2
1
−−−−−−
888888
Instead of solving the resulting six equations, we may note some symmetry considerations in Figure 2.2(b). It is clear that
6521 Φ=Φ=Φ=Φ and that 43 Φ=Φ Taking these symmetry considerations into account, the number of equations reduces to two, and we obtain the following solution:
,56.41 =Φ Φ 72.53 = To improve the accuracy of the potential distribution, finer mesh such as the one shown in Figure 2.2 is required. Because of the large number of nodes in this case, symmetry should be used, and a solution for only one-quarter of the rectangular geometry is desired. The application of the difference equation at nodes 1, 2, 4, and 5 should proceed routinely, whereas special care should be exercised at the boundary nodes 3, 6, 9, 10, and 11, and also at the corner node 12. For example, applying the difference equation at node 6 yields
02)4(165932 =+Φ−Φ+Φ+Φ+Φ ah
Or
02)42(165932 =+Φ−Φ+Φ+Φ
h
Chapter 2 – Page 24
MEE3017 Computer Modeling Techniques in Engineering
In equation )4(10432122
2
2
22 Φ−Φ+Φ+Φ+Φ=
∂Φ∂
+∂
Φ∂=Φ
hyx
5Φ
∇ , symmetry was used to
complete the five-point star difference equation. Specifically the potential at node a to the right of 6 was taken equal to . Similarly at the corner node 12, we obtain
02)4(1129112 =+Φ−Φ+Φ+Φ+Φ cbh
Because of symmetry, and bΦ=Φ11 cΦ=Φ9 , hence,
02)422(1129112 =+Φ−Φ+Φ
h
Fig. 2.3 The finer mesh solution and symmetry consideration of example 2. The matrix equation for the twelve nodes shown in Figure 2.3 is then
Chapter 2 – Page 25
MEE3017 Computer Modeling Techniques in Engineering
−−
−−
−−
−−
−−
−−
4221412
1421421
114111141
142111411
1141142
1141114
ΦΦΦΦΦΦΦΦΦΦΦΦ
12
11
10
9
8
7
6
5
4
3
2
1
=
−−−−−−−−−−−−
222222222222
The “2” coefficient in the coefficient matrix (to the left) of equation appears whenever symmetry consideration is used at boundary and corner nodes. It should be noted that the
coefficient matrix in equation is the same for both Laplace’s and Poissons equations. The constant vector on the right-hand side of equation, however, depends on the charge distribution within and the potential at the boundaries of the region of interest. Furthermore, if instead of a uniform charge distribution we have a given charge distribution
1212×
),( yxvρ , the constants vector on the right-hand side of equation should reflect the value of ),( yxvρ calculated at each node. Solution of equation gives
65.696.582.334.669.566.332.579.412.335.305.3,04.2
121110
987
654
321
=Φ=Φ=Φ=Φ=Φ=Φ=Φ=Φ=Φ=Φ=Φ=Φ
Chapter 2 – Page 26