me 361-system of equations-26 sept 2013

8/14/2019 ME 361-System of Equations-26 Sept 2013

http://slidepdf.com/reader/full/me-361-system-of-equations-26-sept-2013 1/40

ME – 361 NUMERICAL METHODS FOR ENGINEERS

Prof. Dr. Faruk Arınç Fall 2013

SYSTEM OF EQUATIONS

1 1 2 3 11 1 12 2 13 3 1 1

2 1 2 3 21 1 22 2 23 3 2 2

3 1 2 3

f (x , x , x , ...) 0 => a x + a x + a x + .... + a x = c

f (x , x , x , ...) 0 => a x + a x + a x + .... + a x = c

f (x , x , x , ...) 0 =

n n

n n

31 1 32 2 33 3 3 3

1 2 3 1 1 2 2 3 3

> a x + a x + a x + .... + a x = c

....

f (x , x , x , ...) 0 => a x + a x + a x + .... + a x = c

n n

n n n n nn n n

Set of n equations, linear in the variables, xn’s :

Set of n equations, nonlinear in the variables, x, y and z :

2

21 1 1 1 1 1

d z

2 2 2 2 2

3

f (x, y, z) 0 => a x + b ln y + c cos d z = e

1f (x, y, z) 0 => a + b sin y + c e = e

x

f (x, y, z) 0 => .etc





MATRICES

A Matrix is a rectangular array of elements in m rows and n columns, in general.

1 1 1 2 1 n

2 1 2 2 2 n

i j

m 1 m 2 m n

a a . . a

a a . . a A a . . . . .

a a . . a

where i is the row and j is the column.





Square Matrix : m = n

Determinant of a matrix : A = a i j

Transpose of a matrix : AT = a j i

Matrix Operations : Addition A + B = a i j + b i j

Product with a scalar k A = k a i j

Product of two matrices

A is m by n

B is n by p

A . B = C

C matrix is m by p

A must be conformable

with B

Column and Row Matrix Vector

Matrix Definitions and Properties





Unit(y) MatrixI has 1's in the

principal diagonal

If A is 1 by n

B is n by 1

W is n by n

A W B = c where c is a scalar

If c = 0, A and B vectors are mutually orthogonal

with respect to the weight matrix, W.

If A is 1 by n

B is n by 1

B A is called a tensor

1 . . . 0 0

. . . . . .

0 . . . 1 0

0 . . . 0 1

I

ba . . . ba

. . . . .

. . . . .

ba . . . ba

a . . . a

b

.

.

b

AB

nnn1

1n11

n1

n

1





Inverse Matrix : A - 1

(If A is square)

A A- 1 = A- 1 A = I

Not all square matrices have inverses.

Singular Matrix A = 0

Given a square matrix A

If A X = C X = A- 1 C when A 0

A

A

A

AAdj AAof Inverse

T

ji1-

Cofactor of a i , j = A i j = (-1)i + j

Adjoint of A = Adj A = Ai , jT





SYSTEM OF LINEAR EQUATIONS

m

2

1

n

2

1

nm2m1m

n22212

n12111

c

.

c

c

x

.

x

x

a . . . a a

. . . . . .

a . . . a a

a . . . a a

The given equations are linear in the independent variables, xn’s:

a 11 x 1 + a 12 x 2 + ….. + a 1n x n = c 1

a 21 x 1 + a 22 x 2 + ….. + a 2n x n = c 2

…………………………………….

a m1 x 1 + a m2 x 2 + ….. + a mn x n = c m

In Matrix form:





Assume that r number of equations are linearly independent. Then, the number of

residual equations is (m - r).

Linearly independent equation: It cannot be formed by linear combination of the

others.

A number of possibilities exists when m > r

If the equations contradict one another, they are said to be inconsistent or

incompatible and no solution exists

If r = n, a unique solution exists

If r < n, a family of solutions exists. (n - r) is called the defect of the system.

Note that if r > n, the equations must be inconsistent since r number of

equations are linearly independent.





GAUSS(IAN) ELIMINATION

nn2n1n

n22212

n12111

a a a

. . . . . .

. . . . . .

a . . . a a

a . . . a a

*

n2

*

n2

*

22

n12111

a . . . 0 0

. . . . . .

. . . . 0 .

a . . . a 0

a . . . a a

It is a method of systematically re-

ordering and linearly combining a

given set of linear equations to

reduce a “n by n” (n x n) matrix of

coefficients

to an upper-triangular matrix

and then back substitute to determine the unknown variables.





1) If , re-order the equations

2) Multiply (1) by and (2) by , substract (1) from (2), replace (2)

Re-name coefficients

Given a set of m equations in n

unkowns:

0 0 0 0

1 1 1 1 2 2 1 n n 1

0 0 0 0

m 1 1 m 2 2 m n n m

a x a x ... a x c (1)

. . . . . . . .

a x a x ... a x c (m)

Elimination Procedure:

0a011

a0

12

0

11a

ca-ca...xaa-aa00

112

0

2

0

112

0

21

0

12

0

11

0

22

Repeat steps 1 and 2 for all the remaining rows and obtain:





. . . . . .

)(3' cxa...xa 0

)(2' cxa...xa 0

(1) cxa...xaxa

1

3n

1

n32

1

23

1

2n

1

n22

1

22

0

1n

0

n12

0

211

0

11

Steps 1 through 3 are repeated to the extent possible.

)(m' cxaxa

. . . . .

)(2' cxa...xa

(1) cxa...xaxa

*

mn

*

nmm

*

mm

1

2n

1

n22

1

22

01n0n120211011

i) If n > m, the equations may look like this:

The final result may assume a number of different forms:





ii) If n = m, the equations may look like this:

*

nn

*

nn

*

nn

*

nn1-n

*

1-n1-n

1

2n

1

n22

1

22

0

1n

0

n12

0

211

0

11

cxa

1)'-(n cxaxa

. . . . .

)(2' cxa...xa (1) cxa...xaxa

This means there is a unique solution.

This means that there is a family of solutions with m variables expressed as

linear combination of remaining (n - m) variables. If the original equations were not

all linearly dependent, some of the defect equations may be identical.

Note that none of the main diagonal elements must be zero. Otherwise, re-order

equations.





iii) If the process terminates, i.e.

)(m' c0

. .

)1'(k c0

)(k' cxa...

. . . . .

)(2' cxa...xa

(1) cxa...xaxa

*

m

*

1k

*

kn

*

nk

1

2n

1

n22

1

22

0

1n

0

n12

0

211

0

11

there are a number of possibilities:

I. If ck+1 = ck+2 = … = cm = 0 and n > k, there is a family of solutions

II. If ck+1 = ck+2 = … = cm = 0 and n = k, there is a unique solution

III. If any one of ck+1 , …, cm ≠ 0, the equations are inconsistent or incompatible.





EXAMPLE on GAUSS(IAN) ELIMINATION

Given set of linear equations in augmented matrix form:

3- 3- 2 3 1

5 1 2 1 1

4 1- 1 1 2

1- 2- 1- 2 1

(1)

(2)(3)

(4)

2- 1- 3 1 0

6 3 3 1- 0

6 3 3 3- 0

1- 2- 1- 2 1

(1)

(2') = 1 x (2) - 2 x (1)

(3') = 1 x (3) – 1 x (1)

(4') = 1 x (4) – 1 x (1)





1 2 -1 -2 -1

0 -3 3 3 6

0 0 -6 -6 -12

0 0 0 -72 -144

1 2 -1 -2 -1

0 -3 3 3 6

0 0 -6 -6 -12

0 0 -12 0 0

(1)

(2')

(3'') = - 3 x (3') + 1 x (2')

(4'') = - 3 x (4') - 1 x (2')

(1)

(2')

(3'')

(4''') = -12 x (3'') + 6 x (4'')

Back substitution yields:

x 4 = 2 x 3 = 0 x 2 = 0 x 1 = 3

Row-echelon

form






Given set of linear equations:

2 x1 + 3 x2 - x3 = 4

x1 - 2 x2 + x3 = 6

- x1 - 1 2 x2 + 5 x3 = 10

Augmented Matrix:

2 3 -1 4

1 -2 1 6

-1 -12 5 10

2 3 -1 40 7 -3 -8

0 0 0 0

Linear Row Operations:Family of Solution:

2 x1 + 3 x2 – x3 = 4 and 7 x2 – 3 x3 = - 8

Set x2 = C

3 1

7 8 20 1x = C + and x = - C

3 3 6 3






Given set of linear equations:

x1 + x2 + 3 x3 = 1

x1 + 2 x2 + 5 x3 = 2

3 x1 + 1 x2 + 5 x3 = - 1

Augmented Matrix:

1 1 3 1

1 2 5 2

3 1 5 -1

1 1 3 10 1 2 1

0 0 0 -4

Linear Row Operations:

Inconsistency or Incompatibility

No Solution





EXAMPLE on GAUSS-JORDAN REDUCTION

Given set of linear equations in augmented matrix form:

1 2 -1 -2 -1

2 1 1 -1 4

1 1 2 1 5

1 3 2 -3 -3

(1)

(2)

(3)(4)

1 2 -1 -2 -1

0 1 -1 -1 -20 -1 3 3 6

0 1 3 -1 -2

(1)

(2') = [(2) - 2 x (1)] / - 3(3') = (3) - (1)

(4') = (4) - (1)





1 0 0 0 30 1 0 0 0

0 0 1 0 0

0 0 0 1 2

1 0 0 -1 1

0 1 0 0 0

0 0 1 1 2

0 0 0 1 2

1 0 1 0 3

0 1 -1 -1 -2

0 0 1 1 2

0 0 1 0 0

(1') = - 2 x (2') + (1)

(2')

(3'') = [(3') + (2')] / 2

(4'') = [(4') + -1 x (2')] / 4

(1'') = -1 x (3'') + (1')(2'') = (3'') + (2')

(3'')

(4''') = [-1 x (3'') + (4'')] x -1

(1''') = (4''') + (1'')

(2'')

(3''') = -1 x (4''') + (3'')

(4''')





SCALED PARTIAL (ROW) PIVOTING

RULE:

In order to select the pivot row and hence the pivot element at every operation j

(equal to 1 at the beginning), form the ratios

i j

i j

a for each row, i

max a

The row i which has the maximum ratio is the pivot row, and the corresponding

element is the pivot element.

EXAMPLE:

The augmented matrix and the first set of above ratios are given below. The first

question is which one of the rows is to be the first pivot row.





Augmented Matrix Elimination step

1 2 -1 -2 -12 1 1 -1 4

1 1 2 1 5

1 3 2 -3 -3

1 / 22 / 2 Max

1 / 2

1 / 3

The second equation has the largest ratio. So, it is to be the first pivot row.

Interchange first and second rows so that the second equation is the first row (or

the first pivot):

Given Augmented Matrix Pivoting Ratios

2 1 1 -1 4

1 2 -1 -2 -1

1 1 2 1 5

1 3 2 -3 -3

2 1 1 - 1 4

0 1.5 -1.5 -1.5 -3

0 0.5 1.5 1.5 3

0 2.5 1.5 -2.5 -5





The next step is to determine the second pivot. Find the ratios again:

Augmented Matrix Pivoting Ratios

2 1 1 - 1 4

0 1.5 -1.5 -1.5 -3

0 0.5 1.5 1.5 3

0 2.5 1.5 -2.5 -5

-

1.5 / 1.5

0.5 / 1.5

2.5 / 2.5

The second and the fourth rows have the same maximum ratio. Choose any one of

them as the pivot. For instance, if the fourth row is chosen, interchange second and

fourth rows:

2 1 1 - 1 4

0 2.5 1.5 -2.5 -5

0 0.5 1.5 1.5 3

0 1.5 -1.5 -1.5 -3

2 1 1 - 1 4

0 2.5 1.5 -2.5 -5

0 0 etc

0 0

Augmented Matrix Elimination Step





GAUSS-SEIDEL METHOD

This is similar to fixed-point

iteration method of root finding.

Given a set of linear equations:

1 1 1 1 2 2 1 n n 1

2 1 1 2 2 2 2 n n 2

n 1 1 n 2 2 n n n n

a x a x ... a x c

a x a x ... a x c

. . . . . . .

a x a x ... a x c

Re-write the equations in the form:

m+1 m m

1 1 1 2 2 1 n n 1 2 3 n

1 1

m+1 m+1 m2 2 2 1 1 2 n n 2 1 3 n

2 2

m+1 m+1

n n n 1 1

n n

1x (c - a x - ... - a x ) g (x , x , ..., x )

a

1x (c - a x - ... - a x ) g (x , x , ..., x )a

. . . . .

1x (c - a x - ... -

a

m+1

n n-1 n-1 n 1 2 n-1a x ) g (x , x , ..., x )





Note that the coefficients of the diagonal elements must be non-zero. Otherwise,

the set of equations must be re-ordered (re-cast) so that this is satisfied.

Convergence is ensured when this is satisfied for every row (equation). There

may or may not be convergence if this is not satisfied for one or more rows.

Start with an initial set of guesses for the unknowns, x1, x2, … and carry out an

iterative procedure similar to that of fixed-point iteration.

Use the new values of the unknowns as soon as they are available at every

iteration step.

The criterion for convergence is: a a for each row ii i i ji j





Gauss-Seidel Method

Example

1 2 3

1 2 3

1 2 3

x x 3 x 1

x 2 x 5 x 2

3 x x x 1

Re-write the equations in the form

1 2 3

2 1 3

3 1 2

x 1 - x - 3 x

1 5x 1 - x - x

2 2

x 1 - 3 x - x

Show that convergence criterion is not satisfied in any of the rows. Hence,

the iterations will diverge.

Convergence Criterion

1 < 1 + 32 < 1 + 5

1 < 3 + 1





x3-x-1x 03

02

11 x

25 -x

21 -1x 0

311

12 x-x3-1x

1

2

1

1

1

3

10-0-1x1

1

2

1 (0)

2

5 -(1)

2

1 -1x1

2 2

5 -

2

1 -(1)3-1x1

3

8.0x2

1 3.25x

2

2 26.25-x

2

3

Hence, the iterations are diverging

Re-cast the equations such that the criterion is satisfied for at least most of the rows.





Gauss-Seidel Method

Example with re-casting 1x3xx

2xx2x5

1xxx3

321

321

321

Re-write the equations in the form

3/)x-x-(1x

2/)x-x5-(2x3/)x-x-(1x

213

312

321

Show that convergence criterion is not satisfied in the second row. However,

the iterations will converge.

Convergence Criterion

3 > 1 + 12 < 5 + 1

3 > 1 + 1





Gauss-Seidel Iterations

n x1 x2 x3

0 1.00000 1.00000 1.00000

1 -0.33333 1.33333 0.00000

2 -0.11111 1.27778 -0.05556

3 -0.07407 1.21296 -0.04630

4 -0.05556 1.16204 -0.03549

5 -0.04218 1.12320 -0.02701

6 -0.03206 1.09366 -0.02053

7 -0.02438 1.07121 -0.01561

8 -0.01853 1.05414 -0.01187

9 -0.01409 1.04116 -0.00902





Under Relaxation and Over Relaxation with Gauss-Seidel

m+1 * m m

1 1 1 2 2 1 n n 1 2 3 n

1 1

m+1 * m+1 m

2 2 2 1 2 2 n n 2 1 3 n

2 2

m+1 * m+1

n n n 1 1

n n

1(x ) (c - a x - ... - a x ) g (x , x , ..., x )

a

1(x ) (c - a x - ... - a x ) g (x , x , ..., x )

a

. . . . .1

(x ) (c - a xa

m+1

n n-1 n-1 n 1 2 n-1- ... - a x ) g (x , x , ..., x )

m

n

*1m

n

1m

n x)-(1x x Define Relaxation Factor, λ

λ = 1 No relaxation

λ < 1 Under relaxation : To change a diverging case to a converging one

λ > 1 Over relaxation : To increase the rate of convergence





Gauss-Seidel Iterations with over relaxation

λ = 1.20

n x1 x2 x3

0 1.00000 1.00000 1.00000

1 -0.60000 2.20000 -0.44000

2 -0.18400 1.57600 -0.06880

3 -0.16608 1.42432 -0.08954

4 -0.10070 1.27095 -0.05019

5 -0.06816 1.18042 -0.03486

6 -0.04459 1.11860 -0.022637 -0.02947 1.07827 -0.01499

8 -0.01942 1.05159 -0.00987

9 -0.01280 1.03402 -0.00651





Given : A parabola f 1(x , y) = x 2 - 2 x - y + 0.5 = 0

and an ellipse f 2 (x , y) = x 2 + 4 y 2 - 4 = 0

NON-LINEAR SYSTEM OF EQUATIONS

Solution with fixed-point iterations:

y),x(g 2

0.5y-x x 1

2

y),x(g 8

4y8y4-x- y 2

22

Convert to





-1

-0.5

0

0.5

1

1.5

2

2.5

3

3.5

-1 -0.5 0 0.5 1 1.5 2

sqrt(4-x*x)/2-sqrt(4-x*x)/2

x*x-2*x+0.5





Start with (0,1)

n x n y n

0 0 1

1 - 0.25 1

2 - 0.21875 0.99219

3 - 0.22217 0.99399

4 - 0.22231 0.99381

5 - 0.22219 0.99380

6 - 0.22222 0.99381

7 - 0.22221 0.99381





Start with (2,0)

n x n y n

0 2 0

1 2.25 0

2 2.78125 - 0.13228

3 4.18408 - 0.60855

4 9.30755 - 2.48204

5 44.80623 - 15.89109

6 1011.995 - 392.6042

7 512263.2 - 205477.8





Theorem on convergence in fixed-point iteration:

Let the functions, g1(x,y) and g2(x,y) and their first partial derivatives be

continuous on a region that contains the fixed point. Assume that the starting point

is chosen sufficiently close to the fixed point and


1 2g (x,y) g (x,y) < 1 andx x

1 2g (x,y) g (x,y) < 1y y

then, the iterations converge to the fixed point.






Solution with Newton-Raphson:

Given : A catenary f 1(x,y) = y – 0.5 (ex / 2 + e- x / 2) = 0

and an ellipse f 2(x,y) = 9 x2 + 25 y2 – 225 = 0

Use Newton’s method to determine the point of intersection of the curves that

resides in the first quadrant of the coordinate system. Starting with the initialguess, (x1,y1) = (2.5,2.0),





0

1

2

3

4

5

6

7

-4 -2 0 2 4

sqrt((225-9*x*x)/25)

0.5*(exp(x/2)+exp(-x/2))





1 1 1 1

1 1 1 1

1 11 1 1

x ,y x ,y

2 22 1 1

x ,y x ,y

f f x + y = - f (x ,y )

x y

f f x + y = - f (x ,y )x y

1 1 1 1

1 1 1 1

1 1

x ,y x ,y 1 1 1 1 1 1

1 2

2 1 1 2 1 12 2

x ,y x ,y

f f

x y - f (x ,y ) - f (x ,y )x x = or J(f ,f ) =

y - f (x ,y ) y - f (x ,y )f f

x y

or

Expand both non-linear functions of two variables in Taylor series around a chosen

point (x0,y0) (as close to the real solution as possible).





x/2 -x/21 1f f 1

= - e - e = 1x 4 y

2 2f f

= 18 x = 50 yx y

1 1 1 1

1 1 1 1

2 1

1 1 1 2 1 1x ,y x ,y

1 2

1 22 1 1 1 1 1

x ,y x ,y

1 2

f f

- f (x ,y ) + f (x ,y )y yx =

det J(f ,f )

f f - f (x ,y ) + f (x ,y )

x xy =det J(f ,f )

One may use Cramer’s rule to solve the two equations:

where

1 1x /2 x /2

1 2 1 1

1det J(f ,f ) = - e - e 50 y - 18 x

4





xn yn f 1(x,y) f 2(x,y) df 1 /dx df 1 /dy df 2 /dx df 2 /dy det(J) Δx Δy

2.5000 2.0000 0.11157 -68.7500 -0.8009 1.0000 45.0000 100.0000 -125.095 0.6387 0.4000

3.1387 2.4000 -0.10588 7.67331 -1.1488 1.0000 56.4978 120.0026 -194.366 -0.1048 -0.0145

3.0339 2.3854 -0.00338 0.10425 -1.0847 1.0000 54.6105 119.2737 -183.991 -0.0027 0.0003

3.0311 2.3858 -0.00000 0.00007 -1.0830 1.0000 54.5608 119.2932 -183.766 -0.0000 0.0000

3.0311 2.3858 -1.0E-12 3.3E-11 -1.0830 1.0000 54.5607 119.2932 -183.766 -8.E-13 1.E-13

me 361-system of equations-26 sept 2013

Documents