matrix-131010034223-phpapp01

7/27/2019 matrix-131010034223-phpapp01

1/123

MATRIX METHODS

SYSTEMS OF LINEAR EQUATIONS

ENGR 351

Numerical Methods for Engineers

Southern Illinois University Carbondale

College of Engineering

Dr. L.R. Chevalier

Dr. B.A. DeVantier

7/27/2019 matrix-131010034223-phpapp01

2/123

Copyright 2003 by Lizette R. Chevalier and Bruce A. DeVantier

Permission is granted to students at Southern Illinois University at Carbondaleto make one copy of this material for use in the class ENGR 351, Numerical

Methods for Engineers. No other permission is granted.

All other rights are reserved. No part of this publication may be reproduced,stored in a retrieval system, or transmitted, in any form or by any means,electronic, mechanical, photocopying, recording, or otherwise, withoutthe prior written permission of the copyright owner.

7/27/2019 matrix-131010034223-phpapp01

3/123

Systems of Linear AlgebraicEquationsSpecific Study Objectives

Understand the graphic interpretationof ill-conditioned systems and how itrelates to the determinant

Be familiar with terminology: forwardelimination, back substitution, pivot

equations and pivot coefficient

7/27/2019 matrix-131010034223-phpapp01

4/123

Apply matrix inversion to evaluate stimulus-response computations in engineering

Understand why the Gauss-Seidel method isparticularly well-suited for large sparsesystems of equations

Know how to assess diagonal dominance of a

system of equations and how it relates towhether the system can be solved with theGauss-Seidel method

Specific Study Objectives

7/27/2019 matrix-131010034223-phpapp01

5/123

Understand the rationale behindrelaxation and how to apply this

technique

Specific Study Objectives

7/27/2019 matrix-131010034223-phpapp01

6/123

How to represent a system of linearequations as a matrix

[A]{x} = {c}

where {x} and {c} are both column vectors

7/27/2019 matrix-131010034223-phpapp01

7/123

44.0

67.0

01.0

5.03.01.0

9.115.0

152.03.0

}{}{

44.05.03.01.0

67.09.15.0

01.052.03.0

3

2

1

321

321

321

x

x

x

CXA

xxx

xxx

xxx

How to represent a system of linearequations as a matrix

7/27/2019 matrix-131010034223-phpapp01

8/123

Practical application

Consider a problem in structuralengineering

Find the forces and reactions associatedwith a statically determinant truss

hinge: transmits bothvertical and horizontalforces at the surface

roller: transmitsvertical forces

30

90

60

7/27/2019 matrix-131010034223-phpapp01

9/123

1000 kg

30

90

60

F1

H2

V2V3

2

3

1

FREE BODY DIAGRAM F

F

H

v

0

0

F2

F3

7/27/2019 matrix-131010034223-phpapp01

10/123

Node 1 F1,V

F1,H

F3F1

6030

F F F F

F F F F

F F

F F

H H

V V

0 30 60

0 30 60

30 60 0

30 60 1000

1 3 1

1 3 1

1 3

1 3

cos cos

sin sin

cos cos

sin sin

,

,

7/27/2019 matrix-131010034223-phpapp01

11/123

F H F F

F V F

H

V

0 30

0 30

2 2 1

2 1

cos

sin

Node 2

F2

F1

30

H2V2

7/27/2019 matrix-131010034223-phpapp01

12/123

F F F

F F V

H

V

0 60

0 60

3 2

3 3

cos

sin

Node 3

F2

F3

60

V3

7/27/2019 matrix-131010034223-phpapp01

13/123

060sin

060cos

030sin

030cos

100060sin30sin

060cos30cos

33

23

12

122

31

31

VF

FF

FV

FFH

FF

FF

SIX EQUATIONSSIX UNKNOWNS

7/27/2019 matrix-131010034223-phpapp01

14/123

F1 F2 F3 H2 V2 V3

1

2

3

4

5

6

-cos30 0 cos60 0 0 0

-sin30 0 -sin60 0 0 0

cos30 1 0 1 0 0

sin30 0 0 0 1 0

0 -1 -cos60 0 0 0

0 0 sin60 0 0 1

0

-1000

0

0

0

0

Do some book keeping

7/27/2019 matrix-131010034223-phpapp01

15/123

This is the basis for your matrices and the equation[A]{x}={c}

0 866 0 0 5 0 0 0

0 5 0 0 866 0 0 0

0 866 1 0 1 0 0

0 5 0 0 0 1 0

0 1 0 5 0 0 0

0 0 0 866 0 0 1

0

1000

0

0

0

0

1

2

3

2

2

3

. .

. .

.

.

.

.

F

F

F

H

V

V

7/27/2019 matrix-131010034223-phpapp01

16/123

System of Linear Equations

We have focused our last lectures onfinding a value of xthat satisfied a

single equation f(x) = 0

Now we will deal with the case of

determining the values of x1, x2, .....xn,that simultaneously satisfy a set ofequations

7/27/2019 matrix-131010034223-phpapp01

17/123

System of Linear Equations

Simultaneous equations

f1(x1, x2, .....xn) = 0

f2(x1, x2, .....xn) = 0

.............

fn(x1, x2, .....xn) = 0

Methods will be for linear equations

a11x1+ a12x2+...... a1nxn=c1 a21x1+ a22x2+...... a2nxn=c2 ..........

an1x1+ an2x2+...... annxn=cn

7/27/2019 matrix-131010034223-phpapp01

18/123

Mathematical BackgroundMatrix Notation

a horizontal set of elements is called a row

a vertical set is called a column first subscript refers to the row number

second subscript refers to column number

A

a a a a

a a a a

a a a a

n

n

m m m mn

11 12 13 1

21 22 23 2

1 2 3

...

...

. . . .

...

7/27/2019 matrix-131010034223-phpapp01

19/123

mnmmm

n

n

aaaa

aaaa

aaaa

A

...

....

...

...

321

2232221

1131211

This matrix has mrows an ncolumn.

It has the dimensions mby n(m x n)

7/27/2019 matrix-131010034223-phpapp01

20/123

mnmmm

n

n

aaaa

aaaa

aaaa

A

.. .

....

...

.. .

321

2232221

1131211

This matrix has mrows and ncolumn.

It has the dimensions mby n(m x n)

note

subscript

7/27/2019 matrix-131010034223-phpapp01

21/123

A

a a a a

a a a a

a a a a

n

n

m m m mn

11 12 13 1

21 22 23 2

1 2 3

...

...

. . . ....

row 2

column 3Note the consistentscheme with subscriptsdenoting row,column

7/27/2019 matrix-131010034223-phpapp01

22/123

Row vector: m=1

Column vector: n=1 Square matrix: m = n

B b b bn 1 2 .......

C

c

c

cm

1

2

.

. A

a a a

a a aa a a

11 12 13

21 22 23

31 32 33

Types of Matrices

7/27/2019 matrix-131010034223-phpapp01

23/123

Symmetric matrix

Diagonal matrix

Identity matrix

Inverse of a matrix

Transpose of a matrix

Upper triangular matrix

Lower triangular matrix

Banded matrix

Definitions

7/27/2019 matrix-131010034223-phpapp01

24/123

Symmetric Matrix

aij= aji for all is andjs

A

5 1 21 3 7

2 7 8Does a23= a32 ?

Yes. Check the other elementson your own.

7/27/2019 matrix-131010034223-phpapp01

25/123

Diagonal Matrix

A square matrix where all elementsoff the main diagonal are zero

A

a

a

a

a

11

22

33

44

0 0 0

0 0 0

0 0 0

0 0 0

7/27/2019 matrix-131010034223-phpapp01

26/123

Identity Matrix

A diagonal matrix where all elementson the main diagonal are equal to 1

A

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

The symbol [I] is used to denote the identify matrix.

7/27/2019 matrix-131010034223-phpapp01

27/123

Inverse of [A]

IAAAA 11

7/27/2019 matrix-131010034223-phpapp01

28/123

Transpose of [A]

A

a a a

a a a

a a a

t

m

m

n n mn

11 21 1

12 22 2

1 2

. . .

. . .

. . . . . .

. . . . . .

. . . . . .

. . .

7/27/2019 matrix-131010034223-phpapp01

29/123

Upper Triangle Matrix

Elements below the main diagonalare zero

Aa a a

a a

a

11 12 13

22 23

33

0

0 0

7/27/2019 matrix-131010034223-phpapp01

30/123

Lower Triangular Matrix

All elements above the main diagonalare zero

A

5 0 0

1 3 0

2 7 8

7/27/2019 matrix-131010034223-phpapp01

31/123

Banded Matrix

All elements are zero with theexception of a band centered on the

main diagonal

A

a a

a a a

a a a

a a

11 12

21 22 23

32 33 34

43 44

0 0

0

0

0 0

7/27/2019 matrix-131010034223-phpapp01

32/123

Matrix Operating Rules

Addition/subtraction

add/subtract corresponding terms

aij+ bij= cijAddition/subtraction are commutative

[A] + [B] = [B] + [A]

Addition/subtraction are associative [A] + ([B]+[C]) = ([A] +[B]) + [C]

7/27/2019 matrix-131010034223-phpapp01

33/123


Multiplication of a matrix [A] by a scalarg is obtained by multiplying everyelement of [A] by g

B g A

ga ga gaga ga ga

ga ga ga

n

n

m m mn

11 12 1

21 22 2

1 2

. . .

. . .

. . . . . .

. . . . . .

. . . . . .

. . .

7/27/2019 matrix-131010034223-phpapp01

34/123


The product of two matrices is represented as[C] = [A][B]

n = column dimensions of [A]

n = row dimensions of [B]

c a bij ik kjk

N

1

Si l t h k h th

7/27/2019 matrix-131010034223-phpapp01

35/123

[A] mx n [B] nx k = [C] mx k

interior dimensions

must be equal

exterior dimensions conform to dimension of resulting matrix

Simple way to check whethermatrix multiplication is possible

7/27/2019 matrix-131010034223-phpapp01

36/123

Recall the equation presented formatrix multiplication

The product of two matrices is represented as[C] = [A][B]

n = column dimensions of [A]

n = row dimensions of [B]

c a bij ik kjk

N

1

7/27/2019 matrix-131010034223-phpapp01

37/123

Example

Determine [C] given [A][B] = [C]

203

123

142

320241

231

B

A

7/27/2019 matrix-131010034223-phpapp01

38/123

Matrix multiplication

If the dimensions are suitable, matrixmultiplication is associative ([A][B])[C] = [A]([B][C])

If the dimensions are suitable, matrixmultiplication is distributive ([A] + [B])[C] = [A][C] + [B][C]

Multiplication is generally notcommutative [A][B] is not equal to [B][A]

7/27/2019 matrix-131010034223-phpapp01

39/123

Determinants

Denoted as det A or lAl

for a 2 x 2 matrix

bcaddcba

bcaddcba

7/27/2019 matrix-131010034223-phpapp01

40/123

Determinants

254

329

132

For a 3 x 3

254

329

132

254

329

132

+ - +

7/27/2019 matrix-131010034223-phpapp01

41/123

516

234

971

Problem

Determine the determinant of the matrix.

7/27/2019 matrix-131010034223-phpapp01

42/123

Properties of Determinants

det A = det AT

If all entries of any row or column is

zero, then det A = 0 If two rows or two columns are

identical, then det A = 0

Note: determinants can be calculatedusing mdetermfunction in Excel

7/27/2019 matrix-131010034223-phpapp01

43/123

Excel Demonstration

Excel treats matrices as arrays

To obtain the results of multiplication,

addition, and inverse operations, youhit control-shift-enteras opposed toenter.

The resulting matrix cannot be

alteredlets see an example usingExcel in class

matrix.xls
http://localhost/var/www/apps/conversion/tmp/scratch_6/Matrix.xlshttp://localhost/var/www/apps/conversion/tmp/scratch_6/Matrix.xls

7/27/2019 matrix-131010034223-phpapp01

44/123

Matrix Methods

Cramers Rule

Gauss elimination

Matrix inversion Gauss Seidel/Jacobi

Aa a a

a a a

a a a

11 12 13

21 22 23

31 32 33

7/27/2019 matrix-131010034223-phpapp01

45/123

Graphical Method2 equations, 2 unknowns

a x a x c

a x a x c

x

a

a

x

c

a

x

a

a

x

c

a

11 1 12 2 1

21 1 22 2 2

2

11

121

1

12

2

21

22

1

2

22

x2

x1

( x1, x2 )

7/27/2019 matrix-131010034223-phpapp01

46/123

3 2 18

32

9

1 2

2 1

x x

x x

x2

x1

3

2

9

7/27/2019 matrix-131010034223-phpapp01

47/123

x x

x x

1 2

2 1

2 2

1

21

x2

x1

2

1

1

7/27/2019 matrix-131010034223-phpapp01

48/123

3 2 18

2 2

32

9

1

21

1 2

1 2

2 1

2 1

x x

x x

x x

x x

x2

x1

( 4 , 3 )

3

2

2

1

9

1

Check: 3(4) + 2(3) = 12 + 6 = 18

7/27/2019 matrix-131010034223-phpapp01

49/123

Special Cases

No solution

Infinite solution

Ill-conditioned

x2

x1

( x1, x2 )

7/27/2019 matrix-131010034223-phpapp01

50/123

a) No solution - same slope f(x)

xb) infinite solution f(x)

x

-1/2 x1+ x2= 1-x1 +2x2 = 2

c) ill conditionedso close that the points of

intersection are difficult todetect visually

f(x)

x

7/27/2019 matrix-131010034223-phpapp01

51/123

If the determinant is zero, the slopesare identical

a x a x ca x a x c

11 1 12 2 1

21 1 22 2 2

Rearrange these equations so that we have an

alternative version in the form of a straight line:

i.e. x2= (slope) x1+ intercept

Ill Conditioned Systems

7/27/2019 matrix-131010034223-phpapp01

52/123

x a

ax

c

a

x a

ax

c

a

211

12

11

12

221

22

12

22

If the slopes are nearly equal (ill-conditioned)

a

a

a

a

a a a a

a a a a

11

12

21

22

11 22 21 12

11 22 21 12 0

a a

a aA

11 12

21 22

det

Isnt this the determinant?


7/27/2019 matrix-131010034223-phpapp01

53/123

If the determinant is zero the slopes are equal.This can mean:

- no solution- infinite number of solutions

If the determinant is close to zero, the system is illconditioned.

So it seems that we should use check the determinant of a

system before any further calculations are done.

Lets try an example.


7/27/2019 matrix-131010034223-phpapp01

54/123

Example

Determine whether the following matrix is ill-conditioned.

12

22

5.22.19

7.42.37

2

1

x

x

7/27/2019 matrix-131010034223-phpapp01

55/123

37 2 4 7

19 2 2 537 2 2 5 4 7 19 2

2 76

. .

. .. . . .

.

What does this tell us? Is this close to zero? Hard to say.

If we scale the matrix first, i.e. divide by the largesta value in each row, we can get a better sense of things.

Solution

7/27/2019 matrix-131010034223-phpapp01

56/123

-80

-60

-40

-20

0

0 5 10 15

x

y

This is further justifiedwhen we consider a graphof the two functions.

Clearly the slopes arenearly equal

1 0 1261 0 130

0004..

.

Solution

7/27/2019 matrix-131010034223-phpapp01

57/123

Another Check

Scale the matrix of coefficients, [A], so that thelargest element in each row is 1. If there areelements of [A]-1 that are several orders ofmagnitude greater than one, it is likely that thesystem is ill-conditioned.

Multiply the inverse by the original coefficient matrix.If the results are not close to the identity matrix, the

system is ill-conditioned. Invert the inverted matrix. If it is not close to the

original coefficient matrix, the system is ill-conditioned.

We will consider how to obtain an inverted matrix later.

7/27/2019 matrix-131010034223-phpapp01

58/123

Cramers Rule

Not efficient for solving large numbersof linear equations

Useful for explaining some inherent

problems associated with solving linearequations.

bxAb

bb

x

xx

aaa

aaaaaa

3

2

1

3

2

1

333231

232221

131211

7/27/2019 matrix-131010034223-phpapp01

59/123

Cramers Rule

x A

b a a

b a ab a a

1

1 12 13

2 22 23

3 32 33

1

to solve for

xi- place {b} in

the ith column

7/27/2019 matrix-131010034223-phpapp01

60/123

Cramers Rule

to solve for

xi- place {b} inthe ith column

33231

22221

11211

3

33331

23221

13111

2

33323

23222

13121

1

1

11

baa

baa

baa

Ax

aba

aba

aba

Ax

aab

aab

aab

Ax

EXAMPLE

7/27/2019 matrix-131010034223-phpapp01

61/123

EXAMPLEUse of Cramers Rule

2 3 5

5

2 3

1 1

5

5

1 2

1 2

1

2

x x

x x

x

x

7/27/2019 matrix-131010034223-phpapp01

62/123

2 3

1 1

5

5

2 1 3 1 2 3 5

1

5

5 3

5 1

1

55 1 3 5

20

54

1

5

2 5

1 5

1

52 5 5 1

5

51

1

2

1

2

x

x

A

x

x

Solution

Eli i ti f U k

7/27/2019 matrix-131010034223-phpapp01

63/123

Elimination of Unknowns( algebraic approach)

2112221111121

1212122111121

112222121

211212111

2222121

1212111

caxaaxaa

SUBTRACTcaxaaxaa

acxaxa

acxaxa

cxaxa

cxaxa

Eli i ti f U k

7/27/2019 matrix-131010034223-phpapp01

64/123

21122211

2121221

11222112

2111212

1122112112222111

2112221111121

1212122111121

aaaa

cacax

aaaa

cacax

acacxaaxaa

caxaaxaa

SUBTRACTcaxaaxaa

NOTE: same result as

Cramers Rule

Elimination of Unknowns( algebraic approach)

7/27/2019 matrix-131010034223-phpapp01

65/123

Gauss Elimination

One of the earliest methods developedfor solving simultaneous equations

Important algorithm in use today Involves combining equations in orderto eliminate unknowns and create anupper triangular matrix

Progressively back substitute to findeach unknown

Two Phases of Gauss

7/27/2019 matrix-131010034223-phpapp01

66/123

Two Phases of GaussElimination

a a a c

a a a c

a a a c

a a a c

a a c

a c

11 12 13 1

21 22 23 2

31 32 33 3

11 12 13 1

22 23 2

33 3

0

0 0

|

|

|

|

|

|

' ' '

' ' ' '

ForwardElimination

Note: the prime indicatesthe number of times theelement has changed fromthe original value.

Two Phases of Gauss

7/27/2019 matrix-131010034223-phpapp01

67/123

Two Phases of GaussElimination

11

31321211

'22

3

1

23

'

2

2

''

33

''

33

''

3

''

33

'

2

'

23

'

22

1131211

|00

|0

|

a

xaxacx

a

xac

x

a

cx

ca

caa

caaa

Back substitution

7/27/2019 matrix-131010034223-phpapp01

68/123

Rules

Any equation can be multiplied (ordivided) by a nonzero scalar

Any equation can be added to (orsubtracted from) another equation

The positions of any two equations in

the set can be interchanged.

7/27/2019 matrix-131010034223-phpapp01

69/123

EXAMPLE

2 3 1

4 4 7 1

2 5 9 3

1 2 3

1 2 3

1 2 3

x x x

x x x

x x x

Perform Gauss Elimination of the following matrix.

7/27/2019 matrix-131010034223-phpapp01

70/123

Solution

2 3 1

4 4 7 1

2 5 9 3

1 2 3

1 2 3

1 2 3

x x x

x x x

x x x

Multiply the first equation bya21/ a11= 4/2 = 2

Note: a11is called the pivot element

2624 321 xxx

7/27/2019 matrix-131010034223-phpapp01

71/123

2 3 1

4 4 7 1

2 5 9 3

1 2 3

1 2 3

1 2 3

x x x

x x x

x x x

2624 321 xxx

a21/ a11= 4/2 = 2

7/27/2019 matrix-131010034223-phpapp01

72/123

2 3 1

4 4 7 1

2 5 9 3

1 2 3

1 2 3

1 2 3

x x x

x x x

x x x

3952

17442624

321

321

321

xxx

xxxxxx

a21/ a11= 4/2 = 2

7/27/2019 matrix-131010034223-phpapp01

73/123

Subtract the revised first equation from thesecond equation

2 3 1

4 4 7 1

2 5 9 3

1 2 3

1 2 3

1 2 3

x x x

x x x

x x x

a21/ a11= 4/2 = 2

3952

17442624

321

321

321

xxx

xxxxxx

7/27/2019 matrix-131010034223-phpapp01

74/123

4 4 4 2 7 6 1 20 2 1

1 2 3

1 2 3

x x x

x x x

2 3 1

4 4 7 1

2 5 9 3

1 2 3

1 2 3

1 2 3

x x x

x x x

x x x

a21/ a11= 4/2 = 2


3952

17442624

321

321

321

xxx

xxxxxx

7/27/2019 matrix-131010034223-phpapp01

75/123

4 4 4 2 7 6 1 20 2 1

1 2 3

1 2 3

x x x

x x x

2 3 1

4 4 7 1

2 5 9 3

1 2 3

1 2 3

1 2 3

x x x

x x x

x x x

a21/ a11= 4/2 = 2


3952

17442624

321

321

321

xxx

xxxxxx

3952

120

132

321

321

321

xxx

xxx

xxx

NEWMATRIX

7/27/2019 matrix-131010034223-phpapp01

76/123

4 4 4 2 7 6 1 20 2 1

1 2 3

1 2 3

x x x

x x x

2 3 1

4 4 7 1

2 5 9 3

1 2 3

1 2 3

1 2 3

x x x

x x x

x x x

a21/ a11= 4/2 = 2


3952

17442624

321

321

321

xxx

xxxxxx

3952

120

132

321

321

321

xxx

xxx

xxx

NOW LETSGET A ZEROHERE

7/27/2019 matrix-131010034223-phpapp01

77/123

Multiply equation 1 by a31/a11= 2/2 = 1and subtract from equation 3

2 2 5 1 9 3 3 10 4 6 2

1 2 3

1 2 3

x x x

x x x

2 3 1

4 4 7 1

2 5 9 3

1 2 3

1 2 3

1 2 3

x x x

x x x

x x x

Solution

7/27/2019 matrix-131010034223-phpapp01

78/123

2 3 1

4 4 7 1

2 5 9 3

2 3 1

2 1

4 6 2

1 2 3

1 2 3

1 2 3

1 2 3

2 3

2 3

x x x

x x x

x x x

x x x

x x

x x

Following the same rationale, subtract the 3rd equationfrom the first equation

Continue the

computationby multiplyingthe second equationby a32/a22 = 4/2 =2

Subtract the thirdequation of the newmatrix

Solution

7/27/2019 matrix-131010034223-phpapp01

79/123

2 3 1

2 1

4 6 2

2 3 1

2 1

4 4

1 2 3

2 3

2 3

1 2 3

2 3

3

x x x

x x

x x

x x x

x x

x

THIS DERIVATION OFAN UPPER TRIANGULAR MATRIXIS CALLED THE FORWARDELIMINATION PROCESS

Solution

7/27/2019 matrix-131010034223-phpapp01

80/123

From the system we immediately calculate:

x34

41

Continue to back substitute

2 3 1

2 1

4 4

1 2 3

2 3

3

x x x

x x

x

x

x

2

1

1 1

21

1 3 1

2

1

2

THIS SERIES OFSTEPS IS THEBACKSUBSTITUTION

Solution

7/27/2019 matrix-131010034223-phpapp01

81/123

Pitfalls of the Elimination Method

Division by zero

Round off errors magnitude of the pivot element is small compared

to other elements Ill conditioned systems

7/27/2019 matrix-131010034223-phpapp01

82/123

Pivoting

Partial pivoting

rows are switched so that the pivot element is notzero

rows are switched so that the largest element isthe pivot element

Complete pivoting

columns as well as rows are searched for the

largest element and switched rarely used because switching columns changes

the order of the xs adding unjustified complexityto the computer program

7/27/2019 matrix-131010034223-phpapp01

83/123

For example

Pivoting is used here to avoid divisionby zero

2 3 8

4 6 7 3

2 6 5

2 3

1 2 3

1 2 3

x x

x x x

x x x

4 6 7 3

2 3 8

2 6 5

1 2 3

2 3

1 2 3

x x x

x x

x x x

7/27/2019 matrix-131010034223-phpapp01

84/123

Another Improvement: Scaling

Minimizes round-off errors for cases wheresome of the equations in a system have muchlarger coefficients than others

In engineering practice, this is often due tothe widely different units used in thedevelopment of the simultaneous equations

As long as each equation is consistent, thesystem will be technically correct and solvable

7/27/2019 matrix-131010034223-phpapp01

85/123

Use Gauss Elimination to solve the following setof linear equations. Employ partial pivoting whennecessary.

3 13 50

2 6 45

4 8 4

2 3

1 2 3

1 3

x x

x x x

x x

Example (solution in notes)

7/27/2019 matrix-131010034223-phpapp01

86/123

3 13 50

2 6 45

4 8 4

2 3

1 2 3

1 3

x x

x x x

x x

First write in matrix form, employing short handpresented in class.

0 3 13 50

2 6 1 45

4 0 8 4

We will clearly run intoproblems of divisionby zero.

Use partial pivoting

Solution

7/27/2019 matrix-131010034223-phpapp01

87/123

0 3 13 50

2 6 1 45

4 0 8 4

Pivot with equationwith largest an1

7/27/2019 matrix-131010034223-phpapp01

88/123

50133045162

4804

4804

45162

501330

7/27/2019 matrix-131010034223-phpapp01

89/123

501330

43360

4804

50133045162

4804

4804

45162

501330

Begin developingupper triangular matrix

7/27/2019 matrix-131010034223-phpapp01

90/123

4 0 8 4

0 6 3 43

0 3 13 50

4 0 8 4

0 6 3 43

0 0 14 5 28 5

28 5

14 51966

43 3 1966

68149

4 8 1 966

4

2 931

3 8149 13 1 966 50

3 2

1

. .

.

..

..

..

. .

x x

x

CHECK

okay...end ofproblem

7/27/2019 matrix-131010034223-phpapp01

91/123

Gauss-Jordan

Variation of Gauss elimination

Primary motive for introducing this method isthat it provides a simple and convenient

method for computing the matrix inverse. When an unknown is eliminated, it is

eliminated from all other equations, ratherthan just the subsequent one

7/27/2019 matrix-131010034223-phpapp01

92/123

All rows are normalized by dividing them bytheir pivot elements

Elimination step results in an identity matrix

rather than an UT matrix

Aa a a

a a

a

11 12 13

22 23

33

0

0 0 A

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

Gauss-Jordan

G hi l d i i f G J d

7/27/2019 matrix-131010034223-phpapp01

93/123

Graphical depiction of Gauss-Jordan

a a a c

a a a c

a a a c

c

c

c

n

n

n

11 12 13 1

21 22 23 2

31 32 33 3

2

3

1 0 0

0 1 0

0 0 1

1

|

|

|

|

|

|

' ' '

' ' ' '

7/27/2019 matrix-131010034223-phpapp01

94/123

1 0 0

0 1 0

0 0 1

1

1

2

3

1

2 2

3 3

|

|

|

c

c

c

x c

x c

x c

n

n

n

n

n

n

a a a c

a a a c

a a a c

c

c

c

n

n

n

11 12 13 1

21 22 23 2

31 32 33 3

2

3

1 0 0

0 1 0

0 0 1

1

|

|

|

|

|

|

' ' '

' ' ' '

Graphical depiction of Gauss-Jordan

7/27/2019 matrix-131010034223-phpapp01

95/123

Matrix Inversion

[A] [A] -1 = [A]-1 [A] = I One application of the inverse is to solve

several systems differing only by {c}

[A]{x} = {c} [A]-1[A] {x} = [A]-1{c}

[I]{x}={x}= [A]-1{c}

One quick method to compute the inverse is

to augment [A] with [I] instead of {c}

Graphical Depiction of the Gauss-Jordan

7/27/2019 matrix-131010034223-phpapp01

96/123

p pMethod with Matrix Inversion

A I

a a a

a a a

a a a

a a a

a a a

a a a

I A

11 12 13

21 22 23

31 32 33

11

1

12

1

13

1

21

1

22

1

23

1

311 321 331

1

1 0 0

0 1 0

0 0 1

1 0 0

0 1 0

0 0 1

Note: the superscript

-1 denotes thatthe original valueshave been convertedto the matrix inverse,not 1/aij

7/27/2019 matrix-131010034223-phpapp01

97/123

WHEN IS THE INVERSE MATRIX USEFUL?

CONSIDER STIMULUS-RESPONSECALCULATIONS THAT ARE SO COMMON INENGINEERING.

Sti l R C t ti

7/27/2019 matrix-131010034223-phpapp01

98/123

Stimulus-Response Computations

Conservation Laws

mass

force

heat

momentum

We considered the conservation

of force in the earlier example ofa truss

Stimulus Response Computations

7/27/2019 matrix-131010034223-phpapp01

99/123

[A]{x}={c} [interactions]{response}={stimuli}

Superposition

if a system subject to several different stimuli, the

response can be computed individually and theresults summed to obtain a total response

Proportionality

multiplying the stimuli by a quantity results in theresponse to those stimuli being multiplied by the

same quantity These concepts are inherent in the scaling of terms

during the inversion of the matrix

Stimulus-Response Computations

7/27/2019 matrix-131010034223-phpapp01

100/123

Example

Given the following, determine {x} for thetwo different loads {c}

174

321

413

362

1121

T

T

c

c

A

cAx

7/27/2019 matrix-131010034223-phpapp01

101/123

Solution

174

321

413

362

1121

T

T

c

c

A

cAx{c}T= {1 2 3}

x1= (2)(1) + (-1)(2) + (1)(3) = 3

x2= (-2)(1) + (6)(2) + (3)(3) = 19

x3= (-3)(1) + (1)(2) + (-4)(3) = -13

{c} T= {4 -7 1)

x1= (2)(4) + (-1)(-7) + (1)(1)=16

x2= (-2)(4) + (6)(-7) + (3)(1) = -47x3= (-3)(4) + (1)(-7) + (-4)(1) = -23

Gauss Seidel Method

7/27/2019 matrix-131010034223-phpapp01

102/123

Gauss Seidel Method

An iterative approach

Continue until we converge within some pre-specified tolerance of error

Round off is no longer an issue, since you controlthe level of error that is acceptable

Fundamentally different from Gauss elimination.This is an approximate, iterative method

particularly good for large number of equations

Gauss Seidel Method

7/27/2019 matrix-131010034223-phpapp01

103/123

Gauss-Seidel Method

If the diagonal elements are all nonzero, thefirst equation can be solved for x1

Solve the second equation for x2, etc.

x c a x a x a x

a

n n1

1 12 2 13 3 1

11

To assure that you understand this, write the equation for x2

7/27/2019 matrix-131010034223-phpapp01

104/123

xc a x a x a x

a

x

c a x a x a x

a

x

c a x a x a x

a

x

c a x a x a x

a

n n

n n

n n

n

n n n nn n

nn

1

1 12 2 13 3 1

11

2

2 21 1 23 3 2

22

3

3 31 1 32 2 3

33

1 1 3 2 1 1

Gauss Seidel Method

7/27/2019 matrix-131010034223-phpapp01

105/123

Gauss-Seidel Method

Start the solution process by guessingvalues of x

A simple way to obtain initial guesses is

to assume that they are all zero Calculate new values of xi starting with

x1= c1/a11

Progressively substitute through theequations

Repeat until tolerance is reached

Gauss Seidel Method

7/27/2019 matrix-131010034223-phpapp01

106/123

x c a x a x a

x c a x a x a

x c a x a x a

x c a a a c

a x

x c a x a a x

x c a x a x a x

1 1 12 2 13 3 11

2 2 21 1 23 3 22

3 3 31 1 32 2 33

1 1 12 13 111

111

2 2 21 1 23 22 2

3 3 31 1 32 2 33 3

0 0

0

/

/

/

/ '

' / '

' ' / '

Gauss-Seidel Method

E l

7/27/2019 matrix-131010034223-phpapp01

107/123

Example

2 3 1 2

4 1 2 2

3 2 1 1

Given the following augmented matrix,complete one iteration of the Gauss Seidelmethod.

2 3 1 2

4 1 2 2

GAUSS SEIDEL

7/27/2019 matrix-131010034223-phpapp01

108/123

4 1 2 2

3 2 1 1

x c a a a c

a x

x

x c a x a a x

x

x c a x a x a x

x

1 1 12 13 111

111

1

2 2 21 1 23 22 2

2

3 3 31 1 32 2 33 3

3

0 0

2 3 0 1 0

2

2

2

1

0

2 4 1 2 0

1

2 4

16

1 3 1 2 6

1

1 3 12

110

/ '

' / '

' ' / '

Jacobi Iteration

7/27/2019 matrix-131010034223-phpapp01

109/123

Jacobi Iteration

Iterative like Gauss Seidel

Gauss-Seidel immediately uses the

value of xiin the next equation topredict xi+1

Jacobi calculates all new values of xis

to calculate a set of new xivalues

Graphical depiction of difference between Gauss-Seidel and Jacobi

7/27/2019 matrix-131010034223-phpapp01

110/123

FIRST ITERATION

x c a x a x a x c a x a x a



SECOND ITERATION


x c a x a x a x c a x

1 1 12 2 13 3 11 1 1 12 2 13 3 11

2 2 21 1 23 3 22 2 2 21 1 23 3 22

3 3 31 1 32 2 33 3 3 31 1 32 2 33

1 1 12 2 13 3 11 1 1 12 2 13 3 11

2 2 21 1 23 3 22 2 2 21 1

/ /

/ /

/ /

/ /

/

a x a


23 3 22

3 3 31 1 32 2 33 3 3 31 1 32 2 33

/

/ /

E l

7/27/2019 matrix-131010034223-phpapp01

111/123

2 3 1 2

4 1 2 2

3 2 1 1

Note: We worked the Gauss Seidel method earlier

Given the following augmented matrix, completeone iteration of the Gauss Seidel method andthe Jacobi method.

Example

Gauss-Seidel Methodit i

7/27/2019 matrix-131010034223-phpapp01

112/123

convergencecriterion

a ii

j

i

j

i

j s

x x

x,

1

100

as in previous iterative procedures in finding the roots,we consider the present and previous estimates.

As with the open methods we studied previously with onepoint iterations

1. The method can diverge2. May converge very slowly

Convergence criteria for twoli ti

7/27/2019 matrix-131010034223-phpapp01

113/123

Class question:where do these

formulas come from?

linear equations

u x x c

a

a

ax

v x x

c

a

a

a x

consider the partial derivatives of u and v

u

x

u

x

a

av

x

a

a

v

x

1 21

11

12

11

2

1 2

2

22

21

22

2

1 2

12

11

1

21

22 2

0

0

,

,

Convergence criteria for two linear

7/27/2019 matrix-131010034223-phpapp01

114/123

equations cont.

u

x

v

x

uy

vy

1

1

Criteria for convergencewhere presented earlierin class materialfor nonlinear equations.

Noting that x = x1andy = x2

Substituting the previous equation:


7/27/2019 matrix-131010034223-phpapp01

115/123

equations cont.

a

a

a

a

21

22

12

11

1 1

This is stating that the absolute values of the slopes mustbe less than unity to ensure convergence.

Extended to n equations:

a a where j n excluding j iii ij 1,


7/27/2019 matrix-131010034223-phpapp01

116/123

equations cont.


This condition is sufficient but not necessary; for convergence.

When met, the matrix is said to be diagonally dominant.

Diagonal Dominance

7/27/2019 matrix-131010034223-phpapp01

117/123

Diagonal Dominance

4

3

9

x

x

x

9.05.01.0

4.08.02.0

4.02.01

3

2

1

To determine whether a matrix is diagonally

dominant you need to evaluate the values onthe diagonal.

Diagonal Dominance

7/27/2019 matrix-131010034223-phpapp01

118/123

Diagonal Dominance

Now, check to see if these numbers satisfy the

following rule for each row (note: each rowrepresents a unique equation).


4

3

9

x

x

x

9.05.01.0

4.08.02.0

4.02.01

3

2

1

7/27/2019 matrix-131010034223-phpapp01

119/123

x2

x1

Review the conceptsof divergence andconvergence by graphicallyillustrating Gauss-Seidelfor two linear equations

u x x

v x x

:

:

11 13 286

11 9 99

1 2

1 2

11 9 99

7/27/2019 matrix-131010034223-phpapp01

120/123

x1

Note: we are convergingon the solution

v x x

u x x

:

:

11 9 99

11 13 286

1 2

1 2

CONVERGENCE

x2

11 13 286

7/27/2019 matrix-131010034223-phpapp01

121/123

x1

Change the order ofthe equations: i.e. change

direction of initialestimates

u x x

v x x

:

:

11 13 286

11 9 99

1 2

1 2

DIVERGENCE

x2

Improvement of ConvergenceU i R l ti

7/27/2019 matrix-131010034223-phpapp01

122/123

Using Relaxation

This is a modification that will enhance slow convergence.

After each new value of x is computed, calculate a new valuebased on a weighted average of the present and previous

iteration.

x x xinew inew iold 1

Improvement of Convergence UsingR l ti

7/27/2019 matrix-131010034223-phpapp01

123/123

Relaxation

if = 1unmodified

if 0 < < 1 underrelaxation nonconvergent systems may converge

hasten convergence by dampening outoscillations

if 1

matrix-131010034223-phpapp01

Documents