section 3 systems of eqs

8/9/2019 Section 3 Systems of Eqs

1/27

ENGRD 241 Lecture Notes Section 3: Systems of Equations page 3-1 of 3-27

SYSTEMS OF EQUATIONS (C&C 4th T ! "Chs# $%12'

"A )* + c*

S,ste-s o. E/u0tos (C&C 4th3 T !#13 # 215'Determine x

1,x

2,x

3,…,xn such that

f 1(x1,x2,x3,…,xn) !f 2(x

1,x

2,x

3,…,xn) !

f n(x1,x2,x3,…,xn) !

Le0r A67e8r0c E/u0tos

a11x1 " a12x2 " a13x3 " 9 " a1nxn #1

a21x1 " a22x2 " a23x3 " 9 " a2nxn #2

an1x1 " an2x2 " an3x3 " 9 " anxn #n $here a%% ai&'s an #i's are constants

I -0tr) .or-: (C& C 4th3 T!#2#13 #22; 0< T!#2#!3 # 22='

11 12 13 1n 1 1

21 22 23 2n 2 2

31 32 33 3n 3 3

n1 n2 n3 nn n n

a a a a x #

a a a a x #

a a a a x #

a a a a x #

K

K

K

M M M

K

) ) 1 ) 1

or simp%y "A)* + 8*

A6c0tos o. So6utos o. Le0r S,ste-s o. E/u0tos* Steay-state reactor in +hemica% Engineering (121)* Static structura% ana%ysis (122)* otentia%s an currents in e%ectrica% net$ors (123)* Spring - mass moe%s (12.)* So%ution of partia% ifferentia% Equations (2/-32)

0eat an f%ui f%o$o%%utants in enironmenteather Stress ana%ysis materia% science4

* 5nerting matrices* 6u%tiariate 7e$ton-8aphson for

non%inear systems of equations (/9)* Dee%oping approximations:

east squares (1;)Sp%ine functions (1


2/27


C0te7ores o. So6uto Methooron- ? Decomposition (Doo%itt%e an +ho%esy)

2 Iterative - >aco#i 6etho- =auss - Seie% 6etho

- Conjugate Gradient Method

Out6e o. our stue G0uss0 E6-0to: (C& C 4th3 $#23 # 2!?'

So%e @4Cx Cc for Cx, $ith @4 n x n an Cx, Cc ∈ 8 n

Basic Approach:1or$ar E%imination: Fae mu%tip%es of ro$s a$ay from su#sequent

ro$s to Gero out co%umns such that @4 is conerte to ?4

2Bac Su#stitution: Beginning $ith ?4, #ac-su#stitute to so%e for the xi's

Details:

1 Forward Elimination (8o$ 6anipu%ation)a orm augmente matrix @Hc4

11 12 1n 1 1 11 12 1n 1

21 22 2n 2 2 21 22 2n 2

n1 n2 nn n n n1 n2 nn n

a a a x # a a a #a a a x # a a a #

a a a x # a a a #

= ==>

L LL L

M M O M M M M M O M ML L

# By e%ementary ro$ manipu%ations, reuce @ H #4 to ? H #'4

$here ? is an upper triangu%ar matrix:DA i 1, n-1

DA i"1, n8o$() 8o$() - (aiIaii) J 8o$(i)

E7DDAE7DDA


3/27


N0>e G0uss0 E6-0to (cot@e G0uss0 E6-0to: Example

+onsier the system of equations

1

2

3

L! 1 2 x 1

1 .! . x 2

2 9 3! x 3

Fo 2 significant figures, the exact so%ution is:

{ }true!!19

x !!.1

!!/1

e $i%% use 2 ecima% igit arithmetic $ith rouning

Start $ith the augmente matrix:

L! 1 2 1

1 .! . 2

2 9 3! 3

6u%tip%y the first ro$ #y K1IL! an a to seconro$

6u%tip%y the first ro$ #y K2IL! an a to thirro$:

L! 1 2 1! .! . 2

! 9 3! 3

6u%tip%y the secon ro$ #y K9I.! an a to thirro$:

L! 1 2 1

! .! . 2

! ! 2/ 2);


4/27


N0>e G0uss0 E6-0to: Example (cot@


5/27


N0>e G0uss0 E6-0to: Example (cont'd)

E..ect o.


6/27


• 8o$ pioting oes not affect the orer of the aria#%es

• 5nc%ue in any goo =aussian E%imination routine

IOTING (cot@


7/27


* on't actua%%y sca%e, #ut use imaginary sca%ing factors to etermine$hat pioting is necessary

* sca%e on%y #y po$ers of 2: no rounoff or iision require

o to .oo6 sc067:

@ poor choice of units can unermine the a%ue of sca%ing Begin $ith our

origina% examp%e

L! 1 2 1

1 .! . 2

2 9 3! 3

5f the units of x1 $ere expresse in g instea of mg the matrix might rea:

L!!!! 1 2 1

1!!! .! . 2

2!!! 9 3! 3

Sca%ing yie%s:

1 !!!!!2 !!!!!1 !!!!!1

1 !!. !!!. !!!2

1 !!!3 !!1L !!!1L

hich equation is use to etermine x1TTT

h, 8other to sc06eB

OERATION COUNTING (C&C 4th3 $#2#13 #242'

7um#er of mu%tip%ies an iies often etermines the +? time

Ane f%oating point mu%tip%yIiie an associate asIsu#tracts is ca%%e

a A: FLoating point Oeration

So-e Use.u6 I


8/27


3m2 2 2 2 2

i1

m(m"1)(2m"1) m9) i 1 " 2 " " m " (m )

9 3∑ K "

A(mn) means Uterms of orer mn an %o$erU

S-6e E)0-6es o. Oer0to Cout7:

1 DA i 1 to nV(i) W(i)Ii K 1

E7DDA

5n each iteration W(i)Ii K 1 represents 1 A #ecause it requires oneiision X one su#traction

Fhe DA %oop extens oer i from 1 to n iterations:

n

i 1

1 n

=

=∑ AS

2 DA i 1 to nV(i) W(i) W(i) " 1DA & i to n

Y(&) V(&) I W(i) 4 V(&) " W(i)E7DDA

E7DDA

ith neste %oops, a%$ays start from the innermost %oop

V(&)IW(i)4 J V(&) " W(i) represents 2 AS

n

& i

2

=

∑ n

& i

2 1

=

∑ 2(n K i " 1) ASor the outer i-%oop:

W(i)•W(i) " 1 represents 1 A

n

i 1

1 2(n i 1)4

=

+ − +∑ (3 " 2n)n n

i 1 i 1

1 2 i

= =

−∑ ∑

(3 " 2n)n K2

)1n(n2 +

3n " 2n

2

K n

2

K n

n2 " 2n

2 O('


9/27


Oer0to Cout7 .or G0uss0 E6-0to

#or$ard Elimination:DA 1 to nK1

DA i "1 to nr @(i,)I@(,)DA & "1 to n

@(i,&)@(i,&) K r J@(,&)E7DDA+(i) +(i) K r J+()

E7DDA

E7DDA

Bac% Su&stitution:W(n) +(n)I@(n,n)DA i nK1 to 1 #y K1

S?6 !DA & i"1 to n

S?6 S?6 " @(i,&)JW(&)

E7DDAW(i) +(i) K S?64I@(i,i)E7DDA

Forward Elimination

5nner %oop

n

& 1

1

= +∑ n K ("1) " 1 H

Secon %oop

n

i 1

2 (n )4

= +

+ −∑

(2 " n) K 4 (n K )

(n2 " 2n) K 2(n " 1)

2

Auter %oop +

n 12 2

1

(n 2n) 2(n 1) 4−

=

+ − + +∑

(n2"2n)

n 1

1

1−

=∑ K 2(n"1)

n 1

1

−

=∑

n 12

1

−

=∑

(n2"2n)(n-1) K 2(n"1)(n 1)(n)

2

−

9

)1n2)(n)(1n( −−

+!

! (

2'


10/27


Back Substitution

5nner oop

n

& i 1

1

= +∑ n K (i "1) " 1 H

Auter oop [ ]

n 1

i 11 (n i)

−

= + −∑ (1"n)n 1

i 11

−

=∑ Kn 1

i 1i

−

=∑ (1"n) (n–1) –

(n 1)n

2

−

2

2 ('

*otal #lops or$ar E%imination " Bac Su#stitution

n3I3 " " (n2) " n2I2 " " (n)

≈ n3I3 " " (n2)

Fo conert (@,#) to (?,#') requires n3I3, p%us terms of orer n2 ansma%%er, f%ops

*o back solve re%uires 1 " 2 " 3 " . " " n n (n"1) I 2 f%opsP

Grand *otal the entire effort requires !J! O(2' .6os 06to7ether

G0uss%Kor


11/27


G0uss%Kor


12/27


1 ! !!3< !!2 !!!L !

! 1 !1 !!!!L !!2L !

! ! 2< !!3; !1L 1

− − − −

1 ! ! !!2 !!!!2/ !!!1.! 1 ! !!!!3; !!29 !!!39

! ! 1 !!!13 !!!L. !!39

− − − − − −

MATRI INERSE "A%1

CHECK:[ A ] [ A ]-1 = [ I ]

L! 1 2 !!2! -!!!2/ -!!!1. !//; !13 !!!2

2 .! . -!!!!3; !!29 -!!!39 !!!! 1!19 !!!1

2 9 3! -!!!13 -!!!L. !!39 !!!1 !!12 1!L9

− −

= −

− −

[ A ]-1 { c } = { x } Gaussian Elimination

!!2! -!!!2/ -!!!1. 1 !!1L

-!!!!3; !!29 -!!!39 2 !!33

-!!!13 -!!!L. !!39 3 !!//

=

!!19

x !!.1

!!/1true =

!!19

x !!.!

!!/3

=

LU Deco-osto (C&C 4th3 1;#13 # 2=4'

@ practica%, irect metho for equation so%ing, especia%%y $hen seera%

rhs's are neee anIor $hen a%% rhs's are not no$n at the start of the pro#%em

E)0-6es o. 06c0tos:

* time stepping: @ u t"1 B u t " ε t* seera% %oa cases: @xi #i* inersion: @-1

* iteratie improement

Do@t co-ute A%1

8equires n3 f%ops to get @-1

7umerica%%y unsta#%e ca%cu%ation

LU co-ut0tosStores the ca%cu%ations neee to repeat'=aussian' e%imination on a ne$ rhs c-ector


13/27


LU Decomposition (See C&C Figure 10.1)

@ 4 C x C #

? 4 4 a) ecomposition @4 Z 4?4

4 C C # #) for$ar su#stitution H # 4 Z C

? 4 C x C c) #ac$ar su#stitution

C x

Doolittle /0 Decomposition (See C&C 4th3 1;#1#2%!3 # 2=='? is &ust the upper triangu%ar matrix from =aussian e%imination

@ H #4 ? H #′ 4

4 has one's on the iagona% (ie, it is a Uunit %o$er triangu%ar matrixUan therefore can #e enote 14), an e%ements #e%o$ iagona% are &ust the factors use to sca%e ro$s $hen oing =aussian e%imination,

eg, 1 1 11Ii ia a=l for i 2, 3, …, n

[ ]

11 12 13 1n

21 22 23 2n

31 32 33 3n

n1 n2 n3 nn

a a a a

a a a a

@ a a a a

a a a a

=

L

L

L

M M M O M

L

21

31 32

n1 n2 n3

1 ! ! !

1 ! !

1 !

1

L

l L

l l L

M M M O M

l l l L

11 12 13 1n

22 23 2n

33 3n

nn

u u u u

! u u u

! ! u u

! ! ! u

L

L

L

M M M O M

L

the 14 C C#

Z ?4 Cx C in $hich C is synonymous $ith C#'

Basic 1pproac& (.2. Fi$ure 343)+onsier @4Cx C#

a) ?se =auss-type UecompositionU of @4 into 4?4 n3I3 f%ops

@4Cx C# #ecomes 4?4Cx C#

#) irst so%e 4C C# for C #y for$ar su#stitution n2I2 f%ops

c) Fhen so%e ?4Cx C for Cx #y #ac su#stitution n2I2 f%ops


14/27


LU Deco-osto 0r0tosDoo%itt%e 14?4 =enera% @4 +X+ 1!12

+rout 4?14 =enera% @4 +X+ 1!1.

+ho%esy 44F

D Symmetic +X+ 1112

+ho%esy $ors on%y for ositie Definite symmetric matrices

Doo6tt6e >ersus Crout:

• Doo%itt%e &ust stores =aussian e%imination factors $here +rout uses a

ifferent series of ca%cu%ations, see +X+ Section 1!1.

• Both ecompose @4 into 4 an ?4 in n3I3 AS

• Different %ocation of iagona% of 1's (see a#oe)

• +rout uses each e%ement of @4 on%y once so the same array can #e

use for @4 an [?14 saing computer memoryN (Fhe 1\s of ?14 arenot store)

M0tr) I>erso (C&C 4th3 1;#23 # 25!'

Frst Ru6e: Dot


15/27


FLOS .or Le0r A67e8r0c E/u0tos3 "A)* + 8*:

=auss e%imination (1 rhs)3

2n " (n )3

"

=auss->oran (1 rhs) 3 2n " (n )2

" L!M more than =auss E%imin4

? ecomposition3

2n " (n )3

"

Each ne$ ? right-han sie n2

+ho%esy ecomposition (symmetric @)3

2n " (n )9

" 0a%f the As of =auss E%imin4

5nersion (naie =auss->oran)

3

2.n " (n )3

"

5nersion (optima% =auss->oran) n3 " "(n2)

So%ution #y +ramer's 8u%e nN


16/27


Errors So6utos to S,ste-s o. Le0r E/u0tos (C&C 4th, 10.3, p. 277)

O8Pect>e: So%e @4Cx C#

ro86e-: 8oun-off errors may accumu%ate an een #e exaggerate #y the so%ution proceure Errorsare often exaggerate if the system is i%%-conitione

oss86e re-e


17/27


5as to detect ill+conditionin$1 +a%cu%ate CxP mae a sma%% change in @4 or C# an etermine effect on the ne$ so%ution Cx2 @fter for$ar e%imination examine iagona% of upper triangu%ar matrix if aii ]] a &&, ie there is a

re%atie%y sma%% a%ue on the iagona%, then this may inicate i%%-conitioning3 +ompare CxS57=E $ith CxDA?BE.

Estimate Uconition num#erU for @

Su#stituting the ca%cu%ate Cx into @4Cx an checing this against C# $i%% unfortunate%y nota%$ays $orN (See +X+ Box 1!1)

Nor-s 0< the Co


18/27


M0tr) Nor-s (C&C 4th3 o) 1;#23 # 25$'

Sca%er measure of the magnitue of a matrix

6atrix norms corresponing to the ector norms a#oe are efine #y the genera% re%ationship:

[ ] { } p 1

x p p

@ max @ x

=

=

1 argest co%umn sum:

n

1 i& & i 1

@ max a

== ∑

2 argest ro$ sum:

n

i&i & 1

x max a∞

=

= ∑

3 Spectra% norm: 1I 22 max@ ( )= µ

$here max is the %argest eigena%ue of @4F@4

5f @4 is symmetric, (max)1I2 λ max, the %argest eigena%ue of @4

Matrix Norms

or matrix norms to #e usefu% $e require that

! HH @x HH ≤ HH @ HH HHx HH

=enera% properties of any matrix norm:

1 HH @ HH ≥ ! an HH @ HH ! iff @4 !2 HH @ HH HH @ HH $here is any positie sca%ar

3 HH @ " B HH ≤ HH @ HH " HH B HH UFriang%e 5nequa%ityU

. HH @ B HH ≤ HH @ HH HH B HH

h, 0re or-s -ort0tB

• 7orms permit us to express the accuracy of the so%ution Cx in terms of HH x HH

• 7orms a%%o$ us to #oun the magnitue of the prouct @4Cx an the associate errors


19/27


D7resso: A re. Itro06ue ro86e-

or a square matrix @4, consier the ector Cx an sca%ar λ for $hich

@4Cx λ Cx

x is ca%%e an eigenvector an λ an eigenvalue of @ Fhe pro#%em of fining eigena%ues aneigenectors of a matrix @ is important in scientific an engineering app%ications, eg, i#ration pro#%ems,sta#i%ity pro#%ems See +hapra X +ana%e +hapter 2; for #acgroun an examp%es

Fhe a#oe equation can #e re$ritten

( @4 K λ 54 ) Cx !

inicating that the system ( @4 K λ 54 ) is singu%ar Fhe characteristic equation

et (@4 – λ 54) !,

yie%s n rea% or comp%ex roots $hich are the eigena%ues λ i, an n rea% or comp%ex eigenectors Cxi 5f

@4 is rea% an symmetric, the eigena%ues an eigenectors are a%% rea% 5n practice, one oes not usua%%yuse the eterminant to so%e the a%ge#raic eigena%ue pro#%em, an instea emp%oys other a%gorithmsimp%emente in mathematica% su#routine %i#raries

Forward and Backward Error 1nalsis

or$ar an #ac$ar error ana%ysis can estimate the effect of truncation an rounoff errors on the precision of a resu%t Fhe t$o approaches are a%ternatie ie$s:

1 or$ar (a priori) error ana%ysis tries to trace the accumu%ation of error through each process of the

a%gorithm, comparing the ca%cu%ate an exact a%ues at eery stage2 Bac$ar (a posteriori) error ana%ysis ie$s the fina% so%ution as the exact so%ution to a pertur#e

pro#%em Ane can consier ho$ ifferent the pertur#e pro#%em is from the origina% pro#%em

0ere $e use the co


20/27


Error A06,ss .or "A)* + 8* .or errors 8*

Suppose the coefficients C# are not precise%y represente

hat might #e the effect on ca%cu%ate Cx" δxT

Le--0: @4Cx C# yie%s HH @ HH HH x HH ≥ HH # HH

or # @1x@ x #

≥ ⇒ ≤

7o$ an error in C# yie%s a corresponing error in Cx:

@ 4Cx " δx C# " δ #

@4Cx " @4Cδx C# " Cδ #

Su#tracting @4Cx C# yie%s:

@4Cδx Cδ # KKZ Cδx @4-1 Cδ #

Faing norms an using the %emma:

1x # #@ @x # #

−δ δ δ≤ × × = κ

Define the co


21/27


Est-0te o. Loss o. S7.c0ce:

+onsier the possi#%e impact of errors δ@4 on the precision of Cx

5f p@ ^ 1!

@

−δ, then

x @_

x x @

δ δ≤

+ δimp%ies

that ifx

^ 1!x x

sδ+ δ

− , then 1!_ 1! s p− −≤

or, taing the %og of #oth sies, one o#tains: s ≥ 6o71;( '

* %og1!(κ ) is the %oss in ecima% precision, ie, $e start $ith p significant figures an en-up $ith ssignificant figures (Fhis iea is expresse in $ors at #ottom of p2

* Ane oes not necessari%y nee to fin @4-1 to estimate κ con@4 +an use an estimate #aseupon iteration of inerse matrix using ? ecomposition

* Ane oes not necessari%y nee to fin @4

-1

to estimate κ con@4 orexamp%e, if @4 is symmetric positie efinite, κ ` maxI` min an one can #oun ` max #y any matrix norm an ca%cu%ate ` min using the ?ecomposition of @4 an a metho ca%%e inerse ector iterationb

* rograms such as 6@F@B hae #ui%t-in functions to ca%cu%ate κ con@4or the reciproca% conition num#er 1Iκ (conb an rconb)

Iter0t>e So6uto Metho


22/27




23/27


Fhis iteratie metho is ca%%e the ,acobi -et&od ina% form:

Cx &"1 D4-1( Cc K ( o4"?o4 )Cx & )

or, $ritten out more fu%%y:

( ) & 1 & & &1 12 13 2n n 111 2 3x # a x a x a x a+ = − + + +

L

( ) & 1 & & &2 21 23 2n n 222 1 3x # a x a x a x a+ = − + + + L

( ) & 1 & & &3 31 32 2n n 333 1 2x # a x a x a x a+ = − + + + L

:

( ) & & & & 1n n n1 n2 n,n 1 nn1 2 n 1x # a x a x a x a+ − − = − + + + L

7ote that, a%though the ne$ estimate of x1 $as no$n, $e i not use it to ca%cu%ate the ne$ x2



24/27


Co>er7ece o. Iter0t>e So6uto Metho


25/27


I-ro>7 R0te o. Co>er7ece o. Iter0t>e So6uto Sche-es

(C&C 4th3 11#2#23 # 2$4:

+elaxation Schemes:ne$ tria% o%i i ix `x " (1- `) x $here !! ] λ ≤ 2!

Uerre60)0to ( 1! ] λ ≤ 2! )6ore $eight is p%ace on the ne$ a%ue@ssume the ne$ a%ue is heaing in the right irection, an hence pushes the ne$ a%ue c%oser tothe true so%ution

Fhe choice of λ is high%y pro#%em epenent an is empirica%, so re%axation is usua%%y on%y use foroften repeate ca%cu%ations of a particu%ar c%ass

h, Iter0t>e So6utosB

Vou often nee to so%e @ x # $here n 1!!!'s

* Description of a #ui%ing or airframe,

* inite-Difference approximation to DE

6ost of @'s e%ements $i%% #e GeroP for finite-ifference approximations to ap%ace equation hae fie

ai& ≠ ! in each ro$ of @

Direct met&od (Gaussian elimination)

* 8equires n3I3 f%ops (n L!!!P n3I3 . x 1!1! f%ops)

* i%%s in many of n2-Ln Gero e%ements of @

Iterative met&ods (,acobi or Gauss+Seidel)* 7eer store @

(say n L!!!P on\t nee to store .n2 1!! 6ega#yte)

* An%y nee to compute (@-B)xP an to so%e Bx t"1 #

* Effort:

Suppose B is iagona%, so%ing B # n f%ops

+omputing (@–B) x .n f%ops

or m iterations Lmn f%ops

or n m L!!!, Lmn 12Lx1!


26/27


*'"l"bl') a*+ th' $r!con"ition!" conjuat! ra"i!nt m!t#o"s a%' ''*$%' %ap+l $*'%"'*t.


27/27

section 3 systems of eqs

Documents