section 2 rootfinding

8/11/2019 Section 2 Rootfinding

http://slidepdf.com/reader/full/section-2-rootfinding 1/15

ROOTS OF EQUATIONS (C&C 4 th , PT 2)

Objective: Solve for x, given that f(x) = 0-or equivalently, solve for x such that

g(x) = h(x)==> f(x) = g(x) – h(x) = 0

Applic ti!"# ( C&C 4 th , T ble PT2$%, p$ % '):• heat balance• mass balance• total energy balance in mechanical

an structural systems

• !irchhoff"s #a$% V i

i = &

n

∑ aroun a loo'

• inimi ing an maximi ing functions% 0=dS dF

*n many esign a''lications+- one as'ect of system is stu ie- equations are nonlinear an im'licit, no analytical or close form

solution is 'ossible- force to consi er numerical metho s

Illustrative examples of performance of rootfinding methods can be found in C&C Ch 8.

ete *i"e e l !!t# !+ :• lgebraic quations• .ranscen ental quations

• /ombinations thereof E" i"ee i" Ec!"!*ic# E- *ple:

unici'al bon has annual 'ayout of &,000 for 10 years2*t costs 3,400 to 'urchase no$2 *m'licit interest rate, i 5Solution+ 6resent-value, 67, is+

n& (& i)67

i

− − +=

in $hich+ 67 = 'resent value or 'urchase 'rice = 3,400= annual 'ayment = &,000

n = number of years = 10i = interest rate = 5

8e nee to solve the equation for i+( ) 10& & i

3,400 &,000i

− − + =

quivalently, fin the root of+( ) 10& & i

f (i) 3,400 &, 000i

− − + = −

= 0



T.! F/"0 *e"t l App ! che#:&2 Bracketing or Closed Methods

- 9isection etho- :alse-'osition etho

12 Open Methods- ;ne 6oint *teration- <e$ton- a'hson *teration- Secant etho

1i#ecti!" eth!0 ( C&C 4 th , 3$2, p$ %%')

iven lo$er an u''er boun s, x l an xu $hich brac?et the root+

f(x&) f(xu) @ 0

&) stimate the oot by mi 'oint+ l ur

x xx

1+=

1) evise the brac?et+

f(xl) f(x

r ) @ 0, x

r –> x

u,

f(x&) f(xr ) > 0, x r –> x l

A) e'eat ste's &-1 until+

(a) B f(xr ) B @κ (b) εa @εs , $ith εa = x r

new − x r ol d

x r new

×&00C (c) u lx x− ≤ δ ( ) maximum D of iterations is reache

f(x 1) f(x r) > 0x

f(x)

l ur

x xx

1+= f(x u)

(x u)(x 1)

f(x 1)

f(x r)

f(x u)

f(x 1)

(x 1)

(x u)(x r)

f(x r)

f(x)

xx r => x l



S/** !+ 1i#ecti!" eth!0 Advantages:

&2 Sim'le12 oo estimate of maximum error

& umax

x x1−≤

A2 /onvergence guaranteeii E E

max

&

max420=+

Disadvantages:&2 Slo$12 equires initial interval aroun root+

Ese gra'h of function,incremental search, ortrial F error

F l#e5p!#iti!" eth!0 ( C&C 4 th , 3$6, p$ %24)

Similar to bisection2 Eses linear inter'olation to a''roximate root x r

&) u & ur u

& u

f (x ) (x x )x x

f (x ) f (x )−= −

−

1) evise the brac?et+f(x&) f(x r ) @ 0, xr –> x u,f(x&) f(x r ) > 0, x r –> x &

A) e'eat ste's &-1 until+

(a) B f(xr ) B @κ (b) εa @εs , $ith εa = x r

new − x r ol d

x r new

×&00C (c) Bxu – x&B≤ δ ( ) maximum D of iterations is reache

s

x

f(x)

f(x u)

(x u)

(x 1)

f(x 1)

f(x r)

u & ur u

&

f (x )(x x )x x

f (x ) f (xu)−= −

−

f(x 1) f(x r) > 0

x 1 = x r

f(x)

f(x u)

f(x 1)

(x 1)(x u)

(x r)

f(x r)

x



Sc! e Sheet +! R!!t+i"0i" E- *ple:

*nitial st(s) <o2 of *terations <o2 of *terationsetho Gexact = 02&&HA4I forεs = 1x&0-1 for εs = 1x&0-3

9isection (0200, &200) H 1J(0204, 02&4) J 11

:alse-'os2 (0200, &200) && 1K(0204, 02&4) A &L

S/** !+ F l#e5P!#iti!" *eth!0 Advantages

&2 Sim'le12 9rac?ets rootA2 ives maximum error

Disadvantages&2 /an be 7 M slo$12 #i?e 9isection, nee s an initial interval aroun the root

R!!t# !+ E7/ ti!"# 5 Ope" eth!0# ( C&C 4 th , Ch$ ', p$ %66)

/haracteristics+&2 *nitial estimates nee not brac?et root12 enerally converge faster A2 NOT guarantee to converge

;'en etho s /onsi ere +- ;ne 6oint *teration

- <e$ton- a'hson *teration- Secant etho

O"e5p!i"t Ite ti!" ( C&C 4 th , '$%, p$ %64)

• 're ict a value x iN& as a function of x i2

/onvert f(x) = 0 to x = g(x)

• iteration ste's+ x iN& = g(x i)

ne$ oli i &x x +=

Example I: 3400 = &000 G & – (&Ni)-10IO i

iiN& = G & – (&Nii)-10

I O 324

Example II: ( ) ( )sin x

f x &20 020x

= − =

x = sin(x) xiN& = sin(x i) OR x = arcsin(x) xiN& = arcsin(x i)



C!"ve e"ce:

Poes x move closer to real root (5)Pe'en s on+

&2 nature of the function12 accuracy of the initial estimate

*ntereste in+

&2 8ill it converge or $ill it diverge 512 Qo$ fast $ill it converge 5(rate of convergence)

C!"ve e"ce !+ the O"e5p!i"t Ite ti!" eth!0:

oot satisfies+ x r = g(x r ) g(x r ) – xr = 0

.he .aylor series for function g is+xiN& = g(x i)

= g(x r ) N g "(ξ) (x i – x r ) xr @ξ @ xi

Subtracting yiel s(xr – x iN&) = g "(ξ) (x r – x i)

or i & ig "( )+ = ξ

&2 .rue error for next iteration smaller than true errorin 'revious iteration if Bg"( ξ)B @ &20 (it $illconverge )2

12 9ecause g "(ξ) is almost constant, the ne$ error is irectly 'ro'ortional to theol error ( linear rate of convergence )2

F/ the C!"#i0e ti!"#:

/onvergence e'en s on ho$ f(x)=0 is converte into x = g(x), so222

/onvergence may be im'rove by recasting the 'roblem2

C!"ve e"ce P !ble*:

:or slo$ly converging functions

ne$ ola

ne$

x x&00C

x−ε = ×

can be small, even if x ne$ is not close to root2

Remedy: Po not com'letely rely on εa to ensure the 'roblem $as solve 2

/hec? to ma?e sure that B f(x ne$ ) B @κ 2



Ne.t!"5R ph#!" eth!0 ( C&C 4 th , '$2, p$ %68)

9e!*et ic l e iv ti!":

Slo'e of tangent R x i is

ii

i i &

f (x ) 0f "(x )

x x +

−=−

solve for x iN&+

ii & i

i

f (x )x x

f "(x )+ = −

G<ote that this is the same form as the generali e one-'oint iteration, x iN& = g(x i)I

.aylor Series Perivation+ 0 = f(x r ) ≈ f(xi) N f " (xi) (xr – xi )

8e solve for x r to yiel next guess x iN&

xr ≈ ii & i

i

f(x )x x

f (x )+ = − ′

.his has the form x iN& = g(x i) $ith g" (x r ) = 1(f "f " f f "")

& 0(f")

−− =

<e$ton- a'hson iteration+ x i+& = x i − f(x i )

′f (x i ) ne$ oli i &x x +=

.his iteration is re'eate until+&2 f(x)≈ 0, i2e2, B f(xiN&) B @κ

12

ε a =x i + & − x i

x i + &

× &00C ≤ ε s

A2 ax2 D iterations is reache

xi = x

i+1

ii & i

i

f (x )x x

f "(x )+ = −

Tangentw/slope=f '(x

i )

x

f(x)

f(xi)

xi

f(xi+1

)

iii i &

f (x ) 0f "(x )x x +

−= −

xi+

1

x

f(x)

f(xi)

(xi)

f(xi+1

)

iii i &

f (x ) 0f "(x ) x x +

−= −

xi+1



1!"0 E- *ple:

.o a''ly <e$ton- a'hson metho to+10& (& i)

f (i) 3400 &000 0i

− − += − =

<ee erivative of function+

101&&000 & (& i)

f "(i) 10(& i)i i

−− − + = − +

Score Sheet for Newton-Raphson Example:

Method Initial Est(s). es = 2x10 -2 es = 2x10 -7

9isection (0200, &200) H 1J(0204, 02&4) J 11

:alse-'os2 (0200, &200) && 1K(0204, 02&4) A &L

<- &20 iverges iverges02 4 1, but $rong LK

0214 4 3

02&4 A 4

0204 L 4

E ! A" l #i# +! N5R :

ecall that ii & ii

f(x )x xf "(x )+ = −

.aylor Series gives+1

r i i r i r if ""( )

f (x ) f (x ) f "(x )(x x ) (x x )1ξ= + − + −

$here x r ≤ ξ ≤ xi an f(xr ) = 0

Pivi ing through by f "(x i) yiel s

1i i r i r i

i

f ""( )0 f(x )Of "(x ) (x x ) (x x )

1f " (x )ξ= + − + −

1i & i r i r i

i

f ""( )(x x ) (x x ) (x x )

1f " (x )+ξ= − − + − + −

; + 1

r i & r ii

f ""( )(x x ) (x x )

1f " (x )+ξ− = − − 1

i & ii

f ""( )

1f " (x )+ξ

=

iN& is 'ro'ortional to i1, uadratic rate of convergence



i small → iN& very small% i large → anything can ha''en2



S/** !+ Ne.t!"5R ph#!" *eth!0 Advantages

• /an be very fast

Disadvantages• may not converge• requires erivative to be evaluate

• ero erivative blo$s u'

Sec "t eth!0 ( C&C 4 th , '$6, p$ %43)

Secant metho solution+ ''rox2 f " (x) $ith bac?$ar :PP+

i & ii

i & i

f(x ) f(x )f "(x )

x x−

−

−≈−

Substitute this into the <- equation+ ii & i

i

f(x )x x

f "(x )− = −

to obtain the iterative ex'ression+ x i+& = x i − f(x

i)(x

i−& −x

i)

f(x i−&) −f(x i )

&) equires t$o initial estimates+xi-& an xi .hese o <;. have to brac?et root

1) aintains a strict sequence+ x i+& = x i − f x i( )x i−& −x i( )

f x i−&( )− f x i( )e'eate until+

a2 B f(xiN&) B @κ $ith κ = small number

b2 ε a =x i + & − x i

x i + &

× &00C ≤ ε s

c2 ax2 D iterations reache (note no δ)

A) *f xi an xiN& $ere chosen to brac?et the root, this $oul be the same as the:alse-6osition etho 2 9E. 8 P;<".

xi = x

i+1

i i & ii & i

i & i

f (x )(x x )x x

f (x ) f (x )−

+−

−= −−

x

f(x)

f(xi)

xi

f(xi-1

)

i-& ii

i-& i

f(x ) -f(x )f "(x )

x - x≈

f(x)

xi-1

xi+

1

x

f(xi)

xi

f(xi-1

)

i-& ii

i-& i

f(x ) -f(x )f "(x )

x - x≈

xi-1

xi+1



Score Sheet for Secant Exampleno. of iterationsno. of iterations

Method Initial Est(s). !ith es = 2x10 -2 es = 2E-7

Bisection (0.00, 1.00) 9 26(0.05, 0.15) 6 22

False- os. (0.00, 1.00) 11 2!(0.05, 0.15) " 1#

$-% 1.0 &i'er es &i'er es0.5 2, ut *ron #!

0.25 50.15 " 50.05 # 5

ecant (0, 1) &i'er es &i'er es

(0.00, 0.50)#, ut *ron (near c aotic) 2(0.05, 0.15) " 6

$-% 1.00 " # as i f(i)/ 0.150 2 #

0.050 # 5

0.0# cra results0.0" con'er es to i=0

h 0! !pe" *eth!0# + il;

:unction may not loo? linear2 Remedy: recast into a linear form2 :or exam'le,

f(i) = 7,500 - 1, 0001-(1+i) -20

i = 0

is a 'oorly constraine 'roblem in that there is a large, nearly flat one for $hichthe erivative is near ero2 ecast as+

i f(i) = 0 = 3,400 i – &000 G & – (&Ni)-10I

• .he recast function, Ti f(i)T $ill have the same roots as f(i) 'lus an a itionalroot at i = 02

• *t $ill not have a large, flat one2.hus, h(i) = i f(i) = 3,400 i – &000 G & – (&Ni) -10I <- also nee s the first erivative+ h"(i) = 3,400 – 10,000 (&Ni) -1&



Fi / e: +(i) (#!li0 li"e) , i +(i) (0!tte0 li"e)

R!!t# !+ E7/ ti!"# < C #e# !+ /ltiple R!!t# ( C&C 4 th , '$4, p$ %3 )

/ltiple R!!t#: f(x) = (x – 1) 1(x – L)

x = 1 re'resents t$o of the three roots2

P !ble*# "0 App ! che#: C #e# !+ /ltiple R!!t#&2 9rac?eting etho s fail locating x = 12

f(xl) f(xr ) > 0212 t x = 1, f(x) = f "(x) = 02

<e$ton- a'hson an Secant may ex'erience 'roblems2ate of convergence ro's to linear2

#uc?ily, f(x) 0 faster than f "(x) 0A2 ;ther reme ies, recasting 'roblem+

( ) ( )

( )f x

u xf " x

= = 0

u(x) an f(x) have same roots2

<;. + /F/ /h2 3 on U oots of 6olynomialsV is not covere in etail in thiscourse exce't the useful sections 3212& an 3232

#.0".02.01.0-"

-2

-1

0

1

2

"

-0.2 0.0 0.2 0.# 0.6 0.! 1.0

x

f ( x

)

-#000

-2000

0

2000

#000



S/** < R te# !+ C!"ve e"ce

i

mii-&

lim 0→∞

= >

m = &+ linear convergencem = 1+ qua ratic convergence

etho m9isection &:alse 6osition &Secant, mult2 oot &

< , mult2 oot &Secant, single root &2J&K Usu'er linearV

< , single root 1ccel2 < , mult2 oot (/F/, 42L) 1

NR #!l/ti!"# !+ */ltiv i te e7/ ti!"# ( C&C 4 th , '$3$2, p$ %33)

3e a'e n nonlinear e4uations, f = 0, = 1, ,n, eac *it nin&e en&ent 'aria les, x 7, 7 = 1, ,n. 3e see a solution for t eset of x 'alues t at si8ultaneousl satis es t e con&ition t at allf = 0.

:n 'ector an& 8atrix notation, t e $e*ton-%a son al orit8see s t e roots ; x < of ; f < = 0 a se4uence of a roxi8ations; x <0, ; x <1 , , ; x < i, ; x < i+1 , , in * ic all t e 'ectors in&icate&

races are of &i8ension n x 1 an& ; x <0 is a suita le initial

uess of t e roots. i'en t e it

esti8ate, *e use a rst-or&er a lor series ex ansion to see t e (i+1) st ?

{ }( ){ } { }( ){ } { } { }( )& &0

i i i ii

f f x f x x x

x+ +

∂ = + − ≈ ∂ (1)

@ere *e &e ne t e n x n s4uare 8atrix of artial &eri'ati'es

[ ]ii

f !

x∂ ≡ ∂ as t e Aaco ian of t e s ste8, an& t e ele8ent in

t e t ro* an& 7 t colu8n of J/ i is "

#

f x∂∂ . e su scri t i on J/ i

in&icates t at it is e'aluate& at ; x < i. 3e also &e ne t e c an ein t e esti8ate of t e root as { } { } { }&i i i

x x x+ − ≡ ∆ . erefore, * en

*e su stitute t ese notations into e4uation (1) an& rearran e,*e o tain si8ultaneous linear al e raic e4uations for t e i t iteration of t e $-% solution al orit 8?

[ ] { } { }( ){ }i ii ! x f x∆ = − (2)

3e sol'e for { }i x∆ , n& t e ne* esti8ate of t e root usin



A Th ee Ph #e R!!t+i"0i" St te

real rootfin ing 'roblem can be vie$ as having thee 'hases+

%)Opening moves : one nee s to fin the region of the 'arameter s'ace in $hichthe esire root can be foun 2

8hen using brac?eting metho s (bisection, false 'osition), this involves fin ingan interval $ith f(x l) f(x u) H 0 2

En erstan ing of the 'roblem, 'hysical insight, an common sense are valuablehere2 (*f it is feasible to gra'h the function, loo?ing at a gra'h hel's2)

Esing a 'o$erful <e$ton- a'hson algorithm to loo? for a root $here noneexists is a futile effort2

2) Middle !ame : Qere one uses a robust algorithm to re uce the initial region ofuncertainty so that the value of the root can be roughly etermine 2 *n one

imension, bisection is a goo strategy here2

*n multivariate 'roblems, a sequence of one- imensional line searches in thegra ient irection may $or?% minimi ing the sum of squares of the in ivi ualerrors (Uresi ualsV) allo$s an algorithm to Xu ge if 'rogress is being ma e2

6) "nd game : ;nce a relatively goo estimate of a root is foun , a <e$ton-a'hson or Secant algorithm can use use to generate a highly accurate

solution is a fe$ iterations2

.his may not have been 'ossible in ste' (1) because a linear a''roximation ofthe function may not be an a equate escri'tion of the function unless one is

very near the root2



Q/e#ti!": :or the A cases belo$ escribe the 'rogress of the <e$ton- a'hson metho $hensolving f i(x) = 0 as either linear, su'erlinear, qua ratic, ivergent, chaotic, stuc? near a singularity,or another a''ro'riate term, an ex'lain 8QM2

555 CASE A 555 555 C #e 1 555 555 C #e C 555- + (-) - + b(-) - + c(-)12000 -&12JHJ A2000 -K2000 02&00 &200 N01

A23&K 1&2&0K A2JJ3 -12A30 02&40 L2LL N0&A210J L24H& L2&&& -02301 02114 &2HK N0&A2010 02L01 L2L03 -0210K 02AAK K23K N00A2000 0200L L2J04 -020J1 0240J A2H0 N00A2000 02000 L23A3 -020&K 0234H &23A N00A2000 02000 L2K1L -02004 &2&AH 323& -0&A2000 02000 L2KKA -02001 &230H A2LA -0&A2000 02000 L2H11 02000 124JA &241 -0&A2000 02000 L2HLK 02000 A2KLL J233 -01A2000 02000 L2HJ4 02000 423J3 A20& -01A2000 02000 L2H33 02000 K2J40 &2AL -01

A2000 02000 L2HK4 02000 &12H34 42HL -0AA2000 02000 L2HH0 02000 &H2LJ1 12JL -0A

8hen <e$ton- a'hson is $or?ing $ell, one has qua ratic convergence $hen /#;S to theroot2 .his means that near to the root x r + (xr – xiN&) = G024f V(xr )Of Y(xr )I (x r – xi)

1 Qere the ifference bet$een the next estimate of the root x iN& <P the real root x r is

'ro'ortional to (x r –xi)12 .hus, once the errors (x r –xi) get small, they get very small very fast2 See

/ase 2 Mou can create your o$n exam'les $ith f Y(xr ) Z 02 Similarly $hen the secant metho is$or?ing $ell, one obtains su'erlinear convergence2 .his means the ex'onent is about &2J insteaof 1% but the 'erformance loo"s about the same2

t a multi'le root, the qua ratic convergence of <e$ton- a'hson fails an instea one

obtains linear convergence $hich means that (x r – xiN&) = g Y(ξ) (xr – xi) $here ξ is a 'oint$ithin the interval an x iN&= g(x i)2 .his occurs because the linear mo el < uses is no longergoo near the root2 .he f Y(xr ) in the enominator of the error ex'ression for the < methosuggests there $ill be trouble2 See case 92 .ry fin ing $ith < the root of (x-r) ? for integer ? > &2*n this case the < metho yiel s+ (x r – xiN&) = G& – (&O?)I (xr – xi)

<ear a 'ole or singularity $here f(x) infinity, the < algorithm can get stuc?+ it ta?esvery small ste's because the ste' length x iN& – x i = – f (x i) O f " (xi) can be 7 M small eventhough f(x) is large2 istinguishing feature of such cases is the large value of f "(x)2 lternativelyif the function in question is something li?e ex'(-x), $hich a''roaches ero as x ξ, the <algorithm can iverge in the sense that x continues to increase $ithout boun % but f(x i) oes getsmaller slo$ly See case /2 Pivergent behavior $oul also occur if x i oscillate bet$eenincreasing larger values2 G.ry f(x) = ln(x 1) – 023 = 0 for large x2I

/haotic behavior is obtaine if one searches for the root of f(x) = (x-1) 1 N 020&2 .ry it2 <can almost fin a root, an then it shoots 'ast an starts searching again from the other si e2;ther exam'les $ere 'rovi e in class2 %&ese 'e&aviors easily arise in m$ltivariate rootfinding

pro'lems and are t&en &arder to recogni e

section 2 rootfinding

Documents