the solutio on f transcendental equations* · the solution of transcendental equations 161 j u...

F

9

The solution of transcendental equations*

J. F. Traub Bell Telephone Laboratories

I, F U N C T I O N

O u r c h a p t e r c o n c e r n s t h e a p p r o x i m a t i o n of a zero off(x) o r , equ iva len t ly , t h e a p p r o x i m a t i o n pf a r o o t o f t h e e q u a t i o n f(x) = 0 b y i te ra t ive m e t h o d s . W e shal l t a k e / as t r a n s c e n d e n t a l a l t h o u g h t h e m e t h o d s w e discuss c a n a lso b e u s e d if / is a p o l y n o m i a l . I f w e use these m e t h o d s in th is l a t t e r case , we a r e n o t t a k i n g full a d v a n t a g e of t h e fact t h a t we a r e dea l ing w i th t h e special case o f a p o l y n o m i a l . !

W e res t r ic t ourse lves t o f(x) w h i c h a r e rea l s ingle-valued func t ions of a rea l va r i ab le , possess ing a ce r t a in n u m b e r of c o n t i n u o u s der iva t ives i n t h e n e i g h b o r h o o d of a rea l ze ro a. I n Sec t ion 2g, / is a vec to r func t ion of a vec to r va r i ab le . T h e n u m b e r of c o n t i n u o u s der iva t ives a s s u m e d var ies u p w a r d s f r o m ze ro .

A z e r o a is o f mul t ip l ic i ty m if

f(x) = (x- *rg(x)

* The author would like to thank Prentice-Hall for permission to use material from his book Iterative Methods for the Solution of Equations, which is referred to in the text and bibliography as [I].

t For method specifically aimed at the polynomial case see Chapter 10.

w h e r e g(x) is b o u n d e d a t a a n d g ( a ) is n o n z e r o . W e shal l a lways t a k e m a s a pos i t ive in teger . I f m = 1, a is sa id t o b e s i m p l e ; if m > 1, a is sa id t o b e n o n s i m p l e . I f a is n o n s i m p l e , i t is ca l led a mu l t ip l e ze ro . E x c e p t in Sec t ion 2f, we a s s u m e t h a t a is s imple .

2. M A T H E M A T I C A L D I S C U S S I O N

a. S y m b o l s U s e d

fU)(*)

a

Pn+1,8

c

/!/'(*) A z e r o o f / D o m i n a n t z e r o of g n + l i S

A s y m p t o t i c e r r o r c o n s t a n t = x, — a

/(*)

A ce r t a in family of i t e r a t i on func t ions

F u n c t i o n w h o s e z e r o is s o u g h t T h e inverse func t ion o f /

3=0

T h e inverse of t h e J a c o b i a n m a t r i x

I.F. I t e r a t i o n func t ion 160

The Solution of Transcendental Equations 161

Ju J a c o b i a n m a t r i x m T h e mul t ip l ic i ty of a n T h e n u m b e r o f p o i n t s a t w h i c h

o ld i n f o r m a t i o n is r eused p O r d e r Pn 8(t) H y p e r o s c u l a t o r y p o l y n o m i a l

f o r / <p N a m e of i t e r a t i on func t ion q>ns F a m i l y o f i t e r a t i o n func t ions

gene ra t ed b y inverse in ter p o l a t i o n

Q>ns F a m i l y o f i t e r a t i on func t ions g e n e r a t e d b y d i rec t in te r p o l a t i o n

Qn>s(t) H y p e r o s c u l a t o r y p o l y n o m i a l for

r = s(n + 1) s s — 1 der iva t ives o f / a r e u s e d

in m a n y i t e r a t i on func t ions

"(*) = /(*)//'(*) xt A p p r o x i m a n t t o a

b. Bas ic Concept s

L e t xi9 xt_l9 • • •, x{_n b e n + 1 a p p r o x i m a n t s t o a . L e t x i + 1 b e un ique ly d e t e r m i n e d b y i n f o r m a t i o n o b t a i n e d a t xi9 xt_l9 • • • , # t _ n . T h e i n f o r m a t i o n u s e d is t h e va lues of / a n d ce r t a in of i ts der iva t ives a t xi9 xt_l9 • • • , x{_n. L e t t h e func t ion t h a t m a p s xi9 xf_l9 • • •, x ^ n i n t o x i + 1

b e ca l led (p. T h u s

xi+l = Wipii Xi-U ' ' ' 9 xi-n)

W e cal l cp a n iteration function. The abbreviation IJF. will henceforth be usedfor iteration function and its plural. W e shal l use t h e a b b r e v i a t i o n

fll) = f{l)(xt) for t h e 7 t h de r iva t ive o f / a t xi%

T h e a r g u m e n t s t r ing o f <f> s h o u l d a l so i nc lude s y m b o l s r ep re sen t ing va lues o f / a n d i t s der iva t ives s ince these n u m b e r s a r e u sed in t h e ca l cu la t ion o f x i + 1 . F o r s impl ic i ty we neve r l ist t he se s y m b o l s explici t ly .

T w o I . F . a r e a l m o s t universa l ly k n o w n . T h e y a r e t h e N e w t o n - R a p h s o n I . F . a n d t h e secan t I . F . given respect ively by

Si <p(*i) = x i - y , (*)

Ji

rtafeaV-i)-*!-//?"?-1!, fi^ft-x (2) -Ji ~~Ji-l-*

W e shal l see t h a t these a r e t h e s imples t e x a m p l e s of t w o genera l classes o f i t e r a t i o n func t ions .

W e shal l a l so i n t r o d u c e a t h i r d genera l class of i t e r a t i on func t ions .

W e shal l classify I . F . b y t h e i n f o r m a t i o n wh ich t h e y r equ i r e . L e t x i + 1 b e d e t e r m i n e d on ly b y n e w i n f o r m a t i o n a t xt. N o o ld i n f o r m a t i o n is r eused . T h u s

xi+i = <p(xi) (3)

T h e n <p will b e cal led a one-point I.F. M o s t ; I . F . wh ich h a v e b e e n u s e d for roo t - f ind ing are? o n e - p o i n t I . F . T h e m o s t c o m m o n l y k n o w n e x a m p l e is t h e N e w t o n - R a p h s o n I .F . ;

N e x t let x i + l b e d e t e r m i n e d b y n e w infor

m a t i o n a t xi a n d r eused i n f o r m a t i o n a t

" ' 9

xi-w T h u s

= <P(Xi; xi-l> Xi-n) (4)

T h e n <p will b e ca l led a one-point I.F, with memory. T h e semico lon in (4) s epa ra t e s t h $ p o i n t a t wh ich n e w d a t a a r e u s e d f r o m t h e p o i n t s a t wh ich o ld d a t a a r e r eused . T h i s t y p e of I . F . is n o w of special in te res t s ince t h e o ld i n f o r m a t i o n is easily saved in t h e m e m o r y of a c o m p u t e r . T h e case o f p rac t i ca l in te res t i t w h e n t h e s a m e i n f o r m a t i o n , t h e va lues o f / a n d

/ for e x a m p l e , is u s e d a t all p o i n t s . T h e bes t -k n o w n e x a m p l e of a o n e - p o i n t I . F . w i th m e m o r y is t h e secan t I . F .

Le t x i + 1 b e d e t e r m i n e d b y n e w i n f o r m a t i o n a t xi9 c o ^ . ) , c o 2 ( ^ ) , • • • , cok{x^9 k>\» ( T h e cQj(x) a r e func t ions d e p e n d i n g o n / a n d i ts der ivat ives . E x a m p l e s m a y be f o u n d in Sec t ion 2e.) N o o ld i n f o r m a t i o n is r eused . T h u s

xi+i = <p[*p <»i(xi\ • ' ', ^(^)]

T h e n cp is cal led a multipoint I.F. T h e r e a r e n o we l l -known example s o f m u l t i p o i n t l . F . T h e y were i n t r o d u c e d in [I] b e c a u s e o f the i r va lue in avo id ing ce r t a in l imi t a t ions o n o n e - p o i n t I . F . a n d o n e - p o i n t I . F . w i th m e m o r y .

A f o u r t h c lass , m u l t i p o i n t I . F . w i th m e m o r y , is i n t r o d u c e d o n p . 9 of [I] . T h i s class is o f less in te res t a n d will n o t b e d iscussed fu r the r h e r e .

W e t u r n t o t h e i m p o r t a n t c o n c e p t of t h e o r d e r of a n I . F . Le t x09 xl9 - • •, xi9 • • • b e a sequence conve rg ing t o a. L e t ei = xt — a. I f t h e r e exists a rea l n u m b e r p a n d a n o n z e r o c o n s t a n t C such t h a t

^ i - > C (5)

162

*i+l

I f 9? gene ra tes a sequence of o r d e r p, we say 99 is of o r d e r p.

W e shal l see t h a t t h e o r d e r of a n I . F . d e p e n d s o n t h e mul t ip l ic i ty o f a. I t m a y a lso d e p e n d o n / .

Example: Le t a b e a s imple ze ro . L e t a sequence of a p p r o x i m a n t s b e g e n e r a t e d b y t h e N e w t o n - R a p h s o n I . F . T h e n , if / " ( a ) 5* 0,

^+1 (7) e? 2/ ' (a)

While, i f / " (a ) = 0 , / ' " ( a ) # 0,

e i + 1 /"(«)

*8 ~ V ( a ) H e n c e t h e N e w t o n - R a p h s o n I . F . is of

s e c o n d o r d e r for all func t ions w h o s e second der iva t ive d o e s n o t van i sh a t t h e ze ro . O t h e r -Wise, i t is o f a t leas t t h i r d o rde r .

A s th i s e x a m p l e s h o w s , we s h o u l d say t h a t a Cfertain I . F . h a s o r d e r at least p. H o w e v e r , we shal l c o n t e n t ourse lves wi th s imply say ing t h e I . F . is o f o r d e r p.

c. O n e - P o i n t I terat ion Funct ions

General Theory W e s t u d y sequences g e n e r a t e d b y o n e - p o i n t

I . F . , x i + 1 = cp{x?). O n e s imple w a y t o o b t a i n <p is t o rewr i t e f(x) = 0 a s x — cp(x). H o w e v e r , s u c h a n I . F . is i n genera l o f on ly first o rde r .

A s s u m e (p(p) is c o n t i n u o u s i n t h e n e i g h b o r h o o d of a. L e t 99(a) = a; <p ( i )(a) = 0, j = 1, 2 , • • • , / ? - 1 ; <p ( p )(a) 5* 0. By T a y l o r ' s t h e o r e m ,

( z , - a ) » (8)

w h e r e f f l ies i n t h e in t e rva l s p a n n e d b y x{ a n d a. F r o m th i s t h e fo l lowing t h e o r e m fol lows easily.

Theorem: L e t <p b e a o n e - p o i n t I . F . a n d let xp{v) b e c o n t i n u o u s in a n e i g h b o r h o o d of a. T h e n cp is o f o r d e r p if a n d on ly if

cp{0L) = a, 99<>'>(a) = 0,

/ = 1 ,2 , • • • , / > - 1 ; <p<*>(a )^0

Mathematical Methods for Digital Computers

F u r t h e r m o r e

I t m i g h t seem t h a t th is is on ly a n exis tence t h e o r e m , t h a t is, t h a t i t w o u l d b e difficult ac tua l ly t o c o n s t u c t I . F . s u c h t h a t q>U)(<x) = 0 for all / . A s we shal l see (p . o n e c a n give a n u m b e r of a l g o r i t h m s for gene ra t i ng I . F . o f all o rde r s .

C o n d i t i o n s wh ich g u a r a n t e e t h a t xi —• a a r e given in t h e fo l lowing :

Theorem: Le t cp{p) b e c o n t i n u o u s in t h e in te rva l / ,

L e t / = {x j \x - oc| < \x0 - a|}

\<P™(x)\ <M (9)

for all x g / , a n d let

M \ e ^ < \ , e0 = x 0 - o i (10)

T h e n xi g / , i = 0, 1, • • • , a n d xi - > a.

Proof: C lear ly x0 g / . A s s u m e xi e J. R e wri te (8) as

^ ( f i ) e i + 1 = Mt = (11)

T h e n by (9), (10), (11), a n d t h e defini t ion o f / ,

\ei+1\ < M \et\^ \e€\ < M \e0\^ \e0\ £\e0\

H e n c e x i + 1 e J a n d we c o n c l u d e t h a t

\ei+1\ < M k J * > , / = 0 , l , . "

wh ich impl ies

k , | < M ^ - * ^ 1 \e0\pt

H e n c e

Wi\ < m - [ i / ( 3 > - i ) ] ( m w Y ^ - 1 ] p > 1

(12)

leJ^Afkol, p = l (13) W e c o n c l u d e f r o m (10) t h a t et - > 0.

O b s e r v e a l so t h a t (12) a n d (13) p r o v i d e b o u n d s o n t h e e r r o r af ter t h e rth s tep . O b s e r v e t h a t if p = 1 ( l inear o r first-order convergence)* t h e c o n d i t i o n for c o n v e r g e n c e is \q>\ < 1 in t h e n e i g h b o r h o o d of a. I f p > 1 ( supe r l inea r convergence ) , t h e r e is a lways a n e i g h b o r h o o d of & for w h i c h t h e i t e r a t i on converges .

t h e n p is ca l led t h e order of t he sequence a n d C is cal led t h e asymptotic error constant. I f p is a n in teger , we define C by

C (6)


T h e r e a d e r fami l ia r wi th I.F. h a s n o d o u b t obse rved t h a t a s e c o n d - o r d e r m e t h o d r equ i r e s t h e e v a l u a t i o n of / a n d / ' , a t h i r d - o r d e r m e t h o d r equ i re s t h e e v a l u a t i o n of / , / ' , / " , e tc . T h i s sugges ts t h a t p e r h a p s I.F. of o r d e r p a lways r e q u i r e a t l eas t p f unc t ion e v a l u a t i o n s a n d in fact r e q u i r e t h e e v a l u a t i o n of a t leas t / ,

/ ' > " • > / ( p _ 1 ) - T h a t th i s is i n d e e d t h e case for one-point I.F. is t h e essence of t h e fundamental theorem of one-point I.F. ( T h e o r e m 5-3, p . 98 in [I]). T h i s t h e o r e m is o n e o f t h e p r inc ipa l r e a s o n s for s t u d y i n g o n e - p o i n t I.F. w i t h m e m o r y a n d m u l t i p o i n t I.F. w h e r e this res t r ict i o n d o e s n o t app ly .

Some One-Point I t e r a t i o n Func t ions T h e fo l lowing is a gene ra l m e t h o d for de r iv ing

b n e - p o i n t I.F. o f a n y o r d e r . L e t / ' be n o n z e r o i n a n e i g h b o r h o o d of a a n d le t f(s) toe c o n t i n u o u s in th i s n e i g h b o r h o o d . T h e n / h a s a n inverse

a n d ^ { s ) is c o n t i n u o u s ift a n e i g h b o r h o o d o f z e r o . L e t q(t) b e t h e p o l y n o m i a l w h o s e first s — 1 der iva t ives agree wi th ^ a t t h e p o i n t y = f(x). - T h e n

T h e n

^ ( 1 ) = , ( ) ) + ^ p ( ,

s-l ,

3=0

(14)

(t - y)j

w h e r e 0(0 lies in t h e in te rva l s p a n n e d by y a n d t. Def ine

£ s = 4(0)=lV<%)^ 3=0 j !

I t is easy t o s h o w t h a t Es is of oidei* s. ( T h e s e I . F . a r e s tud i ed in de ta i l o n p p . 7 8 - 8 8 of [I].) I n pa r t i cu l a r ,

E2 = x — u ( N e w t o n - R a p h s o n )

E3 = E2 — A2u*

E, = £3 - (2A2

2 - Az)u* Svhere

Hi)

ilf Example : C o n s i d e r f(x) = xn — A, w i t h n

a n in teger . I f n > 2 , th i s l eads t o a f o r m u l a for nth r o o t s ; if n = — 1, th is l eads t o a f o r m u l a for t h e r ec ip roca l of A. W e h a v e

&(y) = (A + y)lln

&^\y) = a ; 1 " ^ ! ! ( i - k) k^o \n /

s-l i l a 7l\3 3-1

Ee = x + x2 LlA—3L\ U (1 - kn) 3=1 j ! \ nxn 1 k=o

I n pa r t i cu la r ,

*-;[- , +?] If « = - 1 ,

^ = ^2(1 - Ax)'

3=0

3=0

T h e geome t r i c series converges t o 1/A if |1 < 1. T h e r e a r e a n u m b e r of o t h e r genera l m e t h o d s for der iv ing o n e - p o i n t I . F . of a n y o rde r . U s i n g di rec t i n t e r p o l a t i o n t o gene ra t e a n sth-o r d e r I . F . r equ i re s t h e so lu t i on o f a n a lgebra ic e q u a t i o n of degree s — 1. I t a l so ra ises t h e p r o b l e m of en s u r i n g t h a t t h e i n t e r p o l a t o r y p o l y n o m i a l possesses a rea l z e ro . T h e s e m a t t e r s a r e d iscussed o n p p . 6 7 - 7 7 , 9 2 - 9 7 of [I]. T h e s e c o n d - o r d e r case is aga in N e w t o n - R a p h s o n , T h e t h i r d - o r d e r case , wh ich is given b y t h e so lu t ion of

0 = / ( * ) +f'(x)(t - x) + if'(x)(t - xf

h a s s o m e in te res t ing charac te r i s t i c s . F o r c o m p u t a t i o n a l p u r p o s e s t he I . F . is be s t wr i t t en a s

2u. cp = x —

1 ± (1 - 4A2u) Tf2 (15)

A n o t h e r i m p o r t a n t family of o n e - p o i n t I . F . m a y be o b t a i n e d f rom t h e coefficients of t h e e x p a n s i o n "of 1 / / i n t o a T a y l o r series. T h e in te res ted r e a d e r m a y consu l t p p . 103-106 , 123-125 of [3]. T h e t h i r d - o r d e r I . F . of th is family is Ha l l ey ' s often red i scovered I . F .

cp = x — 1 — A2u (16)

d. O n e - P o i n t I terat ion Functions W i t h M e m o r y

W h e n a o n e - p o i n t I . F . h a s r e a c h e d t h e p o i n t xi9 i t forgets t h e va lues of / a n d i ts der iva t ives a t ear l ier p o i n t s . B u t th is is j u s t t h r o w i n g a w a y v a l u a b l e i n f o r m a t i o n . W e t u r n t o t h e s t u d y of h o w t o r euse th i s o l d i n f o r m a t i o n . Since t h e t e c h n i q u e s w e e m p l o y

164 Mathematical Methods for Digital Computers

d e p e n d heavi ly o n i n t e r p o l a t i o n t h e o r y , we beg in w i th a s h o r t review of th i s subject .

I n te rpo l a t i on Theory

Let / be given a n d let s be a pos i t ive in teger a n d n a n o n n e g a t i v e in teger . Le t r = s(n + 1). Le t xi9 x{__l9 • • , xt_n be n + 1 rea l n u m b e r s a n d let f(r)(t) be c o n t i n u o u s in t h e in te rva l s p a n n e d b y xi9 xt_l9 • • • , t. T h e r e exists a u n i q u e p o l y n o m i a l Pn>s(t) s uch t h a t

j = 0, 1, • • • , n; k = 0, 1, • • • , s — 1 a n d

/(f)[«01 A /(0 = Pn,S(0+^ r ! j=o

w h e r e i{(t) lies in t h e in te rva l s p a n n e d by

Le t / ' be n o n z e r o a n d let f(r) be c o n t i n u o u s o n a n in te rva l / . L e t / m a p / i n t o K. T h e n / h a s a n inverse J 5", a n d ^ ~ ( r ) is c o n t i n u o u s o n K. L o t Vi = f(Xi)- T h e r e exists a u n i q u e p o l y n o m i a l 2WjS(0 such t h a t

j = 0, 1, • • • , n; k = 0, 1, • • • , s - 1 a n d

= On A) + ^ T ) [ 9 m IT (/ -r! j=o

(18)

Where 0 (0 ^ e s m the i n te rva l s p a n n e d by Vi, V i - i * , Vi-n> *. E q u a t i o n (14) is j u s t t h e qase n = 0.

T h e r e a r e t w o w e l l - k n o w n r e p r e s e n t a t i o n s of a n i n t e r p o l a t o r y p o l y n o m i a l given b y t h e N e w t o n i a n a n d L a g r a n g e - H e r m i t e f o r m u l a t i o n s . T h e r e p r e s e n t a t i o n s a r e c o m p a r e d o n p . 241 of [1].

L e t / f o , yQ; x ^ yx;. . . ; x{_n9 yn] d e n o t e a conf luen t d iv ided difference w h e r e x ^ o c c u r s Yj t imes . If yi? = 1, it m a y be omi t t ed . T h u s

f[xi9 1 ; x{_l9 1; . . . ; x{_n9 1]

= flxii Xi-1> * * * ' Xi-n\

T h e N e w t o n i a n r e p r e s e n t a t i o n is given by

P « A ) = 2 l c u t )

5=01=0 x f[xn s; xt_l9 s; . . . ; x{_j91 + 1] (19)

cut) = ( t - xi-,)1 ff (' - x ^ y , fi b. i

k=Q 0

If j = 0, t h e n t h e d iv ided difference is t o b e i n t e rp re t ed as f[xi9 / + 1].

I n t h e L a g r a n g e - H e r m i t e r ep re sen t a t i on , t h e i n t e r p o l a t o r y p o l y n o m i a l is expressed as a l inear c o m b i n a t i o n of t h e f ^ ) 9

3=01=0

T h e f o r m u l a s for s = 1 a n d 2 a r e well k n o w n a n d a re given by the f o r m u l a s of L a g r a n g e a n d H e r m i t e . T h e f o r m u l a s for s a r b i t r a r y m a y b e f o u n d in T r a u b [5]. A d d i t i o n a l m a t e r i a l o n i n t e r p o l a t i o n a n d n u m e r o u s e x a m p l e s a r e given in [I] , A p p e n d i x A .

I n t e r p o l a t o r y I t e ra t i on Funct ions : Inverse

Let xi9 x{_l9 • • •, x ^ n b e n + 1 a p p r o x i m a t i o n s t o a. L e t Qns{t) be t h e i n t e r p o l a t o r y p o l y n o m i a l for a t t h e p o i n t s yi9 y{_l9 • • •, yt_n. Def ine a n e w a p p r o x i m a t i o n t o a b y

* » i = fi.,.(0) (20>

T h i s defines a n i t e r a t ion

Xi+1 — Vnjfci* xi-\-> ' ' ' J xi-n)

R e p e a t th is p r o c e d u r e us ing p o i n t s x i + l 9

xi9 • • • , xt_n+l9 a n d so o n . W e n o w e x a m i n e t h e r e l a t i on of t h e e r r o r of x i + 1 t o t h e e r r o r o f t h e n + 1 p o i n t s w h i c h d e t e r m i n e it.

Set t = 0 in (18). T h e n , s ince a = ^ ( 0 ) ,

(-Dr

r\

w h e r e 0{ = 0,(0). Le t = - a. T h e n Vi-i =ff(Vi-j)^i-j w h e r e ^ w lies in t h e in te rva l s p a n n e d by x ^ a n d a. W e c o n c l u d e t h a t

e t + i = ^ n

0=0

M , = -(21)

HII O*-,)]' w h e r e = / ( ^ _ , , ) .

Ana lys i s o f th is e q u a t i o n l eads t o t h e fol lowing t h e o r e m ( [ I ] , T h e o r e m 4 - 1 , p . 66) w h i c h gives c o n d i t i o n s u n d e r w h i c h t h e i t e r a t i on converges a n d gives i ts o rde r .

Theorem : L e t

J = { s | | * - a | ^ T }


L e t r = s(n + 1) > 1. L e t f(r) b e c o n t i n u o u s a n d l e t / ' 5 * 0 o n / . L e t x09 xl9 - < 9 x n e J a n d define a sequence of xt a s desc r ibed a b o v e . L e t

r !

for all x e J9 le t M = A J V * a n d s u p p o s e t h a t M r 7 - - 1 < 1.

T h e n x%eJ for all 1 a n d xi —>• a. F u r t h e r m o r e

Table I. Values of pn+i,s

w h e r e /? is t h e u n i q u e rea l pos i t ive r o o t of

t n + 1 - s 2 t j = 0

a n d

TO = -

i=0

r\[&'(y)Y

T h e o r d e r of these I . F , is t h u s given by t h e u n i q u e rea l pos i t ive r o o t of

fn+l

0=0

T h e b e h a v i o r of th is r o o t is s u m m a r i z e d by t h e following t h e o r e m w h i c h is a special case of T h e o r e m 3-2 (p . 51 in [I]),

Theorem : L e t

g n + M ( 0 = ' n + 1 -*i<' = 0 0=0

I f n = 0, th i s e q u a t i o n h a s a rea l r o o t f}x s = L e t « > 0. T h e e q u a t i o n h a s o n e rea l pos i t ive s imple r o o t j 8 n + 1 > 8 a n d

S < Pn+l,s < s + 1 F u r t h e r m o r e

es s + 1 -

+ d n+1

< S + 1 -(s + i y + 1

w h e r e e is t h e b a s e of n a t u r a l l o g a r i t h m s . H e n c e l im n - > 00 j 8 n + 1 > 8 = .s + 1. Al l o t h e r r o o t s a r e a l so s imple a n d h a v e m o d u l i less t h a n o n e .

Va lues of fin+liS for l o w va lues of n a n d s m a y b e f o u n d in T a b l e 1. O b s e r v e t h a t t h e o r d e r a p p r o a c h e s i ts l imi t ing va lue qu i t e r ap id ly as a func t ion of n9 pa r t i cu la r ly for la rge s. T h e case o f m o s t p rac t i ca l in teres t is H = L T h e n (22)

s n 1 2 3

0 1.000 2.000 3.000 1 1.618 2.732 3.791 2 1.839 2.920 3.951 3 1.928 2.974 3.988 4 1.966 2.992 3.997

m a y be solved exact ly , a n d

ft,. = il* + (*2 + 4*)*]

Obse rve t h a t r = s(n + 1) r ep resen t s t h e p r o d u c t of t h e n u m b e r of n e w pieces of i n f o r m a t i o n p e r i t e r a t i on w i th t h e n u m b e r o f p o i n t s a t wh ich i n f o r m a t i o n is u sed . T h i s n u m b e r p lays a u b i q u i t o u s ro le in o u r t h e o r y ,

E x a m p l e s of i n t e r p o l a t o r y I . F . m a y b e f o u n d o n p p . Jjjjj-JJJ.

I n t e r p o l a t o r y I t e r a t i on F u n c t i o n s

D i r e c t

L e t xi9 xt_l9 • • • , xt_n b e n + 1 a p p r o x i m a t i o n s t o a ze ro a o f / . L e t Pn)S(t) b e t h e i n t e r p o l a t o r y p o l y n o m i a l for / a t t h e p o i n t s xi> xi-i>'*' 9 xi-n- Def ine a n e w a p p r o x i m a t i o n t o a b y

Pn,s(xi+l) = 0 (23)

T h e n r e p e a t th i s p r o c e d u r e for x i + l 9 xi9 • • • f

xi-n+i. T h i s defines a n i t e r a t i on

xi+l = ®n,s(Xi> Xi-l9 ' ' ' J xi~n)

Since P n s is a p o l y n o m i a l of degree r — 1> xi+i w i H n o t genera l ly b e u n i q u e l y specified b y (23). I t is n o t even c lear a p r io r i t h a t P n s h a s a rea l ze ro in t h e n e i g h b o r h o o d of a. W e shal l s ta te a l e m m a ( [ I ] , L e m m a 4-2, p . 68) w h i c h gives c o n d i t i o n s wh ich ensu re t h a t P n s d o e s h a v e a rea l z e r o in t h e n e i g h b o r h o o d o f a.

L e t J = {x\\x - <x.\ <T} a n d let xi9

xt_l9 • • • , Xi_n g / . Le t / ' b e n o n z e r o o n / . If the x^j bracket a, then it is clear that P n s has a real zero in J. H e n c e it is sufficient t o inves t iga te t h e case w h e r e all t h e x ^ lie o n o n e s ide of a;

Lemma: / = {X\ \x - | a < T}

Le t f(r) be c o n t i n u o u s o n J a n d l e t / V 0 o n / . L e t xi9 xt_l9 • • •, x ^ n eJ a n d let xi9 xt_l9 • • • 5

I$6 Mathematical Methods for Digital Computers

x{_n lie o n o n e s ide of a. Le t

i/<r,i < vlt \f'\ > v2

< 1

for al l xeJ. S u p p o s e t h a t

» i ( 2 T ) '

v, r T h e n P n s h a s a rea l r o o t , x i + l 9 wh ich lies in / .

T o der ive t h e e q u a t i o n wh ich gove rns t h e e r r o r s , we set t = a in (17). T h e n

r ! F r o m

^n.s(a) = (a - ^+ i ) P ; , s (^ 4 + i )

w h e r e rji+1 lies in t h e in te rva l s p a n n e d by a a n d x i + l a n d , a s s u m i n g t h a t P ^ d o e s n o t van i sh in th is in te rva l , we c o n c l u d e t h a t

ei+1 = Hi+1 n 4 - , 3=0 (24)

Hi+i = (~l)r /"(ft)

U s i n g th i s e r r o r e q u a t i o n , o n e c a n give a c o n v e r g e n c e p r o o f for I . F . g e n e r a t e d by d i rec t i n t e r p o l a t i o n ( [ I ] , T h e o r e m 4 -3 , p . 73). W e d o n o t r e p r o d u c e t h e s t a t e m e n t of th is t h e o r e m he re b u t confine ourse lves t o t he conc lu s ion t h a t

(p-l)/(r-l)

w h e r e p is t h e u n i q u e real pos i t ive r o o t of

a n d w h e r e tn+l _ s ^ = q

3=0

f (r)

T h i s c o n c l u s i o n s h o u l d b e c o m p a r e d wi th t he c o n c l u s i o n of t h e T h e o r e m o n p . JJJ.

T h e l e m m a a b o v e i m p o s e s a res t r i c t ion o n t h e size of t h e in te rva l in w h i c h i n t e r p o l a t i o n m u s t t a k e p lace in o r d e r t h a t t h e i n t e r p o l a t o r y p o l y n o m i a l h a v e a rea l z e ro . By i m p o s i n g a c o n d i t i o n o n / we c a n r e m o v e t h e size res t r ict i o n o n t h e in te rva l . W e h a v e t h e fo l lowing l e m m a ( [ I ] , L e m m a 4 -3 , p . 69) a n d t h e o r e m ( [ I ] , T h e o r e m 4-2, p . 71).

Lemma: Le t j = {x | \x - <x| < r>

L e t / < r ) be c o n t i n u o u s o n / a n d let f'f{r) j± O o n L e t x09 xl9 • • • , xn e J a n d a s s u m e t h a t t he se

p o i n t s all lie o n o n e side o f a. Le t these p o i n t s b e labe led such t h a t xn is t h e closest p o i n t t o a. Le t

f(*n)f{r)M>0, ^ v e n

f'Mfir)M < 0, r o d d

T h e n Pn s h a s a real r o o t x n + 1 s uch t h a t

m i n [a, xn] < x n + 1 < m a x [a , xn]

Theorem: L e t / = { * | | * - a | < n

Let r = s(n + 1) > 1. Le t f{r) b e c o n t i n u o u s o n / a n d let f'f(r) ^ 0 o n / . Le t x0, xl9 - • •, xn e / a n d a s s u m e t h a t these p o i n t s all lie o n o n e side of a. S u p p o s e t h a t

/ W ( r ) W > 0 , r e v e n

/ W ( f )(^ < 0, r o d d w h e r e i is a n y of 0, 1, • • • , n. Le t xi9 i = n + 1, n + 2, • • • , be a p o i n t (whose exis tence w e h a v e verified in t h e p r ev ious l e m m a ) s u c h t h a t Xi is rea l , Pni8{x^ = 0 , a n d xt l ies i n t h e in t e rva l s p a n n e d by xt_x a n d a. T h e n { x j converges m o n o t o n i c a l l y t o a.

T h i s resul t is well k n o w n for t h e case n = 0, s = 2, w h i c h is N e w t o n - R a p h s o n . F o r th i s case t he c o n d i t i o n s , wh ich a r e k n o w n as F o u r i e r c o n d i t i o n s , a r e geomet r ica l ly self-ev ident . T h e t h e o r e m gives a sweep ing gene ra l i za t ion of F o u r i e r ' s resul t .

Other Families of One-Point Iteration Functions With Memory I n t h e p r eced ing t w o sec t ions we deve loped

t h e t h e o r y of i n t e r p o l a t o r y I . F . T h e r e a r e o t h e r i m p o r t a n t m e t h o d s for g e n e r a t i n g o n e -p o i n t I . F . wi th m e m o r y . O n e genera l m e t h o d is t o e s t ima te t h e h ighes t der iva t ive in a o n e - p o i n t I . F . T h e t h e o r y of de r iva t ive -es t imated I . F . is g iven in [ I ] , Sec t ion 6.2. (See especia l ly T h e o r e m s 6-1 t o 6-4.)

W e inves t iga te o n e - p o i n t I . F . w i th m e m o r y because t h e fundamental theory of one-point


I.F. tells u s t h a t a m e t h o d of o r d e r p u s ing on ly n e w i n f o r m a t i o n r equ i re s p eva lua t i ons . F o r t h e famil ies of o n e - p o i n t I . F . w i th m e m o r y t h a t we h a v e inves t iga ted , we find t h a t t h e o ld i n f o r m a t i o n a d d s less t h a n un i ty t o t h e o rde r . T h a t th i s res t r i c t ion h o l d s for all o n e - p o i n t I . F . wi th m e m o r y is t h e essence of a con jec tu re r e g a r d i n g o n e - p o i n t I . F . w i th m e m o r y ( [ I ] , p . 124). A special case of th is con jec tu re is t h a t t he r e exists n o o n e - p o i n t I . F . w i th m e m o r y w h i c h is o f s econd o r d e r a n d wh ich d o e s n o t r e q u i r e t h e e v a l u a t i o n of a n y der iva t ives . W h e n we t u r n t o m u l t i p o i n t I . F . in Sec t ion 2e we find n o such res t r ic t ion .

Examples of One-Point Iteration Function With Memory T o use a n I . F . such as cpn s9 n + 1 a p p r o x i -

m a n t s t o a m u s t be avai lab le , T h i s sugges ts u s ing cp0s wh ich r equ i re s b u t o n e a p p r o x i m a n t , fo l lowed successively b y cpls9 • • •, <pns a t t h e beg inn ing of a ca lcu la t ion .

I n t h e fo l lowing e x a m p l e s t h e n o t a t i o n for t h e n a m e of t h e I . F . is d u e t o a genera l classific a t i o n scheme . ( T h e in te res ted r e a d e r is refer red t o [ I ] , C h a p t e r 6.)

Example (see a lso p .

<Pi.i = xi - , r .

f[Xi> Xi-l\

T h i s is t h e w e l l - k n o w n secan t m e t h o d for

which /> = £(!+ V5)~ 1.62, C = [ I ^ a ) ^ - 1 .

Example: L

= xt Xi-l\ Xi-2l -f[Xi-l, Xi-2]

T h i s I . F . h a s p ~ 1.84, C = M3(a)|<*-1 ) / 2. I t is a n e x a m p l e of a der iva t ive e s t ima ted I . F . m e n t i o n e d o n p .

5_ Vi

co ± {co2 - 4ff[xi9 x{_l9 xt_2]}*

co =f[xi9 x^] + (xt - Xi_Jf[xi9 xt_l9 x{_2\

T h i s I . F . differs on ly in f o r m f rom M u l l e r ' s I . F . , ove r w h i c h it en joys a n u m b e r of a d v a n t ages . (See [ I ] , p p . 211-212 . ) W e h a v e / 7 ~ 1.84, C = M 3 (a) | ( p - 1 ) / 2 .

Example (see a lso p .

#2,1

Example: w-2

*£1>2 = xt - u- - -~ Ji

—-1—{2x+fu-m*i,*^i]}\

\-xi - xi~i j = fjfi

T h i s I.F., wh ich r equ i re s n o m o r e eva lua t i ons t h a n t h e m e t h o d of N e w t o n - R a p h s o n , h a s

— 2.73, C = M 4 ( a ) p " 1 ) / 3 . I t is a lso a n e x a m p l e of a der iva t ive e s t ima ted I.F.

e. Mu l t i po in t I terat ion Functions

The fundamental theorem of one-point I.F. i m p o s e s t h e " r e s t r i c t i o n " t h a t o n e - p o i n t I.F. of o r d e r p r equ i r e t h e e v a l u a t i o n of t h e first p — 1 der iva t ives of / . T h i s res t r i c t ion is rel ieved in on ly a l imi ted w a y w h e n o n e tu fns t o o n e - p o i n t I.F. w i th m e m o r y . T h i s r es t r i c t ion d o e s n o t h o l d for m u l t i p o i n t I.F., t h a t is , for I.F. w h i c h s a m p l e / a n d its der iva t ives a t a n u m b e r of va lues of t h e i n d e p e n d e n t va r i ab le , T h e p o w e r of m u l t i p o i n t I.F. seems t o lie in t h e felici tous choice of s a m p l e p o i n t s , a c h a r a c t e r istic wh ich they sha re wi th R u n g e - K u t t a m e t h o d s .

T w o Examples of Multipoint Iteration Functions O n e m e t h o d of c o n s t r u c t i n g a n I.F, of o r d e r

p + 1 wh ich requ i res p eva lua t i ons o f / a n d o n e of / ' is b a s e d o n t h e fo l lowing t h e o r e m ( [ I ] , T h e o r e m 8-1 , p . 166).

Theorem: Let cp be of t h e o r d e r p. T h e n

tp(x) = cp{x)

is of o r d e r p + 1. H e n c e if w e define

/ [?(*)]

/ (* )

t h e n <pv(%) is o f o r d e r p. I n pa r t i cu l a r ,

f[x- u(x)] f(x) <p3 = x — u(x) —•777-^ , u(x) =

is t h i r d o rde r .

a*) (25)

168 Mathematical Mettxds for Digital Computers

A n in te res t ing e x a m p l e of a m u l t i p o i n t I.F. is furn ished by

lf[x- u(x)] -f(x)\ cp(x) = x - u(x)\— — — (26)

12/[a; - u(x)] -f(x))

T h i s uses t w o eva lua t ions o f / a n d o n e o f / ' , b u t is o f o r d e r 4. T h i s I.F. ha.s the fo l lowing n e a t geome t r i c i n t e rp re t a t i on . Le t z = x — u(x). Le t P be t h e p o i n t wh ich bisects that s egmen t of t h e t a n g e n t l ine at [x,f(x)] wh ich lies be tween [x,f(x)] a n d [z, 0]. T h e n <p(x) is t he in te r sec t ion wi th t he a>axis of t he l ine t h r o u g h P a n d [zj(z)].

T w o Families of Multipoint Iteration Functions I n [I] ( C h a p t e r 9) we i n t r o d u c e d t h e family of

IvF. def ined by

Where (p(x) = x - 2 ak

v(ok\x)

io//(x) = f]

3=1

a>^{x) = u(x), o

2 = S o m e of t h e p a r a m e t e r s a r e fixed b y t h e c o n d i t ion t h a t cp be of o r d e r p whi le t h e r e m a i n i n g p a r a m e t e r s a r e c h o s e n t o ga in a d d i t i o n a l des i rab le charac te r i s t i cs . T h i s family of I . F . r equ i res p — I eva lua t i ons o f / a n d o n e o f / ' .

W e a l so i n t r o d u c e d a s econd family of I . F . def ined b y

k=l

3=1

w h e r e

Q*%«0 =/(*)//' Q/(z) = u(x)

Since th i s fami ly r equ i r e s p — I e v a l u a t i o n s of / ' a n d o n e e v a l u a t i o n of / , i t is well su i t ed for finding t h e ze ros of func t ions defined b y in tegra l s .

T h i s I . F . h a s a n in te res t ing i n t e r p r e t a t i o n . I t is easy t o s h o w t h a t ip(x) is o f supe r l i nea r o r d e r on ly if ZjJzJ ck

p = I. Def ine A(x) by

N o t e t h a t l/A(x) is the weighted m e a n of t he rec iproca ls of der ivat ives . W e c a n wr i te

w(x) ~ X m A(x)

which exhib i t s \p{x) a s a genera l i za t ion of t h e N e w t o n - R a p h s o n I . F . in t h e sense t h a t t h e rec ip roca l of t h e der iva t ive is r ep laced by a we igh ted mean of r ec ip roca l s of der iva t ives .

A genera l t r e a t m e n t of these t w o famil ies may be found in [I] ( C h a p t e r 9) . H e r e we s t u d y o n e pa r t i cu l a r case , an I.F. of t h e s econd t y p e wh ich is of t h i r d o rde r .

W e can simplify n o t a t i o n . L e t

tp(x) = x — a1c/j1(o;) — a2co2(x)

m a)x{x) = u(x), oj2(x)

f'[x + 0u(x)]

T h e n ip(x) is of t h i rd o r d e r if

a1 + a2=l, 2fia2= - 1 (27)

F o r /? 5 0 th is defines a o n e - p a r a m e t e r f a m i l y wi th

2$

1 + 2 / ?

2 £ a2 = — — , aA =

F u r t h e r m o r e

- > 2(/? + l)A2\a) - (1 + IfiAM (x - a ) 3

Before p roceed ing fur ther , we lay d o w n # rule c o n c e r n i n g t h e choice of t h e free p a f a m * eter . W h e n us ing y),f' is s a m p l e d a t x a n d a t x + (3u(x). Le t z — x -f flu(x). W e w a n t t h e d i s t ance f r o m z t o a t o b e smal le r t h a n t h e d i s t ance f rom x t o a. Since

z - a = (1 + 0)(x - a ) + 0[(x - a)2J we w a n t |1 + /S| < 1, t h a t is,

0 > p > - 2 (28)

I t s h o u l d be e m p h a s i z e d t h a t \p is o f t h i r d o r d e r for all al9 a2, ft satisfying (27). T h e c o n s t r a i n t (28) is a n a d d i t i o n a l c o n d i t i o n de r ived f r o m s a m p l i n g cons ide ra t i ons .

A n u m b e r of so lu t ions a re of special in te res t . I f = _ f , t h e n ax = 0, a2 = 1,

y^x) = x

f'[x - \u(x)] A(x)

(x - oc)3

(29)


I f f} = — l , t h e n a± = a2 = -|,

y)(X) = X — +

(p(x) — a

- «(*)]

l -4s(a)

(30)

( x - a ) 3

I f jS = — | , t h e n ax = \,a2 = | ,

y ( « ) = x — |Jm(x) + 3

<p — a

tu(a;)]

(31) (a: - a ) 3

f. Mu l t i p le Z e r o s

W e deal on ly briefly wi th t h e in te res t ing subject of mu l t ip l e ze ros . A n extensive t r ea t m e n t m a y be f o u n d in [I] ( C h a p t e r 7) .

O b s e r v e t h a t u —flf h a s on ly s imple ze ros . L e t / a n d its derivative:: be r ep l aced b y u a n d its der iva t ives in a n y I . F . T h e n t h e en t i re t h e o r y which pe r t a in s t o s imple ze ros m a y be app l i ed . If, for e x a m p l e , we rep lace / b y u in t h e m e t h o d of N e w t o n - R a p h s o n , t h e n

<P x — .— = x — • 1 — 2A<m

*is s e c o n d o r d e r for ze ros of all mul t ip l ic i t ies . O b s e r v e t h a t th is r equ i res t h e e v a l u a t i o n o f / " .

A l l m e m b e r s of t h e family Es s tud ied o n p p .

' BTSB a r e ° ^ ^ r s t o n * e r f ° r n o n s i m p l e ze ros . H o w e v e r t he fol lowing modi f ica t ions of E2

a n d E3 a r e of s econd a n d th i rd o rde r , respect i ve ly , for ze ros of mul t ip l ic i ty exact ly m ([1], Sec t ion 7.3).

£2 — X

£3 = X

mu

| m ( 3 — m)u — m2A2u2

T h e difficulty wi th us ing these I . F . is t h a t m is n o t k n o w n a p r io r i in p rac t i ce . T h i s difficulty m a y be m e t by us ing

(p(x) = x — u(x)h(x)9

I t m a y be s h o w n t h a t h(x)

0 2 )

In \u\

—> m. F u r t h e r m o r e

<P In \x — oc| • — Inym ml

l/m\

(33)

H e n c e th is I . F . is ne i ther l inear n o r super l inear . I t s o r d e r is i n c o m m e n s u r a t e wi th t h e o r d e r

scale we use . P roof s of these resul ts m a y be f o u n d in [I] ( p p . 154-157) .

I n a genera l roo t - f ind ing r o u t i n e it m i g h t be w o r t h w h i l e a lways t o ca lcu la te h. Af ter / a n d

/ ' h a v e been ca lcu la ted , it is n o t expens ive t o ca lcu la te h = In | / | / l n \u\. Af ter t h e l imi t o f h h a s been d e t e r m i n e d t o t h e nea res t in teger , t h e r o u t i n e c a n swi tch t o e2 = x — mu. A n u m e r i c a l e x a m p l e m a y be f o u n d in [I] ( p p . 279 -280) .

g. Sy s tems of Equat ions

G e n e r a l Theory

W e wish t o solve t he sys tem

/<(*!, ' ' ' > x n ) = 0, i = 1, 2 , • • • , n (34)

I t is of ten conven ien t t o use t he n o t a t i o n

T h e n (34) b e c o m e s f(x) = 0. Since t h e s u b scr ipts d e n o t e c o m p o n e n t s , pa r en thes i zed super scr ipts will d e n o t e i t e r a t i on c o u n t s . T h e I.F> a r e vec to r -va lued func t ions of a vec to r variable?. T h u s for o n e - p o i n t I . F . , x { k + 1 ) =<p [x ( f c ) ] .

We use the convention that summation from 1 to n is performed over all repeated subscripts. T h u s

n

^ A i i x i = AjjXj 3=1

W e use a genera l i za t ion o f t h e first t h e o r e m of p . Q| as t he def ini t ion of o rde r . Let^p(x) h a v e p c o n t i n u o u s der iva t ives wi th respec t t o all c o m p o n e n t s of x. A n I . F . is of o r d e r p if

cp(a) = a

y?<(«) dxH dxh dX;

a n d

= 0, for all 1 < k < p - 1,

dxu dx, • • • dx4

5*0

for a t least o n e va lue of i,j\,j2, • • • ,jP. T h u s <p(x) is of pth o r d e r if a is a fixed p o i n t of <p, if all pa r t i a l der ivat ives of o r d e r less t h a n p van i sh a t a, a n d if a t least o n e /? th-order pa r t i a l der iva t ive does n o t van i sh a t a. T h i s def ini t ion is only val id for I . F . of in tegra l o rde r . ( F o r a def ini t ion which utilizes m a t r i x n o r m s , t he r e a d e r is referred t o [4], Sec t ion 3.4. F o r a def ini t ion of o r d e r in a m o r e genera l se t t ing, see [2], Sec t ion 17.)


Table 2. Data for numerical example

0 1.08 -0.1 1 0.9985 -0.015 2 0.99979 -0.000039 3 1.000000044 -o.ooooooon

This is the generalization of the error estimate in the scalar case, which is

2 / ' ( a ) 1

It is not difficult to show that if the Jacobian is nonsingular at a and if the initial approximation is sufficiently good, then the iteration will converge.

Numerical Example: Let

f(z) P S a3 - 1 , 2 = x1 + ix2

fi(xi, x2) = xi - 3z 2

2 - 1, f&(x\9 X2) = 3x%2x2 ~ X2B

We try to calculate the zero at a x = 1, a 2 = 0 by Newton-Raphson. The error estimate (38) becomes

Starting with an initial approximation of xx = 1.0S, x2 = —0.1, we obtain the data of Table 2.

Higher-Order Iteration Functions Additional I.F. for systems of equations are

given in [I], Chapter 11. Of particular interest are those which are of order greater than two and yet do not require the calculation of more than first partial derivatives. These I.F. are generalizations of multipoint I.F. For numerical examples of these I.F., see [I], pp. 284-287.

h. C o m p u t a t i o n a l Efficiency

We introduce a measure of the efficiency of an I.F. Let the cost of evaluating / be unity and let the cost of evaluating/0* be 0,. Let 0 = £0,.. We define the computational efficiency of an I.F. q> relative t o / as

E(<p,f) = p1/e

where p is the order of (p. A discussion as to why this is a reasonable measure of efficiency may be found in [I], Appendix C.

It is a straightforward procedure to derive a formal series expression for a. Assume that the Jacobian matrix

is nonsingular in the neighborhood of a and let be the inverse to / Then

« = J^(0) (35)

and expanding the right side of (35) into a Taylor series about y yields

«t = t(y) — z Vj + - -—— ViVu +

(36)

where, as always, summation is performed over repeated subscripts. We wish to rewrite (36) fc terms of x and/l(ji:) only. Denote the elements of the inverse Jacobian by H{i. Then one may show ([I], pp. 219-220),

at* = x, - H,/x)//x) + £ffrJfc(x)

x [//,,(x)]/Xx)/,(x) + • - •• (37) The Generalized Newton-Raphson Iteration Function If the right side of (37) is truncated after the

first sum, we obtain

which is the generalization of the Newton-Raphson I.F. It should be clear from the method of derivation that the generalized Newton-Raphson I.F. is of second order. This may also be verified directly by showing that

d<Pi(*) _ Q

Furthermore

A>X«)_ J J i X a )9 2//tt) dxk dxl

1 dxk dxx

Let = *<«> - a,

That is, e\a) is the ith component of the error at the qth iteration. Then

# H , ~tfl . /«) j -^tf , e« (38) oxk oxx


F o r e x a m p l e , t h e secan t m e t h o d h a s 3 c o m p u t a t i o n a l efficiency of | ( 1 + V 5 ) ~ 1.62 whi le t h e N e w t o n - R a p h s o n m e t h o d h a s a c o m p u t a t i o n a l efficiency of 2 1 / ( 1 + 0 i ) . H e n c e if t h e cos t o f ca l cu la t ing t h e first der iva t ive is g rea te r t h a n 0.44, t h e secan t m e t h o d is " c h e a p e r " t h a n N e w t o n - R a p h s o n . A s a fu r the r e x a m p l e , t h e c o m p u t a t i o n a l efficiencies of t h e t h r e e I . F . o n p p . QB-Q£ a r e ^or

* £ 2 ) 1 , 1.84 for 0 2 > 1 a n d 2 . 7 F ( 1 + ^ ) for *Eh2.

O n e m a y i n q u i r e a s t o t h e re la t ive cos t o f ca lcu la t ing t h e f ( j ) for j ^ 0, for different classes of / . I f / is c o m p o s e d of e l emen ta ry func t ions , t h e n / ( i ) is c o m p o s e d o f e l emen ta ry func t ions . T h u s if

f(x) = / ( e®, sin x, cos x) t h e n

/«)(») =fU)(e*9 m x, cos x)

&nd t h e cos t o f eva lua t ing f(j) for j > 0 is mere ly t h e cos t o f c o m b i n i n g these e l e m e n t a r y func t ions . I n such a case , t h e N e w t o n - R a p h s o n I . F . is " c h e a p e r " t h a n t h e secan t I . F . I f / satisfies a s e c o n d - o r d e r differential e q u a t i o n , t h e n t h e s e c o n d der iva t ive is ava i lab le f r o m t h e di f ferent ia l e q u a t i o n . I n th i s case a t h i r d - o r d e r I . F . , such as (15) o r (16) , is a g o o d o n e t o u s e .

j. T h e Cho i ce of an I terat ion Funct ion

W e h a v e a t t e m p t e d t o descr ibe a n a r sena l of m e t h o d s f r o m w h i c h t h e r e a d e r m a y c h o o s e . N o o n e of these is bes t u n d e r all c i r cums tances . H o w e v e r w e shal l t ry t o give a few sugges t ions a s t o w h i c h I . F . t o use .

U n d o u b t e d l y t h e m o s t widely k n o w n I . F . a r e t h e N e w t o n - R a p h s o n a n d t h e secant . T h e y a r e s imple t o use a n d u n d e r s t a n d a n d a r e qu i t e effective.

A s we obse rved in t h e p r eced ing sec t ion , a t h i r d - o r d e r I . F . , such a s (15) o r (16) , c a n b e u s e d t o a d v a n t a g e if / satisfies a s e c o n d - o r d e r differential e q u a t i o n .

I f t h e der iva t ive of / is n o t ava i lab le , t h e I . F . of t h e e x a m p l e s o n p p . QBH9 m a y ^ e

r e c o m m e n d e d . I f / a n d / ' a r e ava i lab le , t h e I . F . of t h e e x a m p l e o n p . QQQ h a s t h e v i r tue of be ing a l m o s t of t h i r d o r d e r whi le r equ i r i ng n o m o r e e v a l u a t i o n s p e r s t ep t h a n N e w t o n -R a p h s o n .

T h e m u l t i p o i n t I . F . h a v e t h e i m p o r t a n t a d v a n t a g e t h a t t hey c a n b e m a d e of t h i r d o r

h ighe r o r d e r w i t h o u t r e q u i r i n g t h e e v a l u a t i o n of s econd o r h ighe r der iva t ives of / . T h e t h i r d - o r d e r I . F . o f (25) a n d (30) a n d t h e f o u r t h -o r d e r I . F . of (26) m a y b e r e c o m m e n d e d for the i r s impl ic i ty . A d i scuss ion of c r i te r ia for t h e se lect ion of a m u l t i p o i n t I . F . m a y b e f o u n d in [I] , p p . 204-j208.

I f o n e uses a n I . F . w h i c h r equ i r e s t h e ca l cu l a t i on of / a n d / ' , it m i g h t b e w o r t h w h i l e a lways t o ca lcu la te t h e q u a n t i t y h = Jn | / | / In |w|. A s we obse rved in Sec t ion 2f, t h e va lues of h c a n b e used t o m o n i t o r t h e mul t ip l ic i ty of t h e z e r o .

3. S U M M A R Y O F C A L C U L A T I O N P R O C E D U R E

A s a n e x a m p l e we c h o o s e t h e o n e - p o i n t i t e r a t i o n func t ion w i th m e m o r y d e n o t e d b y * i s x > 2 i n t h e e x a m p l e o n p . (Q:

Ji Ji

I f . - f . \ ( 3 9 )

x i — x i - i

T h i s I . F . r equ i r e s o n e e v a l u a t i o n of / a n d / ' p e r i t e r a t ion . I t is if o r d e r 2 .73 . A s u b r o u t i n e w h i c h ca lcu la tes / a n d / ' m u s t b e p r o v i d e d b y t h e user . T h e i n p u t p a r a m e t e r s a r e a n in i t ia l a p p r o x i m a t i o n x09 a conve rgence c o n s t a n t e9

a n d a m a x i m u m n u m b e r of i t e r a t i ons c o u n t M. Since (39) r equ i re s t h a t t w o a p p r o x i m a t i o n s b e ava i lab le , w e ca lcu la te xx b y N e w t o n -R a p h s o n , xx = x0 — / 0 / / o ' . O u r conve rgence c r i t e r ion is

K+i - xt\ < s

\xm\ ~

A n o t h e r n a t u r a l cove rgence c r i t e r ion is i f fo+i) I < & M o r e soph i s t i ca ted t e chn iques r equ i r e d t o be c h o s e n o n t h e bas i s of a n e r r o r ana lys i s .

I n t h e o r y , t h e q u a n t i t y A wh ich a p p e a r s in (39) converges t o i/"(oc). I n p rac t i ce , i t a p p r o a c h e s a z e r o over z e r o f o r m w h i c h na tu r a l l y poses c o m p u t a t i o n a l difficulties. N o t e , howeve r , t h a t A is mu l t ip l i ed b y w h i c h goes t o ze ro a s (xt — a ) 2 . H e n c e A n eed n o t b e cal cu la t ed t o o accura te ly . I n a d d i t i o n , t h e las t


t e r m of (39) m a y be r e g a r d e d as a co r r ec t i on t o xi - ut.

I n t h e flowchart we h a v e ind i ca t ed t h a t e i ther t h e " a n s w e r " 6r a c o m m e n t o f fai lure is p r i n t e d o u t a n d t h a t t h e p r o g r a m is t h e n s t o p p e d . I n p rac t i ce th i s r o u t i n e w o u l d l ikely b e p a r t of a sys tem a n d t h e a n s w e r w o u l d be p a s s e d t o a n o t h e r p a r t o f t h e sys tem. I n case of conve rgence fai lure a superv i so ry r o u t i n e w o u l d m a k e a dec is ion as t o w h a t t o d o next . I n p rac t i ce t h e r e m i g h t b e checks t o g u a r a n t e e t h a t d iv i s ion b y a very smal l n u m b e r d i d n o t occur .

4. F L O W C H A R T

5. D E S C R I P T I O N O F T H E F L O W C H A R T

Box 1: R e a d in i n p u t p a r a m e t e r s : x0 is t h e ini t ial a p p r o x i m a t i o n , e is t h e convergence c o n s t a n t , M is t h e m a x i m u m n u m b e r of i t e ra t ions .

Box 2 : Ini t ia l ize x ^ t o x0. B o x 3 : Calculate/(#<>) a n d f'(xo)-Box 4 : Ca lcu l a t e xx b y N e w t o n - R a p h s o n . Box 5 : Ini t ia l ize i t e r a t ion c o u n t t o un i ty . B o x 6 : Ca lcu l a t e f(xx) a n d / ' f o ) . B o x 7 : Ca l cu l a t e t h e nex t a p p r o x i m a t i o n . B o x 8 : T e s t w h e t h e r t h e re la t ive e r r o r is less

t h a n t h e conve rgence c o n s t a n t . Box 9 : If it is, p r i n t t h e " a n s w e r . "

START Read XQ, e, M

V . 3

10

Xi — > * i - l xi+1 — Xi

Ji — - > # - i yi — y't-i

No

> ' 13 i + i -—>-i

I

> ' 12 Print

FAILURE

I STOP

No

<3

Xi-Xi-i—>-D

yi

Yes

> f 9 Print

> r

STOP


B o x 10: I f i t is n o t , p r e p a r e for t h e nex t i t e r a t ion .

B o x 1 1 : Tes t w h e t h e r t h e n u m b e r of i tera t i o n s h a s exceeded t h e m a x i m u m n u m b e r a l lowed .

B o x 12 : I f yes , s ignal fa i lure . B o x 1 3 : I f n o , s t ep t h e i t e r a t ion c o u n t a n d

send c o n t r o l t o B o x 6.

6. P R O G R A M

C ROUTINE FOR CALCULATING ROOTS OF TRANSCENDENTAL C EQUATIONS

READ 1, X , EPSIL, M 1 FORMAT (2E14.7, 12)

XPP = X CALL FUNC (XPP, YPP, YDPP)

C FUNC IS A SUBROUTINE FOR W H I C H THE INPUT IS A VALUE OF X A N D THE C OUTPUT IS F(X) A N D F PRIME (X)

XP = XPP - YPP/YDPP D O 2 I = 1, M CALL FUNC (XP, YP, YDP) D = XP - XPP A = (2 .*YDP + YDPP - 3.*(YP - YPP)/D)/D U = YP/YDP X = XP - U * ( 1 . + U *A/YDP) IF (ABS((X - XP)/X) - EPSIL) 3, 3, 4

4 XPP « XP XP = X YPP = YP

2 YDPP = YDP PRINT 5

5 FORMAT (8H FAILURE) STOP

3 PRINT 6, X 6 FORMAT (E14.7)

STOP END

7. N U M E R I C A L E X A M P L E

W e t a k e f(x) = s i n ( e a j ) a n d t a k e a a s t h e z e r o a t In TT. W e use t h e I . F . o f (39) . O n e m a y h o w t h a t t h e e r r o r s satisfy ( [ I ] , Sec t ion 6.2)

*i+i /<4,(«> 2 ^ 2

i-l 2 4 / » e{ = xt - a

I n Ta^le 3 , x0 is t h e s t a r t ing v a l u e ; xx is c a l c u l a t e b y xx = x0 - / 0 / / 0 ' . T h e va lue of

Table 3s D A T A F O R N U M E R I C A L E X A M P L E

0 1.0 1 1.166 2 1.1447230 3 1.144729885849348 4 1.144729885849400

- 1 . 4 x lO" 1

2.1 x 10~ 2

- 6 . 8 x 10~ 6

- 5 . 2 x 1 0 - 1 4

# 4 agrees w i th a p u b l i s h e d va lue of In IT t o 16 significant figures.

8. E S T I M A T I O N O F T H E R U N N I N G T I M E

I n a l m o s t al l p r o b l e m s t h e t i m e r e q u i r e d for t h e e v a l u a t i o n o f / a n d / ' will d e t e r m i n e t h e r u n n i n g t i m e .

9. R E F E R E N C E S

1. J. F . Traub, Iterative Methods for the Solution of Equations, Prentice-Hall, Englewood Cliffs, NJ . , 1964.

2. L. Collatz, Funktionalanalysis und Numerische Mathematik, Springer-Verlag, Berlin, 1964.

3. A. S. Householder, Principles of Numerical Analysis, McGraw-Hill, New York, 1953.

4. A. KorganofF, Methodes de CalculNumerique, Vol. 1, Dunod, Paris, 1961.

5. J. F. Traub, "On Lagrange-Hermite Interpolation," /. Soc. Indust. Appl. Math., 12 886-891 (1964).

the solutio on f transcendental equations* · the solution of transcendental equations 161 j u...

Documents