8/12/2019 FPI1314

1/13

(1/13)

1.5 1 0.5 0 0.5 1 1.51.5

1

0.5

0

0.5

1

1.5

Solving nonlinear equationsSee online and lecture notes for full details

1.5: Fixed Point IterationMA385/530 Numerical Analysis 1

September 2013

8/12/2019 FPI1314

2/13

Fixed Point Iteration Introduction (2/13)

Newtons method can be considered to be a special case of a verygeneral approach called Fixed Point Iteration or Simple Iteration .

The basic idea is:

If we want to solve f (x ) = 0 in [a , b ], nd a functiong (x ) such that, if is such that f ( ) = 0 , then

g ( ) = . Choose x 0 and for k 0 set x k +1 = g (x k ).

8/12/2019 FPI1314

3/13

Fixed Point Iteration Introduction (3/13)

Example

Suppose that f (x ) = e x

2x 1 and we are trying to nd asolution to f (x ) = 0 in [1, 2]. Then we can take g (x ) = ln(2 x + 1).

If we take x 0 = 1, then we get the following sequence:

k x k | x k |0 1.0000 2.564e-11 1.0986 1.578e-12 1.1623 9.415e-2

3 1.2013 5.509e-24 1.2246 3.187e-25 1.2381 1.831e-2...

... ...

10 1.2558 6.310e-4

8/12/2019 FPI1314

4/13

Fixed Point Iteration Introduction (4/13)

We have to be quite careful with this method: not every choiceis g is suitable .

For example, suppose we want the solution to f (x ) = x 2 2 = 0in [1, 2]. We could choose g (x ) = x 2 + x 2. Then, if take x 0 = 1we get the sequence:

8/12/2019 FPI1314

5/13

Fixed Point Iteration Introduction (5/13)

So we need to rene the method that ensure that it will. Before wedo that in a formal way, consider the following...

ExampleUse the Mean Value Theorem to show that the xed point methodx k +1 = g (x k ) converges if |g (x )| < 1 for all x near the xed point.

This example:

Introduces the tricks of using that g ( ) = andg (x k ) = x k +1 .Leads us towards the contraction mapping theorem .

8/12/2019 FPI1314

6/13

Fixed points and contractions (6/13)Theorem (Fixed Point Theorem)Suppose that g (x ) is dened and continuous on [a , b ], and that

g (x ) [a , b ] for all x [a , b ]. Then there exists [a , b ] suchthat g ( ) = . That is, g (x ) has a xed point in [a , b ].

..........................................................................................................

.........................................................................................................

.............................................................

....................................................................................................... ..................................................

................................

............................

..........................

............................

...........................

a

b

g (x )

8/12/2019 FPI1314

7/13

Fixed points and contractions (7/13)Next suppose that g is a contraction . That is, g (x ) is continuousand dened on [a , b ] and there is a number L (0, 1) such that

|g ( ) g ( )| L| | for all , [a , b ]. (1)

Proposition (Contraction Mapping Theorem)

Suppose that the function g is a real-valued, dened, continuous,and

maps every point in [a , b ] to some point in [a , b ]is a contraction on [a , b ],

then(i) g (x ) has a xed point [a , b ],(ii) the xed point is unique,(iii) the sequence {x k }k =0 dened by x 0 [a , b ] and

x k = g (x k 1) for k = 1 , 2, . . . converges to .

8/12/2019 FPI1314

8/13

How many iterations? (8/13)

The algorithm generates as sequence {x 0, x 1, . . . , x k }. Eventuallywe must stop. Suppose we want the solution to be accurate to say10 6, how many steps are needed? That is, how big do we need totake k so that

|x k | 10 6?

The answer is obtained by rst showing that

| x k | Lk

1 L|x 1 x 0|. (2)

8/12/2019 FPI1314

9/13

How many iterations? (9/13)

ExampleIf g (x ) = ln(2 x + 1) and x 0 = 1, and we want

|x k

| 10 6,

then we can use (2) to determine the number of iterations required.

8/12/2019 FPI1314

10/13

Wrapping up (10/13)

When studying a numerical method (or any piece of Mathematics)you should ask why you are doing this. For example, it might bebecause it will help you can understand other topics later; becauseit is interesting/beautiful in its own right; or (most commonly)because it is useful.

Here are some examples of each of these:

1. The analyses we have used in this section allowed us toconsider some important ideas in a simple setting.

Examples include

Convergence , include rates of convergence Fixed-point theory , and contractions. Well be seeinganalogous ideas in the next section (Lipschitz conditions).The approximation of functions by polynomials (TaylorsTheorem). This point will reoccur in the next section, and allthrough-out next semester.

8/12/2019 FPI1314

11/13

Wrapping up (11/13)

2. Applications come from lots of areas of science andengineering. In nancial mathematics, the Black-Scholes equationfor pricing a put option can be written as

V t

12

2S 2 2V S 2 rS

V S

+ rV = 0 .

V (S , t ) is the current value of the right (but not theobligation) to buy or sell (put or call) an asset at a futuretime T ;

S is the current value of the underlying asset;r is the current interest rate (because the value of the optionhas to be compared with what we would have gained byinvesting the money we paid for it) is the volatility of the assets price.

8/12/2019 FPI1314

12/13

Wrapping up (12/13)

V

t 1

2 2S 2

2V

S 2 rS

V

S + rV = 0 .

Often one knows S , T and r , but not . The method of implied volatility is when we take data from the market and then nd thevalue of which, if used in the Black-Scholes equation, would

match this data. This is a nonlinear problem and so Newtonsmethod can be used. See Chapters 13 and 14 of Highams AnIntroduction to Financial Option Valuation for more details.

(We will return to the Black-Scholes problem again at the end of

the next section).

8/12/2019 FPI1314

13/13

Wrapping up (13/13)

3. Some of these ideas are interesting and beautiful.Suppose that we want to nd the complex nth roots of unity: theset of numbers {z 0, z 1, z 2, . . . , z n 1} whos nth roots are 1. Wecould just express this in terms of polar coordinates:

z = e i where = 2k

n

for some k Z = {. . . , 2, 1, 0, 1, 2 . . . } and i = 1.If we were to use Newtons method to nd a given root, we couldtry to solve f (z ) = 0 with f (z ) = z n 1. The iteration is:

z k +1 = z k (z k )n 1n(z k )n 1 .

We take a number of points in a region of space, iterate on each of them, and then colour the points to indicate the ones that convergeto the same root. This Julia set is an example of a fractal.