chapter2.ppt

Solving Nonlinear Equations

CHAPTER 2

2

Introduction:

In scientific and engineering studies, a frequently occurring problem is to find the roots or zeros of equations of the form f(x) = 0. (Root Finding Problem)

f(x) may be algebraic or transcendental or a combination of both.

Algebraic functions of the form P(x)= a0 + a1 x + … + an xn , are called polynomials.

A non-algebraic function is called a transcendental function.

3

Polynomials up to degree 4 can be solved exactly. Since finding the root of f(x)=0 is not always possible by analytical means, we have to go for some other techniques or methods.

Solution Methods are either Bracketing:Bisection Method, Regula Falsi Method

or Iterative:Secant Method, Newton's Method, Graffe’s Root Squaring Method,

Rate of Convergence. Advantages/Disadvantages.

Solution Methods

4

Multiplicity

A root p of the equation f(x) = 0 is said to be a Root of Multiplicity m if f can be written in the form

A root of multiplicity one is said to be a Simple Root. Theorem: Let f be a continuous function with m continuous

derivatives. The equation f(x) = 0 has a root of multiplicity m at x = p if and only if

( ) ( ) ( ), where lim ( ) 0.m

x pf x x p q x q x

( 1) ( )( ) ( ) ( ) ... ( ) 0 but ( ) 0.m mf p f p f p f p f p

5

The Bisection Method

The Bisection method is based on the Intermediate Value Theorem and Cantor's Intersection Theorem.

These methods essentially work by first finding an interval which is guaranteed to contain a root and then systematically shrinking the size of that interval.

Intermediate Value Theorem: Let f(x) be continuous function in [a,b] and let k be any number between f(a) and f(b). Then, there exists a number c in (a, b) such that f(c) = k.

Hence, an equation f(x) = 0 where f(x) is a real continuous function, has at least one root between a and b if f(a) f(b) < 0.

6

Bisection method continued …

Cantor's Intersection Theorem:

If a sequence of closed intervals [an, bn] is such that

[an+1, bn+1] [an, bn] for all n, and lim(bn - an)=0 as n → ∞,

then consists of exactly one point.

Algorithm for Bisection Method:

Suppose f(x) is continuous and we have two numbers a1 and b1

such that f(a1)f(b1)<0, then by Intermediate Value Theorem we

know that there is a root for f(x) between a1 and b1.

nnn

ba ,0

7


Then divide the interval into two parts and take the middle point suppose that is c,

if f(c ) =0 then we got the root,

if not so then by IVT the root lies within a and c or c and b, depending on whether f(a1)f(c) <0 or f(c)f(b1)<0.

Then we have to take the next interval in which the root lies and continue our process.

Since we are dividing the interval by half each time, we are approaching to zero, hence this algorithm gives us the approximate solution. Now we have to put some accuracy criteria to stop the process.

8


Algorithm for Bisection Method:

Step 1: Choose a and b as two initial guesses for the root such that f(a) f(b) < 0.

Step 2: Set x= (a+b)/2.

Step 3: If f(a) f(x) < 0, the root lies in the interval (a, x).

Then, set b = x

else set a = x

end if (|a – b| ≤ ε1 or f(x) ≤ ε2)

Remark: The method may produce a false root if f(x) is discontinuous on [a, b].

9


For the given values of 1 and 2, the iteration continues until

|bn - an| < 1 and/or |f(x)| < 2.

The root correct up to desired accuracy is (an - bn)/2.

Advantage: This method definitely converges to the root of any type of non-linear equation.

Disadvantages:

• This method is too slow (linear) i.e. where c <1.

• We need two initial guesses for this method.

1 ,n ne c e

10

Example 1: Find a real root of the equation x3 - 2x - 5 = 0, correct up-to 3-significant digits.

11

Bisection method continued … Example 1:

Find a real root of the equation x3-2x-5 = 0, correct up-to 3-significant digits.

Solution: Let f(x)=x3-2x-5. Now f(2)=-1 and f(3)=16. Hence, by IVT, there exist a root between 2 and 3.

Iteration 1. Now we set a0=2, b0=3, and hence x= (a0+b0)/2=2.5. Since f(2.5)= 5.625, we have f(2) f(2.5) < 0, set a1=2, b1 = 2.5 for the next iteration.

Iteration 2. a1=2, b1=2.5, x=(a1+b1)/2= 2.25, f(2.25)= 1.890625. Since f(2)f(2.25)<0, set a2=2, b2 = 2.25.

12


Iteration 3. a2=2, b2=2.25, x=(a2+b2)/2= 2.125, f(2.125)=0.3457. Since f(2)f(2.125)<0, set a3=2, b3=2.125.

Iteration 4. a3=2, b3=2.125, x=(a3+b3)/2= 2.0625, f(2.0625)=-0.3513. Since f(2)f(2.0625)>0, set a4=2.0625, b4=2.125.

Iteration 5. a4=2.0625, b4=2.125, x=(a4+b4)/2 =2.09375, f(2.09375)=-0.0089.

Since f(2.0625)f(2.09375)>0, set a5=2.09375, b5=2.125.

13


Iteration 6. a5=2.09375, b5=2.125 Now, x=(a5+b5)/2=2.10938, f(2.10938)=0.1668. Since f(2.09375)f(2.10938)>0, set a6=2.09375, b6=2.10938.

Iteration 7. a6=2.09375, b6=2.10938, x=(a6+b6)/2 =2.10156.

Thus, a 3-significant digits root of the given equation is 2.10156.

Further iterations can be made to get better accuracy.

14

Example 1…

k a b f(a) f(b) x f(x) 1 2.00000 3.00000 -1.00000 16.00000 2.50000 5.62500 2 2.00000 2.50000 -1.00000 5.62500 2.25000 1.89063 3 2.00000 2.25000 -1.00000 1.89063 2.12500 0.34570 4 2.00000 2.12500 -1.00000 0.34570 2.06250 -0.35132 5 2.06250 2.12500 -0.35132 0.34570 2.09375 -0.00894 6 2.09375 2.12500 -0.00894 0.34570 2.10938 0.16684 7 2.09375 2.10938 -0.00894 0.16684 2.10156 0.07856 8 2.09375 2.10156 -0.00894 0.07856 2.09766 0.03471

15

Bisection method continued … Exercises:

Problem1: Perform 5 Iterations of the bisection method to obtain the smallest positive root of the equation

x3-5x+1 =0.

Problem2: Determine the smallest positive root correct up to two decimal places using bisection method for

tan x + tanh x=0.

17

The Secant Method

Let x0 and x1 be the two approximation of the root . If the function f is linear, the straight line passes through (x0, f(x0)) and (x1, f(x1)) would intersect the X-axis at the root (y=0). But since f is not linear, the intersection of the line with X-axis is not at x= but this should be close to it.

The equation of chord through (x0, f(x0)) and (x1,f(x1)) is given by

).()()(

)( 001

010 xx

xx

xfxfxfy

18

secant method continued …

At the intersection with X-axis, y=0. So,

On simplification we get

Now we take this x as our next approximation and denote it as x2

,)()(

))((

01

0100 xfxf

xxxfxx

,)()(

)()(

)()(

))((

01

0110

01

0100 xfxf

xfxxfx

xfxf

xxxfxx

19


From x1 and x2 we can calculate our next approximation x3 using the previous technique and in general, we can write the iteration scheme as

We should start with x0 and x1 such that x1 is closer to the root than x0 to get better result.

,3,2,1,)()(

)()(

1

111

n

xfxf

xfxxfxx

nn

nnnnn

20

secant method continued …Geometrical Interpretation of Secant Method

The approximation points are a,b, x1,x2,x3,…

21

secant method continued … Example: Find the root of f(x)=x3+x2-3x-3 correct up to 3-significant digits with initial approximation x0=1 and x1=2 by secant method.

Solution: The iteration scheme of the secant method is,

Using this formula we get

,3,2,1,)()(

)()(

1

111

n

xfxf

xfxxfxx

nn

nnnnn

,571429.1)1()2(

)1(2)2(1

)()(

)()(

01

01102

ff

ff

xfxf

xfxxfxx

22


Applying the formula again we get x3 as

Similarly computing further we get

Thus the root correct up to 3-significant digits of the given equation by secant method is 1.731996.

,735136.1)()(

)()(

23

23324

xfxf

xfxxfxx

,705411.1)571419.1()2(

)2(571419.1)571419.1(2

)()(

)()(

12

12213

ff

ff

xfxf

xfxxfxx

,731996.1)()(

)()(

34

34435

xfxf

xfxxfxx

23

secant method continued … Advantages and Disadvantages

Advantages:

• Converges fast (if it converges) in comparison with Bisection method.

• Requires two guesses that do not need to bracket the root.

• Only requires one function evaluation per iteration.

• Avoids calculation of derivatives.

24


Advantages and DisadvantagesDisadvantages:

• Division by zero

• In this method, it is not necessary to have two initial guesses bracketing the root, but on the other hand, convergence is not guaranteed.

• In some cases, swapping the two initial guesses x0 and x1 will change the behavior of the method from convergent to divergent.

25

secant method continued … Order of Convergence:

Let xn be a sequence of approximations to the root by a given method. Let en = - xn be the error in the n-th approximation of xn.

If there exists a number c 0 (independent of n) such that

then p is the order of convergence of the method.

ce

ep

n

n

n

||

|| 1lim

26


Order of Convergence of Secant Method:

Prove that the order of convergence of secant method is approximately 1.618.

Proof: We know that by Secant Method the sequence is generated by

Further let the sequence {x0, x1, x2, …} converges to the root, (i.e. f()=0).

2

51

,3,2,1,)()(

)()(

1

111

n

xfxf

xfxxfxx

nn

nnnnn

27

secant method continued … Order of Convergence of Secant Method:

Now we have en+1= - xn+1

Thus

,)()(

)()(

1

11

nn

nnnn

xfxf

xfxxfx

.,, 11 nnnnnn exexorxe

.)()(

)()()()(

1

111

nn

nnnnn efef

efeefee

(1)

28



Expanding the functions by Taylor’s series we get

Substituting these values in (1) and neglecting the higher degree terms and simplifying

)(!2

)()(!2

)()()(22

fe

fefe

fefef nn

nnn

)(!2

)(

)(!2

)()()(

21

1

21

11

fe

fe

fe

fefef

nn

nnn

29


Order of Convergence of Secant Method:we get

Let (A constant)

Then we have Suppose then . On substituting these values in (2) we get

.)(

)(

2

111

f

feee nnn

.)(

)(

2

1

f

f

.11 nnn eee pnn ee 1

pnn ee 1

(2)

.1)/1(

/11

/1

1

p

npn

p

nnn ee

eee

30



Comparing

and equating the powers of en on right sides we get

Solving for p we get

Hence the order of convergence is 1.618.

Therefore the secant method is faster than linear and slower than Newton's quadratic formula.

and1p

nn ee 1)/1(

/11

pnpn ee

011)/1( 2 pporpp

618.12

51

p

31

Regula Falsi Method:

When the bracketing is done for the secant method, it is known as linear interpolation or the method of false position (in Latin, Regula Falsi).

This technique for Regula Falsi method is similar to bisection except the next iterate is taken at the intersection of a line between the pair of X-values and X-axis rather than at the midpoint.

This method gives faster convergence in comparison with bisection method, but at the expense of a more complicated algorithm.

32

Regula Falsi Method continued… Algorithm for Regula Falsi Method:

Step 1: Choose x0 and x1 as two guesses for the

root such that f(x0 ) f(x1)<0.

Step 2: Set

Step 3:

If f(x0)f(x2)<0, the root lies in the interval (x0, x2).

Then, set x1=x2 and go to Step 2.

If f(x0)f(x2)>0, the root lies in the interval (x0, x2).

Then, set x0=x2 and go to Step 2.

)()(

)()(

01

01102 xfxf

xfxxfxx

33

Regula Falsi Method continued… Algorithm for Regula Falsi Method:

If f(x0)f(x2)=0, x2 is a root of the equation and the computation may be terminated.

Continue till a root is obtained to desired accuracy

34

Regula Falsi Method continued… General remarks on Regula Falsi Method:

Asymptotically, the other end-point will remain fixed for all subsequent iterations while one end-point always being updated.

As a result, unlike the bisection method, the width of the bracket does not tend to zero. As a consequence, Regula Falsi approximation to f(x), which is used to pick the false position, does not improve in its quality.

If the initial end-points x0 and x1 are chosen such that f(x0) and f(x1) are of opposite signs, then one of the end-points will converge to a root of f.

35

Regula Falsi method continued … Example:

Let f(x)=2x3-4x2+3x. Let x0=-1 and x1=1. The left end, -1, is never replaced and thus the width of the bracket never falls below 1. Hence, the right endpoint approaches 0 at a linear rate (with a speed of convergence of 2/3).

To accelerate the rate of convergence in such cases, the Modified Regula Falsi Method is used. In this modified rule, Set

if successive x2 falls on right of .

2/)()(

2/)()(

01

01102 xfxf

xfxxfxx

36


or Set

if successive x2 falls on left of .

)(2/)(

)(2/)(

01

01102 xfxf

xfxxfxx

37


Find a real root of the equation x3-2x-5=0 correct up-to 3-significant digits by using Regula Falsi method.

Solution: Here, f(2)=-1 and f(3)=16 and hence a root lies in (2,3). Let x0=2 and x1=3. We get x2 as

Now, f(x2)=-0.390799917 and hence the root lies between 2.058823529 and 3.

058823592.2)()(

)()(

01

01102

xfxf

xfxxfxx

38


Computing x3 we get

Now, f(x3)=-0.147204057 and hence the root lies between 2.08126366 and 3. Similarly

Thus, the root correct up to 3-significant digits of the given equation by secant method is 2.089639211.

08126366.2)()(

)()(

12

12213

xfxf

xfxxfxx

089639211.2)()(

)()(

23

23324

xfxf

xfxxfxx

39

Newton's Method: Choose a value that is reasonably close to

the root of the equation.

Replace the function by its tangent.

Computes the zero of this tangent.

The zero of this tangent will be a better approximation to root of the equation and the procedure can be iterated.

In Newton's Method, we replace the chord in the secant method by a tangent.

40

Geometrical Interpretation of Newton’s Method

41

Newton's Method Continued…Theorem:

Newton-Raphson Theorem: Assume that fC2[a,b] and there exists a number [a,b], where f()=0. If f’() 0, then there exists a >0 such that the sequence defined by the iteration

will converge to for any initial approximation

.

}{0nx

n

,3,2,1,0for ,)(

)(1

n

xf

xfxx

n

nnn

],[0 x

42

Newton-Raphson Theorem Continued…

Proof: Since fC2[a,b], using Taylor's series expansion of f around x0, we have

where c lies in between x and x0.

Substituting x= in the Taylor's series with the fact f()=0, we get

).(!2

)()()()()(

20

000 cfxx

xfxxxfxf

).(!2

)()()()(0

20

000 cfx

xfxxf

43


If x0 is close enough to , then the last term on the right side will be small compared to the sum of the first two terms. Hence it can be neglected and we can use the approximation,

Solving for we get

).()()(0 000 xfxxf

).(/)( 000 xfxfx

44


This is used to define the next approximation x1 to the root as

In general the Newton Method scheme is given by

,3,2,1,0for ,)(

)(1

n

xf

xfxx

n

nnn

.)(

)(

0

001 xf

xfxx

45

Newton's method continued … Example:

Find a real root of the equation f(x)=3x+sin x- ex

correct up-to 3-significant digits by Newton's method.

Solution: Here, f(x)=3x+sin x-exC2(). Hence, f'(x)=3+cos x-ex. The iterative scheme for Newton's Method is

Let x0=0. Thus, we have x1=0-(-1/3) = 0.33333

,2,1,0.cos3

sin3

)(

)(1

nex

exxx

xf

xfxx

n

n

xn

xnn

nn

nnn

46

Newton-Raphson Method

47

Convergence of Newton's Method:

In Newton's Method, fC2[a,b], f()=0 and f'()0, where [a,b]. Hence,

Where n, n lies between and xn .

1 1

( ) ( )

( ) ( )n n

n n n nn n

f x f xe x x e

f x f x

.)()(

)()2/()()(

)(

)( 2

nn

nnnn

n

nn fef

fefefe

ef

efe

48


(by using the Binomial expansion)

.

)(

)(1)(

)()2/()( 2

f

fef

fefee

nn

nnnn

.)()(

)(1

)(

)()2/( 22

nnnn

nnn eOf

fe

f

feee

).()()(2

1

)(3

2

nnnn eOff

f

e

49


As n, n and . Thus

Thus, Newton's method converges quadratically i.e. Order of convergence p=2.

.0|)(|2

|)(|

||

||lim

21

c

f

f

e

e

n

n

n

50

Advantages of Newton’ method

It is rapidly convergent in most cases since p=2.

It is simple in its formulation, and therefore relatively easy to apply and program.

It is intuitive in its construction. This means it is easier to understand its behavior, when it is likely to behave well and when it may behave poorly.

51

Disadvantages of Newton’ method:

It may not converge.

It is likely to have difficulty if f'(xn)=0. That means the X-axis is tangent to the graph of y=f(x) at x=xn.

The closer the derivative goes to zero, the worse Newton's Method behaves. Flat tangents may diverge the scheme.

This method can spin forever if there is no real root of the equation. (Example: x2 + 2 = 0).

It needs to know both f(x) and f'(x). Contrast this with the bisection method which requires only f(x).

52

Newton's Method for Multiple Roots:

A root is called a multiple root of order m, (m) if f(x) = (x- )m h(x) such that h() 0.

For m=1, is called simple root of f(x).

The Newton's method

converges quadratically, when is a simple root of f(x)=0, but it converges linearly when is a multiple root of f(x) = 0.

,)(

)(1

n

nnn xf

xfxx

53

Fixed Point Method (FPM) Continued…

Modified Newton's Method:

In the modified (accelerated) Newton's method, we take

if is a multiple root of order m of f(x)=0 and it converges quadratically.

,)(

)(1

n

nnn xf

xfmxx

54

Example:

Find the approximate value of the double root of x3 -2x2 - 0.75 x + 2.25 =0, that is correct up to 3 decimal places and close to 1 such that iterations converges quadratically. Take x0 =1.

Solution: Since the root of the equation is a double root, we will use modified Newton's method in order to get quadratic convergence.

55

The End of Chapter 2

chapter2.ppt

Documents

root of fx

real root

false root

simple root

equation fx

graffes root squaring

newtons method

secant method