numerical analysis eciv 3306 chapter 6
TRANSCRIPT
The Islamic University of Gaza
Faculty of Engineering
Civil Engineering Department
Numerical Analysis
ECIV 3306
Chapter 6
Open Methods & System of Non-linear Eqs
Associate Prof. Mazen Abualtayef Civil Engineering Department, The Islamic University of Gaza
PART II: ROOTS OF EQUATIONS
Roots of Equations
Bracketing Methods
Bisection method
False Position Method
Open Methods
Simple fixed point iteration
Newton Raphson
Secant
Modified Newton Raphson
System of Nonlinear Equations
Roots of polynomials
Muller Method
Open Methods
• Bracketing methods are based on assuming an interval of the function which brackets the root.
• The bracketing methods always converge to the root.
• Open methods are based on formulas that require only a single starting value of x or two starting values that do not necessarily bracket the root.
• These method sometimes diverge from the true root.
Open Methods- Convergence and Divergence Concepts
Converging increments
f(x)
xi xi+1 x
f(x)
xi xi+1 x
Diverging increments
6.1 Simple Fixed-Point Iteration
• Rearrange the function f(x)=0 so that x is on the left side of the equation:
)(
)(0)(
1 ii xgx
xxgxf
• Bracketing methods are “convergent”.
• Fixed-point methods may sometime “diverge”, depending on the stating point (initial guess) and how the function behaves.
6.1 Simple Fixed-Point Iteration
Examples: 1.
2. f(x) = x 2-2x+3 x = g(x)=(x2+3)/2
3. f(x) = sin x x = g(x)= sin x + x
3. f(x) = e-x- x x = g(x)= e-x
xxg
or
xxg
or
xxg
xxxxf
21)(
2)(
2)(
02)(
2
2
Simple Fixed-Point Iteration Convergence
• x = g(x) can be expressed as a pair of equations:
y1= x
y2= g(x)…. (component equations)
• Plot them separately.
Simple Fixed-Point Iteration Convergence
1
1
( )
( ) 1
Suppose that the true root:
Subtracting 1 from 2
( ) ( ) (3)
2r
i i
r i r i
r
x g x
x x x
x
x
g
g
x
g
to compute a new estimate xi+1 as expressed by the iterative formula
Simple Fixed-Point Iteration Convergence
Derivative mean value theorem:
If g(x) are continuous in [a,b] then there exist at least one value of x= within the interval such that:
i.e. there exist one point where the slope parallel to the line joining (a & b)
'
g b g ag
b a
Simple Fixed-Point Iteration Convergence
1
'
'
'
1
'
, 1 ,
'
'
( ) ( )
Let and
1.0 theerror decreases with each iteration
1.0 theerror increases with each iteration
r i r i
i r
r i
r i
r i r i
r i r i
t i t i
x x g x g x
a x b x
g x g xg
x x
g x g x x x g
then x x x x g
E g E
If g
If g
'
g b g ag
b a
Simple Fixed-Point Iteration Convergence
• Fixed-point iteration converges if:
( ) 1 (slope of the line ( ) )g x f x x
• When the method converges, the error is roughly proportional to or less than the error of the previous step, therefore it is called “linearly convergent.”
Simple Fixed-Point Iteration-Convergence
1. f(x) is manipulated so that we
get x=g(x) g(x) = e-x 2. Thus, the formula predicting the
new value of x is: xi+1 = e-xi
3. Guess x0 = 0
4. The iterations continues till the
approx. error reaches a certain limiting value
f(x)
Root x
f(x)
x
f(x)=e-x - x
f(x) = e-x - x
Example 6.1: Simple Fixed-Point Iteration
Example 6.1: Simple Fixed-Point Iteration
i xi g(xi) ea% et% 0 0 1.0 1 1.0 0.367879 100.0 76.3 2 0.367879 0.692201 171.8 35.1 3 0.692201 0.500473 46.9 22.1 4 0.500473 0.606244 38.3 11.8 5 0.606244 0.545396 17.4 6.89 6 0.545396 0.579612 11.2 3.83 7 0.579612 0.560115 5.90 2.2 8 0.560115 0.571143 3.48 1.24 9 0.571143 0.564879 1.93 0.705 10 0.564879 1.11 0.399
g(x) = e-x Recall true root is 0.56714329
Flow Chart – Fixed Point
Start
Input: xo , es, maxi
i=0 ea=1.1es
1
Stop
1
while ea< es & i >maxi
i=1 or
xn=0
x0=xn
100%n o
a
n
x x
xe
Print: xo, f(xo) ,ea , i 0
1
nx g x
i i
False
True
6.2 The Newton-Raphson Method
• Most widely used method.
• Based on Taylor series expansion:
)(
)(
)(0
g,Rearrangin
0)f(x when xof value theisroot The
...!2
)()()()(
1
1
1i1i
2
1
i
iii
iiii
iiii
xf
xfxx
xx)(xf)f(x
xxfxxfxfxf
Solve for
Newton-Raphson formula
• A tangent to f(x) at the initial point xi is extended till it meets the x-axis at the improved estimate of the root xi+1.
• The iterations continues till the approx. error reaches a certain limiting value.
f(x)
Root x
xi xi+1
f(x) Slope f /(xi)
f(xi)
)(
)(
)()(
/
/
i
ii1i
1ii
ii
xf
xfxx
xx
0xfxf
6.2 The Newton-Raphson Method
Example 6.3: The Newton Raphson Method
11)(
)(/1
x
x
ix
x
i
i
iii
e
xex
e
xex
xf
xfxx
• Find the root of f (x) = e-x-x= 0 f(xi) = e-x-x and f`(xi)= -e-x-1; thus
Iter. Xi+1 et%
0 0 100
1 0.5 11.8
2 0.566311003 0.147
3 0.567143165 0.00002
4 0.567143290 <10-8
Recall true root is 0.56714329
Flow Chart – Newton Raphson
Start
Input: xo , es, maxi
i=0 ea=1.1es
1
Stop
1
while ea >es & i <maxi
i=1 or
xn=0
x0=xn
100%n o
a
n
x x
xe
Print: xo, f(xo) ,ea , i
0
0 '
0
1
n
f xx x
f x
i i
False
True
Pitfalls of The Newton Raphson Method
6.3 The Secant Method
The derivative is replaced by a backward finite divided difference
)()(
))((
i1i
i1iii1i
xfxf
xxxfxx
Thus, the formula predicting the xi+1 is:
/ 1
1
( ) ( )( ) i i
i
i i
f x f xf x
x x
/ ( )if x
)(
)(/1
i
iii
xf
xfxx
6.3 The Secant Method
•Requires two initial estimates of x , e.g, xo, x1. However, because f(x) is not required to change signs between estimates, it is not classified as a “bracketing” method.
•The secant method has the same properties as Newton’s method. Convergence is not guaranteed for all xo, x1, f(x).
6.3 Secant Method: Example
• Use the Secant method to find the root of e-x-x=0; f(x) = e-x-x and xi-1=0, xi=1 to get xi+1 of the first iteration using:
Iter xi-1 f(xi-1) xi f(xi) xi+1 et%
1 0 1.0 1.0 -0.632 0.613 8.0
2 1.0 -0.632 0.613 -0.0708 0.5638 0.58
3 0.613 -0.0708 0.5638 0.00518 0.5672 0.0048
)()(
))((
1
11
ii
iiiii
xfxf
xxxfxx
xi-1 xi xi+1
Recall true root is 0.56714329
Comparison of convergence of False Position and Secant Methods
False Position Secant Method
Use two estimate xl and xu
Use two estimate xi and xi-1
f(x) must changes signs between xl and xu
f(x) is not required to change signs between xi and xi-1
Xr replaces whichever of the original values yielded a function value with the same sign as f(xr)
Xi+1 replace xi
Xi replace xi-1
Always converge May be diverge
1
1
1
( )( )
( ) ( )
i i i
i i
i i
f x x xx x
f x f x
( )( )
( ) ( )
u l u
r u
l u
f x x xx x
f x f x
Comparison of convergence of False Position and Secant Methods
• Use the false-position and secant methods to find the root of f(x) = lnx. Start computation with xl= xi-1=0.5, xu=xi = 5.
1. False position method
2. Secant method
False Position and Secant Methods
xi-1
xi xu
xl
Although the secant
method may be
divergent, when it
converges it usually
does so at a quicker
rate than the false
position method
• Comparison of
the true percent
relative Errors Et
for the methods to
the determine the
root of
f(x)=e-x-x
Flow Chart – Secant Method
Start
Input: x-1 , x0,es, maxi
i=0 ea=1.1es
1
Stop
1
while ea >es & i < maxi
i=1 or
Xi+1=0
Xi-1=xi
Xi=xi+1
1
1
100%i i
a
i
x x
xe
Print: xi , f(xi) ,ea , i 1
1
1
( )( )
( ) ( )
1
i i i
i i
i i
f x x xx x
f x f x
i i
False
True
6.3.3 Modified Secant Method
Rather than using two initial values, an alternative approach is using a fractional perturbation of the independent variable to estimate
1
( )
( ) ( )
i i
i i
i i i
x f xx x
f x x f x
is a small perturbation fraction
/ ( ) ( )( ) i i i
i
i
f x x f xf x
x
/ ( )if x
Modified Secant Method: Example 6.8
• Use modified secant method to find the root of f(x) = e-x-x, x0=1 and = 0.01. Recall true root is 0.56714329
1
( )
( ) ( )
i i
i i
i i i
x f xx x
f x x f x
6.5 Multiple Roots
x
f(x)= (x-3)(x-1)(x-1)
= x3- 5x2+7x -3
f(x)
1 x
3
Double roots
f(x)= (x-3)(x-1)(x-1)(x-1)
= x4- 6x3+ 125 x2- 10x+3
f(x)
1 3
triple roots
6.5 Multiple Roots
•“Multiple root” corresponds to a point
where a function is tangent to the x-axis.
•Difficulties
- Function does not change sign with double
(or even number of multiple root), therefore,
cannot use bracketing methods.
- Both f(x) and f′(x)=0, division by zero with
Newton’s and Secant methods which may
diverge around this root.
Modified Newton-Raphson Method for Multiple Roots
• Another alternative is introduced such new u(x)=f(x)/f /(x);
• Getting the roots of u(x) using Newton-Raphson technique:
)()()(
)()(
)]([
)()()()()(
)(
)(
//2/
/
1
2/
/////
/1
iii
iiii
i
iiiii
i
iii
xfxfxf
xfxfxx
xf
xfxfxfxfxu
xu
xuxx
This function has roots
at all the same locations
as the original function
Differentiate u(x)=f(x)/f /(x)
Using the Newton-Raphson and Modified Newton-Raphson to evaluate the multiple roots of f(x)= x3-5x2+7x-3 with an initial guess of x0=0
)106)(375()7103(
)7103)(375(
)()()(
)()(
2322
223
//2/
/
1
iiiiii
iiiiii
iii
iiii
xxxxxx
xxxxxx
xfxfxf
xfxfxx
7x10x3
3x7x5xx
xf
xfxx
2
i
i
2
i
3
ii
i
ii1i
)(
)(/
•Newton Raphson formula:
•Modified Newton Raphson formula:
Example 6.10 Modified Newton-Raphson Method for Multiple Roots
Newton Raphson Modified Newton-Raphson
Iter xi et% iter xi et%
0 0 100 0 0 100
1 0.4286 57 1 1.10526 11
2 0.6857 31 2 1.00308 0.31
3 0.83286 17 3 1.000002 00024
4 0.91332 8.7
5 0.95578 4.4
6 0.97766 2.2
• Newton Raphson technique is linearly converging
towards the true value of 1.0 while the Modified Newton
Raphson is quadratically converging.
• For simple roots, modified Newton Raphson is less
efficient and requires more computational effort than the
standard Newton Raphson method.
Modified Newton Raphson Method: Example
6.6 Systems of Nonlinear Equations
• Roots of a set of simultaneous equations:
f1(x1,x2,…….,xn)=0
f2 (x1,x2,…….,xn)=0
. .
fn (x1,x2,…….,xn)=0
• The solution is a set of x values that
simultaneously get the equations to zero.
6.6 Systems of Nonlinear Equations
Example: x2 + xy = 10 and y + 3xy2 = 57
u(x,y) = x2+ xy -10 = 0
v(x,y) = y+ 3xy2 -57 = 0
• The solution will be the value of x and y which makes
u(x,y)=0 and v(x,y)=0
• These are x=2 and y=3
• Numerical methods used are extension of the open
methods for solving single equation; Fixed point
iteration and Newton-Raphson.
1. Use an initial guess x =1.5 and y =3.5
2. The iteration formulae:
xi+1=(10-xi2)/yi and yi+1=57-3xiyi
2
3. First iteration,
x=(10-(1.5)2)/3.5 = 2.21429
y=(57-3(2.21429)(3.5)2 = -24.37516
4. Second iteration:
x=(10-2.214292)/-24.37516 = -0.209
y=57-3(-0.209)(-24.37516)2 = 429.709
5. Solution is diverging so try another iteration formula
6.6 Systems of Nonlinear Equations 1. Fixed Point Iteration
x2 + xy = 10
y + 3xy2 = 57
6.6 Systems of Nonlinear Equations 1. Fixed Point Iteration
1. Using iteration formula:
xi+1=(10-xiyi)1/2 and yi+1=[(57-yi)/3xi]
1/2
First guess: x=1.5 and y=3.5
2. 1st iteration:
x=(10-(1.5)(3.5))1/2=2.17945
y=((57-(3.5))/3(2.17945))1/2=2.86051
3. 2nd iteration:
x=(10-(2.17945)(2.86051))1/2 = 1.94053
y=((57-(2.86051))/3(1.94053))1/2 = 3.04955
4. The approach is converging to true root, x=2 and y=3
x2 + xy = 10
y + 3xy2 = 57
The sufficient condition for convergence for the two-equation case (u(x,y)=0 and v(x,y)=0) are:
1
1
u v
x x
and
u v
y y
6.6 Systems of Nonlinear Equations 1. Fixed Point Iteration
MS Excel: Solver u(x,y)= x2+xy-10 =0
v(x,y)=y+3xy2-57=0