NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
1
NUMERICAL ANALYSIS
Numerical solutions of algebraic equations, Method of iteration and
Newton-Raphson method, Rate of convergence, Solution of systems
of linear algebraic equations using Gauss elimination and Gauss-
Seidel methods, Finite differences, Lagrange, Hermite and spline
interpolation, Numerical differentiation and integration, Numerical
solutions of ODEs using Picard, Euler, modified Euler and Runge-
Kutta methods.
Numerical Solutions of Algebraic Equations
If the goal is to solve , then the function can be plotted in a way that the
intersection of with the -axis is visible and the root is found. We should have a rather
good idea as of where to look for the root. sometimes it is easier to verify that the function
is continuous (which hopefully it is) in which case all that we need is to find a point in
which , and a point , in which . The continuity will then guarantee (due
to the intermediate value theorem) that there exists a point between and for which
.
METHOD OF ITERATION This is a fairly simple method to find the roots of the equation , which
requires the problem to be written in the form -------(1)
for some function . .
We start with an initial guess to the solution, , and then calculate a new estimate as
-------(2)
This process is continued, at each step generating a new approximation If | | for all in the interval in which the root of equation (1) lies and is
the specified accuracy, then the iterations are stopped when the difference between
successive estimates becomes less than or equal to
.
ie. when | |
.
If the process is convergent, then taking a smaller value for , results in a more
accurate solution, although more iterations will need to be performed.
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
2
Y
X
Example 1: Find a real root of the equation -----(1)
(which has the exact solution ).
We first need to write the equation in the form , and then there is more than
one way of doing this.
One way is to write the equation as ⁄ so
⁄
If we start an initial guess of , then above iterative method then gives:
Iteration 1 2 3 4 5 6 7 8 9 10 11
2.5 3.12 3.49 3.71 3.83 3.91 3.95 3.97 3.98 3.99 3.99
3.12 3.49 3.71 3.83 3.91 3.95 3.97 3.98 3.99 3.99 4.00
Example 2: Find a real root of the equation on the interval [ ] with an
accuracy of .
Consider the equation -----(1)
One way is to write the equation as
√ -----(2)
Thus
√ and
⁄
Also | |[ ] |
⁄ |
(
⁄ )
⁄ =
√
Hence | |
i.e. when the absolute value of the difference does not exceed , the required
accuracy will be achieved and then iteration can be terminated.
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
3
Starting with , we obtain the following table
n √
√
0 0.75 1.3228756 0.7559289
1 0.7559289 1.3251146 0.7546517
2 0.7546517 1.3246326 0.7549263
At this stage, we find that | |
which is less than . The iteration is therefore terminated and the root to the
required accuracy is .
Example 3: Find the root of the equation correct to three decimal places.
Consider the equation -----(1)
We rewrite this equation as
-----(2)
Thus
and
and hence | |
Hence | |
Hence the iteration method can be applied to the equation (1) and we start with
⁄ . The successive iterates are , , ,
, , , ,
Hence we take the solution as 1.524 correct to three decimal places.
NEWTON-RAPHSON METHOD This method is also useful to solve the equation .
Let be an approximate root of and let be the correct root so
that . Then by Taylor’s series, we obtain
Neglecting the second and higher-order derivatives, we have
, which gives
A better approximation than is therefore given by , where
Successive approximations are given by , , , ..., , where
which is the Newton- Raphson formula.
Example 4: Use the Newton-Raphson method to find a root of the equation
.
Here and . Hence
gives
-----(1)
Choosing , we obtain and . Putting the value
and then putting the values of and in equation (1), we get
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
4
(
)
Now,
,
and .
Hence by putting the value and then putting the values of and in
equation (1), we get
(
)
Example 5: Find a root of the equation .
Here and .
The iteration formula therefore becomes
With , the successive iterates are given below
0 3.1416 -1.0 2.8233
1 2.8233 -0.0662 2.7986
2 2.7986 -0.0006 2.7984
3 2.7984 0.0 2.7984
Example 6: Find a real root of the equation , using the Newton-Raphson method.
Here and .
Let . Then and . Putting the value and then
putting the values of and in equation (1), we get
(
)
Now , and ,
So that
.
Proceeding in this way, we obtain and .
RATE OF CONVERGENCE The sequences ⁄ , ⁄ , ⁄ , and ⁄ all converge
to .
How fast?
⁄ , ⁄ , ⁄ , and ⁄ is true, but doesn’t really convey
how slow , or how fast . We use functions like these, and more
generally ⁄ and ⁄ as yardsticks (or benchmarks) with which to compare the
speed of convergence of algorithms.
In our context, we are usually trying to compute better and better approximations
to some value , and we want to know how fast the error | | is converging
toward . It is nice to be able to say (our approximations will eventually be
close to the answer), but will it take a second or a week? In the example above
, but .
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
5
We will these standard sequences above as benchmarks. If a sequence converges
about as slow as the , we will say it has a logarithmic rate of convergence, but if it
goes fast like , we will say its rate of convergence is exponential.
Here is the formal definition. Think of as one of the sequences given above. If { } is a positive sequence converging to , then we say that with rate of
convergence if and such that , | | .
In this case we write . Now suppose . What does that mean? What about ? Well, if we can write
⁄ , then we know that at least as fast as ⁄ , for some
constant , and if is not too big, it is probably safe to say that is about
⁄ times more accurate than is .
In the same way that we have just measured the error in a discrete setting, we can
measure the error associated with a continuous parameter. Suppose .
How fast? We say that as with rate of convergence if and
such that | | , | | .
We write . The idea here is that we are approximating by , and we would like to compare | | to , because we have a feeling for how
fast .
For example, . So for small , . But we can do better:
So we can write , or to give more information
, or even more: ⁄ , etc.
SOLUTION OF SYSTEMS OF LINEAR ALGEBRAIC EQUATIONS
(1) Gauss Elimination Method: Consider the system of linear equations in unknowns:
}
-----(1)
If , then the first equation is called the pivot equation, and is called the
pivot or pivot element. (Note that, if the coefficient of in the first equation of
system is zero, then interchange it with another equation in which coefficient of is
non-zero) Here we assume that .
To eliminate from second equation, we multiply the first equation by ⁄ and obtain
Adding the above equation to the second equation of (1), we obtain
(
) (
) (
)
-----(2)
Which can be written as
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
6
,
where ⁄ , ⁄ , ... , ⁄ , ⁄ etc.
Similarly, we can multiply the first equation of (1) by ⁄ and add it to its
third equation. This eliminates the unknown form the third equation of the system
and we obtain .
In a similar fashion, we can eliminate form the remaining equations of the system
and after eliminating form the last equation of (1), we obtain the system
}
-----(2)
Now assume that . i.e.
is second pivot element. We next eliminate
from the last equations of system (2). To eliminate from the third
equation, we multiply the second equation by ⁄ and add it to the third
equation. Repeating this process with the remaining equations, we obtain the system
}
-----(3)
Continuing the process of elimination of coefficients of next variable, at last we
obtain the upper triangular form:
}
-----(n)
where
indicates that the element has changed times. We thus have
completed the first step of elimination of unknowns and reduction to the upper-
triangular form.
We now have to obtain the required solution from the system (n). From the last
equation of this system, we obtain
This is then substituted in the (n-1)th equation to obtain and the process is
repeated to compute the other unknowns.
We have therefore first computed , then , , ..., , , in that order. Due to
this reason, the process is called back substitution.
Example 7: Use Gauss elimination to solve
, , . -----(1)
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
7
First we eliminate form second and third equation by using first equation (first pivot
equation). For this we multiply the first equation by ⁄ and add to the second to
get .
Similarly, we multiply the first equation by ⁄ and add it to third equation to get
.
Hence given system reduced to , , .
-----(2)
Now in the similar way, we can eliminate form second and third equation of system
(2), by multiplying by to second equation and add it to third equation to get
or .
.The upper triangular form is therefore given by
.
It follows that the required solution is , and .
Example 8: Solve the system
.
Multiply first equation by ⁄ i.e. by and add it to the
second equation to get or .
Final system becomes
.
Solving these equations, we get and .
(2) Gauss-Jordan method: This is a modification of the Gauss elimination method; the essential difference being
that when an unknown is eliminated, it is eliminated from all equations.
Example 7*: Use Gauss-Jordan method to solve
, , . -----(1)
Elimination of from the second and third equation is done as in ‘Gauss elimination’
and we obtain the system
(
) (
)
(
) (
)
Next, the unknown is eliminated from both the first and third equations. This gives
us
and .
Hence the system becomes: , , .
From this, we get , and .
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
8
(3) Gauss-Seidel method: Consider the system of linear equations in unknowns:
}
-----(1)
Assume that the diagonal elements . If this is not the case, then the equations
should be arranged so that this condition is satisfied. Now, we rewrite the system (1)
as
⁄ ⁄ ⁄ ⁄
⁄ ⁄ ⁄ ⁄
⁄ ⁄ ⁄ ( ⁄ ) }
-----(2)
We substitute the first approximation (
) for into the
right-hand side of first equation of system (2) and denote the result as
. In the
second equation of that system, we substitute (
) for
and denote the result as
. In the third, we substitute (
) for
and denote the result as
. In this manner, we complete the first
stage of iteration and the entire process is repeated till the values of
are obtained to the accuracy required.
It is called the method of successive displacements, of Gauss-Seidel method.
Example 8: Solve the system
.
We rewrite these equations as follows:
.
Substitute the first approximation for in first equation, we get
Substitute for in second equation, we get
Substitute for in third equation, we get
Substitute for in fourth equation, we get
Substitute the second approximation for in first
equation, we get
Substitute for in second equation, we get
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
9
Substitute for in third equation, we get
Substitute for in fourth equation, we get
Continuing in this way, we get table of approximations as follows:
1 0.3 1.56 2.886 -0.1368
2 0.8869 1.9523 2.9566 -0.0248
3 0.9836 1.9899 2.9924 -0.0042
4 0.9968 1.9982 2.9987 -0.0008
5 0.9994 1.9997 2.9998 -0.0001
6 0.9999 1.9999 3.0 0.0
7 1.0 2.0 3.0 0.0
FINITE DIFFERENCES: 1) Forward Differences: Assume that we have a table of values , of any function
, the values of being equally spaced, i.e. , .
If denote a set of values of , then
are called the differences of .
Denoting these differences by respectively, we have
,
where is called the forward difference operator and are called first
forward differences.
The differences of first forward differences are called second forward differences and
are denoted by .
Similarly, one can define third forward differences, fourth forward differences, etc.
Thus
.
Forward Difference Table
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
10
2) Backward Differences:
Assume that we have a table of values , of any function
, the values of being equally spaced, i.e. , .
If denote a set of values of , then
are called the differences of .
Denoting these differences by respectively, we have
,
where is called the backward difference operator and are called first
backward differences.
The differences of first backward differences are called second backward differences
and are denoted by .
Similarly, one can define third backward differences, fourth backward differences,
etc.
Thus
.
Backward Difference Table
3) Central Differences: The central difference operator is defined by the relations
.
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
11
Central Difference Table
4) Symbolic Relations and Separation of Symbols: The averaging or mean operator is defined by the equation:
( ⁄ ⁄ )
The shift operator is defined by the equation:
Which shows that the effect of is to shift the functional value to the next higher
value .
A second equation with gives us:
and in general,
Relationship between and :
or
The inverse shift operator is defined by the equation:
Which shows that the effect of is to shift the functional value to the previous
lower value .
Relationship between and :
or
Similarly, we have the following relations:
⁄ ⁄
⁄ ( ⁄ ⁄ )
⁄ or √ ⁄
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
12
⁄
Finally we define the operator such that
To relate to , we start with the Taylor’s series
This can be written in the symbolic form
(
)
i.e.
Lagrange interpolation Formula: Let be continuous and differentiable times in the interval . Given
the points where the values of need
not necessarily be equally spaced, we wish to find a polynomial of degree , say
, such that
The polynomial will be given by
∑
Where ,
[ ]
. The polynomial defined above is called Lagrange’s interpolation formula.
The coefficients
defined in above are called Lagrange interpolation coefficients.
Interchanging and in Lagrange’s interpolation formula, we obtain the formula:
∑
Which is useful for inverse interpolation.
Note that the Lagrange interpolating polynomial is unique.
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
13
Example 9: Certain corresponding values of and are and . Find .
.
.
Example 10: If , , and , find .
Example 11: Find the Lagrange interpolating polynomial of degree 2 approximating the
function defined by the following table of values. Hence determine the value of
.
2.0 0.69315
2.5 0.91629
3.0 1.09861
We have
i.e. with usual notation, we have
Similarly, we find
Hence the Lagrange interpolating polynomial of degree 2 is given by
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
14
∑
, which is the required quadratic
Lagrange polynomial.
Hence
, after simplification.
Hermite’s interpolation Formula: Given the set of data points it is required to determine a
polynomial of the least degree, say , such that
and ,
Where the primes denote differentiation with respect to . The polynomial is called Hermite’s interpolation polynomial. This polynomial is given by
∑ [ ][ ]
∑ [ ]
. Where
Example 12: Find the third order Hermite polynomial passing through the points
Putting in Hermite’s interpolation formula, we obtain
[ ][ ] [ ][ ]
[ ]
[ ]
_____(1)
Since
where . Hence
Then (1) simplifies to
*
+
*
+
which is the required Hermite formula.
Example 13: Determine the Hermite polynomial of degree 5, which fits the following data
and hence find an approximate value of .
⁄
2.0 0.69315 0.5
2.5 0.91629 0.4000
3.0 1.09861 0.33333
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
15
The polynomials have already been computed in Example 11. These are
We therefore obtain
.
Hence
Thus
[ ] [ ] [ ]
Thus
[ ]
[ ]
[ ]
Spline interpolation: Let the given data points be Where
And let Further, let be the spline of degree one defined in the interval [ ]. Obviously, represents a straight line joining the points and . Hence we write , ____(3)
Where
Setting successively in equation (3), we obtain different splines of degree one
valid in the subintervals 1 to n, respectively. It is easily seen that is continuous at both
the end points.
Example 14: Given the set of data points (1, -8), (2, -1) and (3,18) satisfying the function
, find the linear splines satisfying the given data. Determine the approximate values
of and . Let A(1, -8), B(2, -1) and C(3,18) be given three points. Equation of AB is
And equation of BC is
.
Since belongs to the interval [ ], we have
,
And .
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
16
Numerical differentiation and integration:
Using Newton’s forward difference formula Consider Newton’s forward difference formula:
where . ____(2)
Then
(
)
This formula can be used for computing the value of ⁄ for non-tabular values of
.
For tabular values of , the formula (3) takes a simpler form, for by setting , we
obtain and hence equation (3) becomes
[
]
(
)
Differentiating (3) once again, we get
(
)
from which we obtain
*
+
(
)
Formula for computing higher derivatives may be obtained by successive
differentiation.
Using Newton’s backward difference formula Consider Newton’s backward difference formula:
where . ____(2)
Then
(
)
This formula can be used for computing the value of ⁄ for non-tabular values of
.
For tabular values of , the formula (3) takes a simpler form, for by setting , we
obtain and hence equation (3) becomes
[
]
(
)
Differentiating (3) once again, we get
(
)
from which we obtain
*
+
(
)
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
17
Formula for computing higher derivatives may be obtained by successive
differentiation.
Using Stirling’s formula
[
]
(
)
And
*
+
(
)
If the derivative is required near the end of a table, one of the following formulae may
be used to obtain better accuracy
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
18
NUMERICAL DIFFERENTIATION AND INTEGRATION
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
25
Numerical solutions of ODEs using Picard
Consider the first order differential equation
With the initial condition,
Integrating this equation, we get
∫
The above equation in which the unknown function appears under the integral sign, is
called an integral equation.
Such as equation can be solved by the method of successive approximations in which the first
approximation to is obtained by putting for on right side of equation (3), and we write
∫
The integral on the right can now be solved and the resulting is substituted for in the
integrand of equation (3) to obtain the second approximation :
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
26
∫ ( )
Proceeding in this way, we obtain and , where
∫ ( )
Hence this method yields a sequence of approximations
and it can be proved that if the function is bounded in some region about the point
and if satisfies the Lipschitz condition, viz.,
| ̅ | | ̅| being constant _____(5)
Then the sequence converges to the solution of equation (1)
Example 1: Solve the equation , subject to the condition when .
Solution: Here and when means and
hence
∫
∫
∫
∫
*
+
The second approximation is
∫ ( )
∫ (
)
∫ *
+
∫ *
+
Example 2: Given the differential equation
With the initial condition when , use Picard’s method to obtain for
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
27
correct to three decimal places.
Solution: We have
∫
∫
Setting , we obtain
∫
∫
∫
∫
*
+
And
∫ ( )
∫ (
)
∫
( )
∫
( )
( )
∫
(
)
[ ] (
)
So that and agree to the first term, viz.,
.
To find the range of values of so that the series with the term
alone will give the result
correct to three decimal places, we put
Which yields . Hence
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
28
Euler’s Method:
Suppose that we wish to solve the equation
For values of at
Integrating equation (1), we obtain
∫
Assuming that in , this gives Euler’s formula
Similarly for the range , we have
∫
Substituting for in , we obtain
Proceeding in this way, we obtain the general formula
Example 3: To illustrate Euler’s method, we consider the differential equation with
the condition .
Solution: Here we have
Thus with , we obtain
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
29
.
The exact solution is and from this value at is 0.9608.
Modified Euler’s Method:
Instead of approximating by in
We now approximate the integral by means of trapezoidal rule to obtain
[ ]
We thus obtain the iteration formula
[ (
)]
Where is th approximation to . The iteration formula (7) can be started by choosing
from Euler’s formula:
Example 4: Determine the value of when given that and .
Solution: We take . With and we have Hence
Euler’s formula gives
Further, and ( ) The average of and (
) is
1.0262. The value of can therefore be computed by using equation (7) and we obtain
Repeating the procedure, we obtain Hence we take which is
correct to four decimal places.
Next, with and we continue the procedure to obtain i.e.
the value of when . The results are
Hence we consider the the value of when is 1.1055.
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
30
Runge-Kutta Methods
If we substitute on the right side of equation
[ ]
We obtain
[ ]
Where If we now set
and
then the above equation becomes
which is the second order Runge-Kutta formula. The error in this formula can be shown to be
order by expanding both sides by Taylor’s series.
Thus, the left side gives
and on the right side
[
]
Since
we obtain
[ ]
so that the right of (8) gives
[ ]
It therefore follows that the Taylor series expansions of both sides of (8) agree up to terms of
order which means that the error in this formula is of order .
More generally, if we set
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
31
where
}
then the Taylor series expansions of both sides of the last equation in (9) gives the identity
(
)
(
)
Equating the coefficients of and its derivatives on both sides,we obtain the relations
Clearly and is assigned any value arbitrarily, then the remaining parameters can
be determined uniquely. If we set, for example, then we immediately obtain
which gives formula (8).
It follows, therefore, that there are several second-order Runge-Kutta formulas and that
formulae (9) and (10) constitute just one of several such formulae.
Higher-order Runge-Kutta formulae exist, of which we mention only the fourth-order
formula defined by
where
}
where the parameters have to be determined by expanding both sides of the first equation of
(13) by Taylor’s series and securing agreement of terms up to and including those containing
. The hoice of the parameters is again, arbitrary and we have therefore several fourth-order
Runge-Kutta formulae. If, for example, we set
(√ )
√
√
√
(
√ )
(
√ )}
we obtain the method of Gill, whereas the choice
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
32
}
leads to the fourth-order Runge-Kutta formula, the most commonly used one in practice:
where
(
)
(
)
}
in which the error is of order . Complete derivation of the formula is exceedingly
complicated, and the interested reader is referred to the book by Levy and Baggot. We
illustrate here the use of the fourth-order formula by means of examples.
Example 5: Given ⁄ where , find and correct to four
decimal places.
Solution: (i) Runge-Kutta second-order formula: With , we find and
.Hence
To determine , we note that and .
Hence, and .
It follows that
.
Proceeding in a similar way, we obtain
and
We next choose and compute and directly. With , and
, we obtain and and hence Similarly, we obtain
.
From the analytical solution , the exact values of and are
respectively 2.4214 and 2.8918. To study the order of convergence of this method, we
tabulate the values as follows:
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
33
Computed Exact Difference Ratio
0.2 2.4214 0.0004
3.5
0.0014
0.4 2.8918 0.0009
4.2
0.0038
It follows that the method has -order of convergence.
(ii) Runge-Kutta fourth-order formula: To determine we have and
. We then obtain Hence
Proceeding similarly, we obtain .
Example 6: Given ⁄ where when , find and
.
Solution: We take . With we obtain from equations:
(
)
(
)
The following values:
and
Which is correct to four decimal places.
To compute we take , and With these values, we have
[ ] [ ]
[ ] [ ]
And
Correct to two decimal places.
Finally, taking and and proceeding as above, we obtain
NET/SET Preparation Numerical Analysis By S. M. CHINCHOLE
34
Example 7: We consider the initial value problem with the condition
Solution: The following table gives the values of by different methods, the exact
value being 1.16722193. It is seen that the fourth-order Runge-Kutta method gives the
accurate value for
Method h Computed value
Euler 0.2 1.100 000 00
0.1 1.132 500 00
0.05 1.149 567 58
Modified Euler 0.2 1.100 000 00
0.1 1.150 000 00
0.05 1.162 862 42
Fourth-order Runge-Kutta 0.2 1.167 220 83
0.1 1.167 221 86
0.05 1.167 221 93