Download - Numerical Differentiation and Integration
file:nd&i.ppt p. 1
NUMERICAL DIFFERENTIATION AND INTEGRATION
ENGR 351 Numerical Methods for Engineers
Southern Illinois University Carbondale
College of EngineeringDr. L.R. ChevalierDr. B.A. DeVantier
file:nd&i.ppt p. 2
Numerical Differentiation and Integration
• Calculus is the mathematics of change.• Engineers must continuously deal with
systems and processes that change, making calculus an essential tool of our profession.
• At the heart of calculus are the related mathematical concepts of differentiation and integration.
file:nd&i.ppt p. 3
Differentiation
• Dictionary definition of differentiate - “to mark off by differences, distinguish; ..to perceive the difference in or between”
• Mathematical definition of derivative - rate of change of a dependent variable with respect to an independent variable
x
xfxxf
xy ii
file:nd&i.ppt p. 4
y
x
f x x f x
xi i
f xi
f x xi
y
x
f(x)
x
file:nd&i.ppt p. 5
Integration• The inverse process of differentiation• Dictionary definition of integrate - “to
bring together, as parts, into a whole; to unite; to indicate the total amount”
• Mathematically, it is the total value or summation of f(x)dx over a range of x. In fact the integration symbol is actually a stylized capital S intended to signify the connection between integration and summation.
file:nd&i.ppt p. 6
f(x)
x
I f x dxa
b
file:nd&i.ppt p. 7
Mathematical Backgroundd
dxx
d
dxe
d
dxx
d
dxa
d
dxx
d
dxx
d
dxx
if u and v are functions of x
d
dxu
d
dxuv
x
x
n
n
sin ? ?
cos ? ?
tan ? ?
ln ?
? ( ) ?
file:nd&i.ppt p. 8
udv
u du
a dx
dx
x
e dx
n
bx
ax
?
?
?
?
?
Mathematical Background
file:nd&i.ppt p. 9
Overview
• Newton-Cotes Integration FormulasTrapezoidal ruleSimpson’s RulesUnequal SegmentsOpen Integration
• Integration of EquationsRomberg Integration Gauss QuadratureImproper Integrals
file:nd&i.ppt p. 10
Overview
• Numerical DifferentiationHigh accuracy formulasRichardson’s extrapolationUnequal spaced dataUncertain data
• Applied problems
file:nd&i.ppt p. 11
Specific Study Objectives• Understand the derivation of the
Newton-Cotes formulas• Recognize that the trapezoidal and
Simpson’s 1/3 and 3/8 rules represent the areas of 1st, 2nd, and 3rd order polynomials
• Be able to choose the “best” among these formulas for any particular problem
file:nd&i.ppt p. 12
Specific Study Objectives
• Recognize the difference between open and closed integration formulas
• Understand the theoretical basis of Richardson extrapolation and how it is applied in the Romberg integration algorithm and for numerical differentiation
file:nd&i.ppt p. 13
Specific Study Objectives
• Recognize why both Romberg integration and Gauss quadrature have utility when integrating equations (as opposed to tabular or discrete data).
• Understand the application of high-accuracy numerical-differentiation.
• Recognize data error on the processes of integration and differentiation.
file:nd&i.ppt p. 14
Newton-Cotes Integration
• Common numerical integration scheme• Based on the strategy of replacing a
complicated function or tabulated data with some approximating function that is easy to integrate
file:nd&i.ppt p. 15
Newton-Cotes Integration
• Common numerical integration scheme• Based on the strategy of replacing a
complicated function or tabulated data with some approximating function that is easy to integrate
I f x dx f x dx
f x a a x a x
a
b
n
a
b
n nn
0 1 ....
file:nd&i.ppt p. 16
Newton-Cotes Integration
• Common numerical integration scheme• Based on the strategy of replacing a
complicated function or tabulated data with some approximating function that is easy to integrate
I f x dx f x dx
f x a a x a x
a
b
n
a
b
n nn
0 1 .... fn(x) is an nth orderpolynomial
file:nd&i.ppt p. 17
The approximation of an integral by the area under- a first order polynomial- a second order polynomial
We can also approximated the integral by using a series of polynomials applied piece wise.
0
1
2
3
4
5
0 5 10
x
f(x)
0
1
2
3
4
5
0 5 10
x
f(x)
file:nd&i.ppt p. 18
0
1
2
3
4
5
0 1 2 3 4 5 6 7 8 9 10
x
f(x)
An approximation of an integral by the area under straight line segments.
file:nd&i.ppt p. 19
Newton-Cotes Formulas
• Closed form - data is at the beginning and end of the limits of integration
• Open form - integration limits extend beyond the range of data.
0
1
2
3
4
5
0 1 2 3 4 5 6 7 8 9 10
x
f(x)
0
1
2
3
4
5
0 1 2 3 4 5 6 7 8 9 10
x
f(x)
file:nd&i.ppt p. 20
Trapezoidal Rule
• First of the Newton-Cotes closed integration formulas
• Corresponds to the case where the polynomial is a first order
I f x dx f x dx
f x a a xa
b
a
b
n
1
0 1
file:nd&i.ppt p. 21
I f x dx f x dx
f x a a xa
b
a
b
n
1
0 1
A straight line can be represented as:
f x f af b f a
b ax a1
file:nd&i.ppt p. 22
I f x dx f x dx
f af b f a
b ax a dx
a
b
a
b
a
b
1
Integrate this equation. Results in the trapezoidal rule.
I b a
f a f b
2
file:nd&i.ppt p. 23
I b a
f a f b
2
The concept is the same but the trapezoid is on its side.
heig
ht
base
base
widthheig
ht
heig
ht
file:nd&i.ppt p. 24
Error of the Trapezoidal Rule
E f b a
where a b
t
1
123
' '
This indicates that is the function being integrated islinear, the trapezoidal rule will be exact.
Otherwise, for section with second and higher order derivatives (that is with curvature) error can occur.
A reasonable estimate of x is the average value of b and a
file:nd&i.ppt p. 25
Multiple Application of the Trapezoidal Rule
• Improve the accuracy by dividing the integration interval into a number of smaller segments
• Apply the method to each segment• Resulting equations are called
multiple-application or composite integration formulas
file:nd&i.ppt p. 26
I f x dx f x dx f x dx
I hf x f x
hf x f x
hf x f x
x
x
x
x
x
x
n n
n
n
( ) ( ) ( )0
1
1
2
1
0 1 1 2 1
2 2 2
Multiple Application of the Trapezoidal Rule
where there are n+1 equally spaced base points.
file:nd&i.ppt p. 27
I f x dx f x dx f x dx
I hf x f x
hf x f x
hf x f x
I b af x f x f x
n
x
x
x
x
x
x
n n
i ni
n
n
n
( ) ( ) ( )0
1
1
2
1
0 1 1 2 1
01
1
2 2 2
2
2
We can group terms to express a general form
} }width average height
file:nd&i.ppt p. 28
I b af x f x f x
n
i ni
n
01
1
2
2} }
width average height
The average height represents a weighted averageof the function values
Note that the interior points are given twice the weightof the two end points
Eb a
nfa
3
212' '
file:nd&i.ppt p. 29
EXAMPLEEvaluate the following integral using the trapezoidal rule and h = 0.1
I b af x f x f x
n
i ni
n
01
1
2
2
I e dxx2
1
1 6. Example Problem
0
10
20
30
40
50
60
0 0.5 1 1.5 2
xf(
x)
hb a
n
file:nd&i.ppt p. 30
Simpson’s 1/3 Rule
• Corresponds to the case where the function is a second order polynomial
I f x dx f x dx
f x a a x a x
a
b
a
b
n
2
0 1 22
file:nd&i.ppt p. 31
Simpson’s 1/3 Rule
• Designate a and b as x0 and x2, and estimate f2(x) as a second order Lagrange polynomial
I f x dx f x dx
x x x x
x x x xf x dx
a
b
a
b
x
x
2
1 2
0 1 0 20
0
2
.......
file:nd&i.ppt p. 32
Simpson’s 1/3 Rule
• After integration and algebraic manipulation, we get the following equations
Ih
f x f x f x
b af x f x f x
34
4
6
0 1 2
0 1 2} }width average height
file:nd&i.ppt p. 33
Error
E f b a
where a b
t
1
123
' '
Single application of Trapezoidal Rule.
Single application of Simpson’s 1/3 Rule
E f b at 1
28804 5( )
file:nd&i.ppt p. 34
Multiple Application of Simpson’s 1/3 Rule
I f x dx f x dx f x dx
I b a
f x f x f x f x
n
Eb a
nf
x
x
x
x
x
x
i j nj
n
i
n
a
n
n
( ) ( ) ( )
, , .., , ..
0
1
1
2
1
02 4 6
2
1 3 5
1
5
44
4 2
3
180
file:nd&i.ppt p. 35
I b a
f x f x f x f x
n
i j nj
n
i
n
02 4 6
2
1 3 5
1
4 2
3, , .., , ..
The odd points represent the middle term for each application. Hence carry the weight 4The even points are common to adjacent applications and are counted twice.
file:nd&i.ppt p. 36
Simpson’s 3/8 Rule
• Corresponds to the case where the function is a third order polynomial
I f x dx f x dx
f x a a x a x a x
Ih
f x f x f x f x
a
b
a
b
n
3
0 1 22
33
0 1 2 3
38
3 3
file:nd&i.ppt p. 37
Integration of Unequal Segments
• Experimental and field study data is often unevenly spaced
• In previous equations we grouped the term (i.e. hi) which represented segment width.
I b af x f x f x
n
I hf x f x
hf x f x
hf x f x
i ni
n
n n
01
1
0 1 1 2 1
2
2
2 2 2
file:nd&i.ppt p. 38
Integration of Unequal Segments
• We should also consider alternately using higher order equations if we can find data in consecutively even segments
trapezoidalrule
1/3rule
3/8rule
trapezoidalrule
file:nd&i.ppt p. 39
EXAMPLEIntegrate the following using the trapezoidal rule, Simpson’s 1/3 Rule and a multiple application ofthe trapezoidal rule with n=2. Compare results withthe analytical solution.
xe dxx2
0
4
0
5000
10000
15000
20000
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
f(x)
file:nd&i.ppt p. 40
0
5000
10000
15000
20000
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
f(x)
Simpson’s 1/3 Rule
f(2) = 109.196
Ih
f x f x f x
b af x f x f x
t
34
4
6
4 00 4 109 196 1192383
68240 41
5216 93 8240 41
5216 93100 57 96%
0 1 2
0 1 2
. ..
. .
..
file:nd&i.ppt p. 41
I b af x f x f x
n
i ni
n
t
01
1
2
2
4 00 2 109 196 1192383
2 212142 22
5216 93 12142 22
5216 93100 133%
. ..
. .
.
Multiple Application ofthe Trapezoidal Rule
We are obviously not doing very well on our estimates.Lets consider a scheme where we “weight” the estimates....end of example
0
5000
10000
15000
20000
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
f(x)
file:nd&i.ppt p. 42
Integration of Equations
• Integration of analytical as opposed to tabular functions
• Romberg Integration Richardson’s ExtrapolationRomberg Integration Algorithm
• Gauss Quadrature• Improper Integrals
file:nd&i.ppt p. 43
Richardson’s Extrapolation• Use two estimates of an integral to compute a
third more accurate approximation• The estimate and error associated with a
multiple application trapezoidal rule can be represented generally as:I = I(h) + E(h)where I is the exact value of the integralI(h) is the approximation from an n-segment
applicationE(h) is the truncation errorh is the step size (b-a)/n
file:nd&i.ppt p. 44
Make two separate estimates using step sizesof h1 and h2 .
I(h1) + E(h1) = I(h2) + E(h2)
Recall the error of the multiple-application of the trapezoidalrule
Eb a
h f12
2 ' '
Assume that is constant regardless of the step sizef ' '
E h
E h
h
h1
2
12
22
file:nd&i.ppt p. 45
E h
E h
h
h
E h E hh
h
1
2
12
22
1 21
2
2
Substitute into previous equation:
I(h1) + E(h1) = I(h2) + E(h2)
E h
I h I h
h
h
21 2
1
2
2
1
file:nd&i.ppt p. 46
Thus we have developed an estimate of the truncation error in terms of the integral estimates and their step sizes. This estimate can then be substituted into:
I = I(h2) + E(h2)
to yield an improved estimate of the integral:
I I hh
h
I h I h
2
1
2
2 2 1
1
1
E h
I h I h
h
h
21 2
1
2
2
1
file:nd&i.ppt p. 47
I I hh
h
I h I h
2
1
2
2 2 1
1
1
What is the equation for the special case where the interval is halved?
i.e. h2 = h1 / 2
file:nd&i.ppt p. 48
hh
hh
I I h I h I h
collecting terms
I I h I h
1
2
2
2
2 2 2 1
2 1
2 2
1
2 1
4
3
1
3
file:nd&i.ppt p. 49
EXAMPLE
Use Richardson’s extrapolation to evaluate:
xe dxx2
0
4
0
5000
10000
15000
20000
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
f(x)
file:nd&i.ppt p. 50
I I h I h
I I h I h
I I h I h
II I
m l
m l
j k
kj k j k
k
4
3
1
316
15
1
1564
63
1
63
4
4 1
2 1
11 1 1
1,, ,
We can continue to improve the estimate by successive halving of the step size to yield a general formula:
k = 2; j = 1
ROMBERG INTEGRATION
Note:the subscriptsm and l refer tomore and lessaccurate estimates
file:nd&i.ppt p. 51
Error orders for j values i = 1 i = 2 i = 3 i = 4j O(h2) O(h4) O(h6) O(h8)
1 h I1,1 I1,2 I1,3 I1,4
2 h/2 I2,1 I2,2 I2,3
3 h/4 I3,1 I3,2
4 h/8 I4,1
Trapezoidal Simpson’s 1/3 Simpson’s 3/8 Boole’s
Rule Rule Rule Rule
hI63
1hI
63
64I m
Following a similar pattern to Newton divided differences, Romberg’s Table can be produced
file:nd&i.ppt p. 52
Gauss Quadrature
f(x)
x
f(x)
x
Extend the areaunder the straightline
file:nd&i.ppt p. 53
Method of Undetermined Coefficients
Recall the trapezoidal rule
I b a
f a f b
2
This can also be expressed as
I c f a c f b 0 1
where the c’s are constant
Before analyzingthis method,answer this question.What are twofunctions thatshould be evaluated exactlyby the trapezoidalrule?
file:nd&i.ppt p. 54
The two cases that should be evaluated exactlyby the trapezoidal rule: 1) y = constant 2) a straight line
f(x)
x
y = 1
(b-a)/2-(b-a)/2
f(x)
x
y = x
(b-a)/2
-(b-a)/2
file:nd&i.ppt p. 55
Thus, the following equalities should hold.
I c f a c f b
c c dx
cb a
cb a
xdx
b a
b a
b a
b a
0 1
0 12
2
0 12
2
1
2 2
FOR y=1since f(a) = f(b) =1
FOR y =xsince f(a) = x =-(b-a)/2andf(b) = x =(b-a)/2
file:nd&i.ppt p. 56
Evaluating both integrals
c c b a
cb a
cb
0 1
0 12
1
20
For y = 1
For y = x
Now we have two equations and two unknowns, c0 and c1.
Solving simultaneously, we get :
c0 = c1 = (b-a)/2
Substitute this back into: I c f a c f b 0 1
file:nd&i.ppt p. 57
I b a
f a f b
2
We get the equivalent of the trapezoidal rule.
DERIVATION OF THE TWO-POINT GAUSS-LEGENDRE FORMULA
I c f x c f x 0 0 1 1
Lets raise the level of sophistication by:- considering two points between -1 and 1- i.e. “open integration”
file:nd&i.ppt p. 58
f(x)
x-1 x0 x1 1
Previously ,we assumed that the equation fit the integrals of a constant and linear function.
Extend the reasoning by assuming that it also fits the integral of a parabolic and a cubic function.
file:nd&i.ppt p. 59
c f x c f x dx c c
c f x c f x xdx c x c x
c f x c f x x dx c x c x
c f x c f x x dx c x c x
0 0 1 1
1
1
0 1
0 0 1 1
1
1
0 0 1 1
0 0 1 12
1
1
0 02
1 12
0 0 1 13
1
1
0 03
1 13
1 2 1 1 2
0 0
2 3 2 3
0 0
`
/ /
Solve these equations simultaneously
f(xi) is either 1, xi, xi2 or xi
3
file:nd&i.ppt p. 60
c c
x
x
I f f
0 1
0
1
1
1
31
31
3
1
3
This results in the following
The interesting result is that the simple additionof the function values at 1
3
1
3and
file:nd&i.ppt p. 61
However, we have set the limit of integration at -1 and 1.
This was done to simplify the mathematics. A simplechange in variables can be use to translate other limits.
Assume that the new variable xd is related to theoriginal variable x in a linear fashion.
x = a0 + a1xd
Let the lower limit x = a correspond to xd = -1 and the upperlimit x=b correspond to xd=1
a = a0 + a1(-1) b = a0 + a1(1)
file:nd&i.ppt p. 62
a = a0 + a1(-1) b = a0 + a1(1)
SOLVE THESE EQUATIONSSIMULTANEOUSLY
ab a
ab a
0 12 2
x a a x
b a b a xd
d
0 1 2
substitute
file:nd&i.ppt p. 63
x a a x
b a b a x
dxb a
dx
dd
d
0 1 2
2
These equations are substituted for x and dx respectively.
Let’s do an example to appreciate the theorybehind this numerical method.
file:nd&i.ppt p. 64
EXAMPLEEstimate the following using two-point Gauss Legendre:
xe dxx2
0
4
0
5000
10000
15000
20000
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
x
f(x)
file:nd&i.ppt p. 65
Higher-Point Formulas
I c f x c f x c f xn n 0 0 1 1 1 1
For two point, we determined that c0 =c1 = 1
For three point:
c0 = 0.556 x0=-0.775c1= 0.889 x1=0.0c2= 0.556 x2=0.775
file:nd&i.ppt p. 66
Higher-Point Formulas
Your text goes on to provide additional weightingfactors (ci’s) and function arguments (xi’s)in Table 22.1 p. 623.
file:nd&i.ppt p. 67
Improper Integrals
f x dxb
How do we deal withintegrals that do nothave finite limitsand boundedintegrands?
Our answer will focuson improper integralswhere one limit (upper orlower) is infinity.
file:nd&i.ppt p. 68
f x dx f x dx f x dxb A
A
b
Choose -A so that is is sufficiently large toapproach zero asymptotically.
Once chosen, evaluate the second part using aNewton-Cotes formula.
Evaluate the first part with the following identity.
f x dxt
ft
dt for aba
b
b
a
1 1
021
1
file:nd&i.ppt p. 69
Numerical Differentiation• Forward finite divided difference• Backward finite divided difference• Center finite divided difference• All based on the Taylor Series
.........
!2
''' 2
1 hxf
hxfxfxf iiii
file:nd&i.ppt p. 70
Forward Finite Difference
21
1
21
2
'''
'
.........!2
'''
hOhxf
h
xfxfxf
hOh
xfxfxf
hxf
hxfxfxf
iiii
iii
iiii
file:nd&i.ppt p. 71
Forward Divided Difference
f xf x f x
x xO x x
f
hO hi
i i
i ii i
i'
1
11
f(x)
x
(xi, yi)
(x i+1,y i+1)
estimateac
tual
file:nd&i.ppt p. 72
f xf
hO hi
i'
first forward divided difference
Error is proportional tothe step size
O(h2) error is proportional to the square of the step size
O(h3) error is proportional to the cube of the step size
file:nd&i.ppt p. 73
f(x)
x
estimate
actual
(xi,yi)
(xi-1,yi-1)
file:nd&i.ppt p. 74
f x f x f x hf x
h
f x f x f x hf x
h
f xf x f x
h
f
h
i i ii
i i ii
ii i i
12
12
1
2
2
'' '
!......
'' '
!.....
'
Backward Difference Approximation of theFirst Derivative
Expand the Taylor series backwards
The error is still O(h)
file:nd&i.ppt p. 75
Centered Difference Approximation of theFirst Derivative
Subtract backward difference approximationfrom forward Taylor series expansion
211
211
1
1
21
2'
6
''''2
'
....!2
'''
hOh
xfxfxf
hxf
hxfxfxf
xx
xfxfxf
hxf
hxfxfxf
iii
iiii
ii
iii
iiii
file:nd&i.ppt p. 76
f(x)
x
estimate
actual
(xi,yi)
(xi-1,yi-1)
(xi+1,yi+1)
211
2' hO
hxfxf
xf iii
file:nd&i.ppt p. 77
f(x)
xf(x)
x
f(x)
xf(x)
x
true derivative forwardfinite divideddifference approx.
backwardfinite divideddifference approx.
centeredfinite divideddifference approx.
file:nd&i.ppt p. 78
Numerical Differentiation
• Forward finite divided differences Fig. 23.1
• Backward finite divided differences Fig. 23.2
• Centered finite divided differences Fig. 23.3
• First - Fourth derivative
file:nd&i.ppt p. 79
Richardson Extrapolation
• Two ways to improve derivative estimates¤ decrease step size¤ use a higher order formula that employs
more points
• Third approach, based on Richardson extrapolation, uses two derivatives estimates to compute a third, more accurate approximation
file:nd&i.ppt p. 80
Richardson Extrapolation
I I hh
h
I h I h
Special case where h h
I I h I h
In a similar fashion
D D h D h
2
1
2
2 2 1
21
2 1
2 1
1
1
24
3
1
3
4
3
1
3
For a centered differenceapproximation withO(h2) the application ofthis formula will yielda new derivative estimateof O(h4)
file:nd&i.ppt p. 81
EXAMPLEGiven the following function, use Richardson’s extrapolation to determine the derivative at 0.5.
f(x) = -0.1x4 - 0.15x3 - 0.5x2 - 0.25x +1.2
Note:
f(0) = 1.2f(0.25) =1.1035f(0.75) = 0.636f(1) = 0.2
file:nd&i.ppt p. 82
Derivatives of Unequally Spaced Data
• Common in data from experiments or field studies• Fit a second order Lagrange interpolating
polynomial to each set of three adjacent points, since this polynomial does not require that the points be equispaced
• Differentiate analytically
f x f xx x x
x x x xf x
x x x
x x x x
f xx x x
x x x x
ii i
i i i ii
i i
i i i i
ii i
i i i i
'
11
1 1 1
1 1
1 1
11
1 1 1
2 2
2
file:nd&i.ppt p. 83
Derivative and Integral Estimates for Data with
Errors• In addition to unequal spacing, the other
problem related to differentiating empirical data is measurement error
• Differentiation amplifies error• Integration tends to be more forgiving• Primary approach for determining derivatives
of imprecise data is to use least squares regression to fit a smooth, differentiable function to the data
• In absence of other information, a lower order polynomial regression is a good first choice
file:nd&i.ppt p. 84
0
100
200
0 10 20
t
y0
102030
0 5 10 15
t
dy/d
t
0
50
100
150
200
0 5 10 15
t
y
0
10
20
30
40
0 5 10 15
t
dy
/dt