1519 differentiation-integration-02

NUMERICAL DIFFERENTIATION AND INTEGRATION

ENGR 351 Numerical Methods for EngineersSouthern Illinois University CarbondaleCollege of EngineeringDr. L.R. ChevalierDr. B.A. DeVantier

Copyright © 2003 by Lizette R. Chevalier

Permission is granted to students at Southern Illinois University at Carbondaleto make one copy of this material for use in the class ENGR 351, NumericalMethods for Engineers. No other permission is granted.

All other rights are reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means,electronic, mechanical, photocopying, recording, or otherwise, withoutthe prior written permission of the copyright owner.

Specific Study Objectives• Understand the derivation of the

Newton-Cotes formulas• Recognize that the trapezoidal and

Simpson’s 1/3 and 3/8 rules represent the areas of 1st, 2nd, and 3rd order polynomials

• Be able to choose the “best” among these formulas for any particular problem

Specific Study Objectives

• Recognize the difference between open and closed integration formulas

• Understand the theoretical basis of Richardson extrapolation and how it is applied in the Romberg integration algorithm and for numerical differentiation

Specific Study Objectives• Recognize why both Romberg

integration and Gauss quadrature have utility when integrating equations (as opposed to tabular or discrete data).

• Understand basic finite difference approximations

• Understand the application of high-accuracy numerical-differentiation.

• Recognize data error on the processes of integration and differentiation.

Numerical Differentiation and Integration• Calculus is the mathematics of change.• Engineers must continuously deal with

systems and processes that change, making calculus an essential tool of our profession.

• At the heart of calculus are the related mathematical concepts of differentiation and integration.

Differentiation

• Dictionary definition of differentiate - “to mark off by differences, distinguish; ..to perceive the difference in or between”

• Mathematical definition of derivative - rate of change of a dependent variable with respect to an independent variable

x

xfxxfxy ii

yx

f x x f xx

i i

f xi

f x xi

y

x

f(x)

x

Integration• The inverse process of differentiation• Dictionary definition of integrate - “to

bring together, as parts, into a whole; to unite; to indicate the total amount”

• Mathematically, it is the total value or summation of f(x)dx over a range of x. In fact the integration symbol is actually a stylized capital S intended to signify the connection between integration and summation.

f(x)

x

I f x dxa

b

Mathematical Backgroundddx

xddx

e

ddx

xddx

a

ddx

xddx

x

ddx

x

if u and v are functions of xddx

uddx

uv

x

x

n

n

sin ? ?

cos ? ?

tan ? ?

ln ?

? ( ) ?

udv

u du

a dx

dxx

e dx

n

bx

ax

?

?

?

?

?

Mathematical Background

Overview• Newton-Cotes Integration Formulas

• Trapezoidal rule• Simpson’s Rules• Unequal Segments• Open Integration

• Integration of Equations• Romberg Integration • Gauss Quadrature• Improper Integrals

Overview

• Numerical Differentiation• High accuracy formulas• Richardson’s extrapolation• Unequal spaced data• Uncertain data

• Applied problems

Newton-Cotes Integration• Common numerical integration scheme• Based on the strategy of replacing a

complicated function or tabulated data with some approximating function that is easy to integrate

I f x dx f x dx

f x a a x a xa

b

na

b

n nn

0 1 ....

Newton-Cotes Integration• Common numerical integration scheme• Based on the strategy of replacing a

complicated function or tabulated data with some approximating function that is easy to integrate

I f x dx f x dx

f x a a x a xa

b

na

b

n nn

0 1 .... fn(x) is an nth orderpolynomial

0

1

2

3

4

5

0 5 10x

f(x)

0

1

2

3

4

5

0 5 10x

f(x)

The approximation of an integral by the area under- a first order polynomial- a second order polynomial

We can also approximated the integral by using a series of polynomials applied piece wise.

0

1

2

3

4

5

0 5 10x

f(x)

0

1

2

3

4

5

0 5 10x

f(x)

The approximation of an integral by the area under- a first order polynomial- a second order polynomial

0

1

2

3

4

5

0 1 2 3 4 5 6 7 8 9 10x

f(x)

An approximation of an integral by the area under straight line segments.

Newton-Cotes Formulas• Closed form - data is at the beginning

and end of the limits of integration• Open form - integration limits extend

beyond the range of data.

0

1

2

3

4

5

0 1 2 3 4 5 6 7 8 9 10x

f(x)

0

1

2

3

4

5

0 1 2 3 4 5 6 7 8 9 10x

f(x)

Trapezoidal Rule

• First of the Newton-Cotes closed integration formulas

• Corresponds to the case where the polynomial is a first order

I f x dx f x dx

f x a a xa

b

a

b

n

1

0 1

I f x dx f x dx

f x a a xa

b

a

b

n

1

0 1

A straight line can be represented as:

f x f af b f a

b ax a1

Trapezoidal Rule

I f x dx f x dx

f af b f a

b ax a dx

a

b

a

b

a

b

1

Integrate this equation. Results in the trapezoidal rule.

I b af a f b

2

Trapezoidal Rule

2

bfafabI

Recall the formula for computing the area of a trapezoid:

height x (average of the bases)

heig

ht

base

base

Trapezoidal Rule

The concept is the same but the trapezoid is on its side.

heig

ht

base

base

widthheig

ht

heig

ht

2

bfafabI

Trapezoidal Rule

Error of the Trapezoidal Rule

E f b a

where a b

t

112

3' '

This indicates that is the function being integrated is linear, the trapezoidal rule will be exact.

Otherwise, for section with second and higher order derivatives (that is with curvature) error can occur.

A reasonable estimate of x is the average value of b and a

Multiple Application of the Trapezoidal Rule• Improve the accuracy by dividing the

integration interval into a number of smaller segments

• Apply the method to each segment• Resulting equations are called multiple-

application or composite integration formulas

I f x dx f x dx f x dx

I hf x f x

hf x f x

hf x f x

x

x

x

x

x

x

n n

n

n

( ) ( ) ( )0

1

1

2

1

0 1 1 2 1

2 2 2

where there are n+1 equally spaced base points.

Multiple Application of the Trapezoidal Rule


I hf x f x

hf x f x

hf x f x

I b af x f x f x

n

x

x

x

x

x

x

n n

i ni

n

n

n

( ) ( ) ( )0

1

1

2

1

0 1 1 2 1

01

1

2 2 2

2

2

We can group terms to express a general form

} }width average height


I b af x f x f x

n

i ni

n

01

1

2

2} }width average height

The average height represents a weighted averageof the function values

Note that the interior points are given twice the weightof the two end points

Eb a

nfa

3

212' '


ExampleEvaluate the following integral using the trapezoidal rule and h = 0.1

I b af x f x f x

n

i ni

n

01

1

2

2

I e dxx2

1

1 6.

hb a

n

0

5

10

15

20

25

30

0 0.3 0.6 0.9 1.2 1.5 1.8x

f(x)

Simpson’s 1/3 Rule

• Corresponds to the case where the function is a second order polynomial

I f x dx f x dx

f x a a x a xa

b

a

b

n

2

0 1 22


• Designate a and b as x0 and x2, and estimate f2(x) as a second order Lagrange polynomial

I f x dx f x dx

x x x xx x x x

f x dx

a

b

a

b

x

x

2

1 2

0 1 0 20

0

2

.......


• After integration and algebraic manipulation, we get the following equations

Ih

f x f x f x

b af x f x f x

34

46

0 1 2

0 1 2} }width average height

Error

E f b a

where a b

t

112

3' '

Single application of Trapezoidal Rule.

Single application of Simpson’s 1/3 Rule E f b at

12880

4 5( )

Multiple Application of Simpson’s 1/3 Rule


I b af x f x f x f x

n

Eb a

nf

x

x

x

x

x

x

i j nj

n

i

n

a

n

n

( ) ( ) ( )

, , .., , ..

0

1

1

2

1

02 4 6

2

1 3 5

1

5

44

4 2

3

180

I b af x f x f x f x

n

i j nj

n

i

n

02 4 6

2

1 3 5

1

4 2

3, , .., , ..

The odd points represent the middle term for each application. Hence carry the weight 4.The even points are common to adjacent applications and are counted twice.

f(x)

x

i=1 (odd) weight of 4

i=2 (even) weight of 2


• Corresponds to the case where the function is a third order polynomial

3210

33

2210

3

338

3 xfxfxfxfhI

xaxaxaaxf

dxxfdxxfI

n

b

a

b

a

Integration of Unequal Segments

• Experimental and field study data is often unevenly spaced

• In previous equations we grouped the term (i.e. hi) which represented segment width.

I b af x f x f x

n

I hf x f x

hf x f x

hf x f x

i ni

n

n n

01

1

0 1 1 2 1

2

2

2 2 2

Integration of Unequal Segments• We should also consider alternately

using higher order equations if we can find data in consecutively even segments

trapezoidalrule


trapezoidalrule

1/3rule

• We should also consider alternately using higher order equations if we can find data in consecutively even segments


trapezoidalrule

1/3rule

3/8rule



trapezoidalrule

1/3rule

3/8rule

trapezoidalrule


ExampleIntegrate the following using the trapezoidal rule, Simpson’s 1/3 Rule, a multiple application of the trapezoidal rule with n=2 and Simpson’s 3/8 Rule. Compare results with the analytical solution.

xe dxx2

0

4

0

2000

4000

6000

8000

10000

12000

14000

0 1 2 3 4x

f(x)

Integration of Equations

• Integration of analytical as opposed to tabular functions

• Romberg Integration • Richardson’s Extrapolation• Romberg Integration Algorithm

• Gauss Quadrature• Improper Integrals

Richardson’s Extrapolation• Use two estimates of an integral to compute a

third more accurate approximation• The estimate and error associated with a multiple

application trapezoidal rule can be represented generally as:• I = I(h) + E(h)• where I is the exact value of the integral• I(h) is the approximation from an n-segment

application• E(h) is the truncation error• h is the step size (b-a)/n

Make two separate estimates using step sizesof h1 and h2 .

I(h1) + E(h1) = I(h2) + E(h2)

Recall the error of the multiple-application of the trapezoidalrule

Eb a

h f12

2 ' '

Assume that is constant regardless of the step sizef ' '

E hE h

hh

1

2

12

22

E hE h

hh

E h E hhh

1

2

12

22

1 21

2

2

Substitute into previous equation:

I(h1) + E(h1) = I(h2) + E(h2)

E h

I h I h

hh

21 2

1

2

2

1

Thus we have developed an estimate of the truncationerror in terms of the integral estimates and their stepsizes. This estimate can then be substituted into:I = I(h2) + E(h2)

to yield an improved estimate of the integral:

I I hh

h

I h I h

21

2

2 2 11

1

E h

I h I h

hh

21 2

1

2

2

1

I I hh

h

I h I h

21

2

2 2 11

1

What is the equation for the special case where the interval is halved?

i.e. h2 = h1 / 2

hh

hh

I I h I h I h

collecting terms

I I h I h

1

2

2

2

2 2 2 1

2 1

2 2

12 1

43

13

Example

Use Richardson’s extrapolation to evaluate:

xe dxx2

0

4

0

2000

4000

6000

8000

10000

12000

14000

0 1 2 3 4x

f(x)

I I h I h

I I h I h

I I h I h

II I

m l

m l

j k

kj k j k

k

43

13

1615

115

6463

163

44 1

2 1

11 1 1

1,, ,

We can continue to improve the estimate by successive halving of the step size to yield a general formula:

k = 2; j = 1

Romberg Integration

Note:the subscriptsm and l refer tomore and lessaccurate estimates

Gauss Quadrature

f(x)

x

f(x)

x

Extend the areaunder the straightline

Method of Undetermined CoefficientsRecall the trapezoidal rule

I b af a f b

2

This can also be expressed as

I c f a c f b 0 1

where the c’s are constant

Before analyzingthis method,answer this question.What are twofunctions thatshould be evaluated exactlyby the trapezoidalrule?

The two cases that should be evaluated exactlyby the trapezoidal rule: 1) y = constant 2) a straight line

f(x)

x

y = 1

(b-a)/2-(b-a)/2

f(x)

x

y = x

(b-a)/2

-(b-a)/2

Thus, the following equalities should hold.

I c f a c f b

c c dx

cb a

cb a

xdx

b a

b a

b a

b a

0 1

0 12

2

0 12

2

1

2 2

FOR y=1since f(a) = f(b) =1

FOR y =xsince f(a) = x =-(b-a)/2andf(b) = x =(b-a)/2

Evaluating both integrals

c c b a

cb a

cb

0 1

0 121

20

For y = 1

For y = x

Now we have two equations and two unknowns, c0 and c1.

Solving simultaneously, we get :

c0 = c1 = (b-a)/2

Substitute this back into: I c f a c f b 0 1

I b a

f a f b

2

We get the equivalent of the trapezoidal rule.

DERIVATION OF THE TWO-POINT GAUSS-LEGENDRE FORMULA

I c f x c f x 0 0 1 1

Lets raise the level of sophistication by:- considering two points between -1 and 1- i.e. “open integration”

f(x)

x-1 x0 x1 1

Previously ,we assumed that the equation fit the integrals of a constant and linear function.

Extend the reasoning by assuming that it also fits the integral of a parabolic and a cubic function.

c f x c f x dx

c f x c f x xdx

c f x c f x x dx

c f x c f x x dx

0 0 1 11

1

0 0 1 11

1

0 0 1 12

1

1

0 0 1 13

1

1

1 2

0

2 3

0

/

We now have fourequations and fourunknowns

c0 c1 x0 and x1

What equations areyou solving?

c f x c f x dx c c

c f x c f x xdx c x c x

c f x c f x x dx c x c x

c f x c f x x dx c x c x

0 0 1 11

1

0 1

0 0 1 11

1

0 0 1 1

0 0 1 12

1

1

0 02

1 12

0 0 1 13

1

1

0 03

1 13

1 2 1 1 2

0 0

2 3 2 3

0 0

`

/ /

Solve these equations simultaneously

f(xi) is either 1, xi, xi2 or xi

3

c c

x

x

I f f

0 1

0

1

113

13

13

13

This results in the following

The interesting result is that the integral can be estimated by the simple addition of the function values at 1

313

and

A simple change in variables can be use to translate other limits.

Assume that the new variable xd is related to theoriginal variable x in a linear fashion.

x = a0 + a1xd

Let the lower limit x = a correspond to xd = -1 and the upper limit x=b correspond to xd=1

a = a0 + a1(-1) b = a0 + a1(1)

What if we aren’t integrating from –1 to 1?

a = a0 + a1(-1) b = a0 + a1(1)

SOLVE THESE EQUATIONSSIMULTANEOUSLY

ab a

ab a

0 12 2

x a a xb a b a x

dd

0 1 2

substitute

x a a xb a b a x

dx b a dx

dd

d

0 1 2

2

These equations are substituted for x and dx respectively.

Let’s do an example to appreciate the theorybehind this numerical method.

ExampleEstimate the following using two-point Gauss Legendre:

xe dxx2

0

4

0

2000

4000

6000

8000

10000

12000

14000

0 1 2 3 4x

f(x)

Higher-Point Formulas

I c f x c f x c f xn n 0 0 1 1 1 1

For two point, we determined that c0 = c1= 1

For three point:

c0 = 0.556 x0=-0.775c1= 0.889 x1=0.0c2= 0.556 x2=0.775

Higher-Point Formulas

Your text goes on to provide additional weightingfactors (ci’s) and function arguments (xi’s)in Table 8.5 p. 593

Numerical Differentiation• Forward finite divided difference• Backward finite divided difference• Center finite divided difference• All based on the Taylor Series

.........!2

''' 21 hxfhxfxfxf i

iii

Forward Finite Difference

21

1

21

2'''

'

.........!2

'''

hOhxfh

xfxfxf

hOh

xfxfxf

hxfhxfxfxf

iiii

iii

iiii

Forward Divided Difference

f xf x f x

x xO x x

fh

O hii i

i ii i

i'

1

11

f(x)

x

(xi, yi)What is derivative at thispoint?


f xf x f x

x xO x x

fh

O hii i

i ii i

i'

1

11

f(x)

x

(xi, yi)

(x i+1,y i+1)

Determine a second pointbase on Dx (h)


f xf x f x

x xO x x

fh

O hii i

i ii i

i'

1

11

f(x)

x

(xi, yi)

(x i+1,y i+1)estim

ateHow doesthis compareto the actualfirst derivativeat xi?


f xf x f x

x xO x x

fh

O hii i

i ii i

i'

1

11

f(x)

x

(xi, yi)

(x i+1,y i+1)estim

ateactua

l

f x fh

O hii'

first forward divided difference

Error is proportional tothe step size

O(h2) error is proportional to the square of the step size

O(h3) error is proportional to the cube of the step size


f(x)

x

estimate

actual

(xi,yi)

(xi-1,yi-1)

f x f x f x hf x

h

f x f x f x hf x

h

f xf x f x

hf

h

i i ii

i i ii

ii i i

12

12

1

2

2

'' '

!......

'' '

!.....

'

Backward Difference Approximation of theFirst Derivative

Expand the Taylor series backwards

The error is still O(h)

Centered Difference Approximation of theFirst Derivative

Subtract backward difference approximationfrom forward Taylor series expansion

211

211

1

1

21

2'

6'''

'2

'

....!2

'''

hOh

xfxfxf

hxf

hxfxfxf

xxxfxf

xf

hxf

hxfxfxf

iii

iiii

ii

iii

iiii

f(x)

x

estimate

actual

(xi,yi)

(xi-1,yi-1)

(xi+1,yi+1)

211

2' hO

hxfxfxf ii

i

f(x)

xf(x)

x

f(x)

xf(x)

x

true derivative forwardfinite divideddifference approx.

backwardfinite divideddifference approx.

centeredfinite divideddifference approx.

Numerical Differentiation

• You should be familiar with the following Tables in your text• Table 7.1: Common Finite Difference

Formulas• Table 7.2: Higher order finite

difference formulas

Richardson Extrapolation

• Two ways to improve derivative estimates¤ decrease step size¤ use a higher order formula that employs

more points• Third approach, based on Richardson

extrapolation, uses two derivatives estimates to compute a third, more accurate approximation

Richardson Extrapolation

I I hh

h

I h I h

Special case where h h

I I h I h

In a similar fashion

D D h D h

21

2

2 2 1

21

2 1

2 1

1

1

243

13

43

13

For a centered differenceapproximation withO(h2) the application ofthis formula will yielda new derivative estimateof O(h4)

ExampleGiven the following function, use Richardson’s extrapolation to determine the derivative at 0.5.

f(x) = -0.1x4 - 0.15x3 - 0.5x2 - 0.25x +1.2

Note:

f(0) = 1.2f(0.25) =1.1035f(0.75) = 0.636f(1) = 0.2

Derivatives of Unequally Spaced Data

• Common in data from experiments or field studies• Fit a second order Lagrange interpolating

polynomial to each set of three adjacent points, since this polynomial does not require that the points be equi-spaced

• Differentiate analytically

f x f x x x xx x x x

f x x x xx x x x

f x x x xx x x x

ii i

i i i ii

i i

i i i i

ii i

i i i i

'

11

1 1 1

1 1

1 1

11

1 1 1

2 2

2

Derivative and Integral Estimates for Data with Errors• In addition to unequal spacing, the other problem

related to differentiating empirical data is measurement error

• Differentiation amplifies error• Integration tends to be more forgiving• Primary approach for determining derivatives of

imprecise data is to use least squares regression to fit a smooth, differentiable function to the data

• In absence of other information, a lower order polynomial regression is a good first choice

050

100150200250

0 5 10 15

t

y

0

10

20

30

0 10

t

dy/dt

050

100150200250

0 5 10 15

t

y

010203040

0 10

tdy/dt

Specific Study Objectives• Understand the derivation of the

Newton-Cotes formulas• Recognize that the trapezoidal and

Simpson’s 1/3 and 3/8 rules represent the areas of 1st, 2nd, and 3rd order polynomials

• Be able to choose the “best” among these formulas for any particular problem

Specific Study Objectives• Recognize the difference between

open and closed integration formulas

• Understand the theoretical basis of Richardson extrapolation and how it is applied in the Romberg integration algorithm and for numerical differentiation

Specific Study Objectives• Recognize why both Romberg

integration and Gauss quadrature have utility when integrating equations (as opposed to tabular or discrete data).

• Understand the application of high-accuracy numerical-differentiation.

• Recognize data error on the processes of integration and differentiation.

…..end of lecture

1519 differentiation-integration-02

Education