Numerical Integration

(Quadrature)

Another application for our

interpolation tools!

Lecture 11 2

Integration: Area under a curve

Curve = data or function

Integrating data • Finite number of data points—spacing

specified

• Data may be noisy

• Must interpolate or regress a smooth between points

Integrating functions • Non-integrable function (usually)

• As many points as you need

• Spacing arbitrary

Today we will discuss methods that work on both of these problems

( )b

af x dxArea =

Area

Area

x

y

x

y

a b

a b

Lecture 11 3

Numerical Integration: The Big

Picture

Virtually all numerical integration methods rely on the following procedure: • Start from N+1 data points (xi ,fi), i = 0,…,N, or sample a specified

function f(x) at N+1 xi values to generate the data set

• Fit the data set to a polynomial, either locally (piecewise) or globally

• Analytically integrate the polynomial to deduce an integration formula of the general form:

Numerical integration schemes are further categorized as either:

• Closed – the xi data points include the end points a and b of the interval

• Open – the xi data points are interior to the interval

wi are the “weights”

xi are the “abscissas”, “points”, or

“nodes”

Lecture 11 4

Further classification of

numerical integration schemes

Newton-Cotes Formulas • Use equally spaced abscissas

• Fit data to local order N polynomial approximants

• Examples: • Trapezoidal rule, N=1

• Simpson’s 1/3 rule, N=2

• Errors are algebraic in the spacing h between points

Clenshaw-Curtis Quadrature • Uses the Chebyshev abscissas

• Fit data to global order N polynomial approximants

• Errors can be spectral, ~exp(-N) ~ exp (-1/h), for smooth functions

Gaussian Quadrature • Unequally spaced abscissas determined optimally

• Fit data to global order N polynomial approximants

• Errors can be spectral, and smaller than Clenshaw-Curtis

Lecture 11 5

Trapezoid rule: 1 interval, 2

points

( )b

af x dxArea =

x

f (x)

a b x

f (x)

a b

1

( ) ( ) ( )2

f a f b b a Area =

average height

1

( ) ( )2

f a f b

( )f b

( )f a

Approximate the function by a linear interpolant between

the two end points, then integrate that degree-1 polynomial

Lecture 11 6

Trapezoid rule: 2 intervals, 3

points

( )b

af x dxArea =

x

f (x)

x0 x2 x

f (x)

a b

0 1 1 2

0 1 2

( ) ( ) ( ) ( )2 2

( ) 2 ( ) ( )2

h hf x f x f x f x

hf x f x f x

Area =

x1

We should be able to improve accuracy by increasing the number

of intervals: this leads to “composite” integration formulas

Lecture 11 7

“Composite” Trapezoid rule: 8 →

n intervals, n +1 points

x

f (x)

x0 x8 x

f (x)

a b

0 1 1 2 1 2 7 8

0 1 2 7 8

7 1

0 8 0

1 1

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )2 2 2 2

( ) 2 ( ) 2 ( ) 2 ( ) ( )2

( ) 2 ( ) ( ) ( ) 2 ( ) ( ) ;2 2

n

i i nni i

h h h hf x f x f x f x f x f x f x f x

hf x f x f x f x f x

h h b af x f x f x f x f x f x h

n

Area =

x4 x1 x2 x3 x5 x6 x7

N=1 linear interpolation within each interval:

more intervals to improve accuracy

Lecture 11 8

Composite Trapezoidal Rule

Notice that this composite formula can be written in the generic form:

where the weights are given by

The truncation error of the composite trapezoidal rule is of order

Error per interval times number of intervals

Lecture 11 9

Simpson’s 1/3 rule

x

f (x)

x

f (x)

Basic idea: interpolate between 3 points using a parabola (N=2

polynomial) rather than a straight line (as in trapezoid rule)

x

f (x)

red

parabola

blue

parabola

x

f (x)

trapezoid Simpson

Lecture 11 10

Simpson’s 1/3 rule: derivation

x

f (x)

x0 x2 x1

Interpolation: find equation for a

parabola (N=2 polynomial) passing

through 3 points

1

2

0 0

0

0

( )

x x

x x

f x x x

at

at

at

0

2

1 1

0

0

( )

x x

x x

f x x x

at

at

at

0

1

2 2

0

0

( )

x x

x x

f x x x

at

at

at

( )p x

p(x) is a parabola (quadratic in x)

p(x) goes through 3 desired points

Use Lagrange form for convenience:

Lecture 11 11

Simpson’s 1/3 rule: derivation

for 2 intervals, 3 points

x

f (x)

x0 x2 x1

( )p x

2

0

( )x

xp x dx Area

2

0

(4) 5

0 1 2

1( ) ( ) 4 ( ) ( ) ; ( )

3 90

x

tx

hp x dx f x f x f x E f h Area

2 1 1 0h x x x x

Remember from calculus:

Lecture 11 12

Composite Simpson’s 1/3 rule:

(n intervals, n+1 points, n even)

compare with trapezoidal rule

The formula is thus:

The error is:

Lecture 11 13

Closed Newton-Cotes Formulas

We have just derived the first two closed Newton-Cotes formulas for equally spaced points that include the end points of the interval [a,b]

Basic truncation errors • Trapezoidal (1 interval, 2 points): O(h3)

• Simpson’s 1/3 (2 intervals, 3 points): O(h5)

Composite formula truncation errors (n intervals, n+1 points) • Trapezoidal (“first-order accurate”): O(h2(b-a)) = O(1/n2)

• Simpson’s 1/3 (“third-order accurate”): O(h4(b-a)) = O(1/n4)

Higher order formulas can evidently be developed by using local polynomials with N>2 to interpolate with a larger number of points

It is also easy to adapt these formulas to composite formulas with unequally spaced intervals, h1 h2 h3 … if the data is unevenly spaced

Lecture 11 14

Example: Simpson’s 1/3 Rule

Let’s try out the composite Simpson’s 1/3 rule on the integral:

Exact result is 2[sin(1)-cos(1)] =0.602337… See SimpsonL11.m

n error

4 2.20 x 10-3

8 1.35 x 10-4

16 8.34 x 10-6

Errors

consistent with

~1/n4 scaling

Lecture 11 15

Open Newton-Cotes formulas

Same idea as closed formulas, but end points x0 = a, xn = b are not included

These are useful if

• Data is not supplied at the edges of the interval [a,b]

• The integrand has a singularity at the edges (see later)

A simple and useful example – the composite midpoint rule:

where x1/2 is located at the midpoint

between x0 and x1, etc. Same accuracy as

composite TR

Lecture 11 16

Options for higher accuracy

If we want highly accurate results for an integral, we have various options:

• Use more points n in Simpson’s rule (since error ~ 1/n4) –

computationally costly

• Derive a higher-order Newton-Cotes formula that fits more points to a polynomial with N>2 – analytically cumbersome for small gains

• Use repeated Richardson extrapolation to develop higher order approximations Romberg integration

• Switch to a method based on global approximants:

• Clenshaw-Curtis Quadrature

• Gauss Quadrature

Lecture 11 17

Clenshaw-Curtis Quadrature

Clenshaw-Curtis quadrature is based on integrating a global Chebyshev polynomial interpolant through all N+1 points

The abscissas are fixed at the Chebyshev points:

If we integrate p(x):

Finally, the weights are obtained by substituting the solution of the collocation equations [T] {c} = {f} into this expression for the ci values

The M-file clencurt.m from L.N. Trefethen, “Spectral Methods in MatLab” (SIAM, 2000) encapsulates these weight calculations

Lecture 11 18

Example: Clenshaw-Curtis

See ClenCurtL11.m

Compares the error vs. N for integrals of 4 functions:

Spectral accuracy

for the two smooth

functions:

Error below

machine precision

for N > 10!

c.f. Simpson’s rule error of 10-5 at N=16

Lecture 11 19

Gauss Quadrature:

Global like Clenshaw-Curtis, but also

optimally choose abscissas

-1 0 1 -1 0 1 1

1( )I f x dx

x1 x2

poor

approximation much better approximation

obtained by adjusting x1 & x2

values and w1 and w2 values

2 point

case:

Lecture 11 20

Finding optimal choice of

parameters

-1 0 1 x1 x2

Note that there are 4 unknowns to be

determined.

Gauss procedure: we choose the 4 unknowns so that

the integral gives an exact answer whenever f (x) is a

polynomial of order 3 or less

2 3( ) 1, ( ) , ( ) , ( ) .f x f x x f x x f x x

it would have to give the exact answer for the following four cases:

Lecture 11 21

Determining the unknowns: Gauss-

Legendre Quadrature (N=2 case)

•More generally, for an N point formula, the abscissas are the N roots of the

Legendre Polynomial PN(x). The weights can be obtained by solving a

linear system with a tridiagonal matrix.

•Notice that Gauss-Legendre is an open formula, unlike Clenshaw-Curtis

Lecture 11 22

Example: Gauss-Legendre

See gauss.m (for abscissas and weights) and GaussL11.m

Compares the error vs. N for integrals of same 4 functions:

Spectral accuracy

for the two smooth

functions:

Error below

machine precision

for N > 6!

Faster

convergence than

even Clenshaw-

Curtis for smooth

functions c.f. Simpson’s rule error of 10-5 at N=16

Gauss exactly integrates a

polynomial of degree 2N-1!

Lecture 11 23

Adjusting the limits of

integration

The spectral accuracy of the Gauss-Legendre and Clenshaw-Curtis methods can be traced to the fact that they employ global polynomial interpolation and cluster their abscissas at the edges of the interval

Remember that the Clenshaw-Curtis and Gauss-Legendre formulas apply only for integrals with limits of -1 and +1!

The limits need to be rescaled for other integrals:

Lecture 11 24

Integrals with Singularities

Occasionally you will encounter integrals with weak (integrable) singularities, e.g.

In this case there is an inverse square root singularity at the endpoint x=0. We obviously cannot apply a closed integration formula in this case!

Alternatives are: • Apply an open formula, like composite midpoint rule or Gauss-Legendre

quadrature

• Use an open formula that explicitly includes the x-1/2 factor (better option)

• Rescale x variable as x = t2, so the singularity is removed (best option, if available):

Lecture 11 25

Indefinite (improper) integrals

Often you will encounter integrals with infinite limits, e.g.

There are several alternatives: • Apply a Newton-Cotes formula to a similar integral, but with 1 replaced

with a large number R

• Rescale x variable as x = - ln t, (assuming resulting integral not singular):

A better approach is to use a Gaussian quadrature formula appropriate for the interval [0,1), such as the Gauss-Laguerre formula:

Don’t forget to factor the

xexp(-x) out of your function!