cs537 numerical analysis - computer science …jzhang/cs537/lecture3.pdf · cs537 numerical...

CS537

Numerical Analysis

Lecture 3

Numerical Integration

Professor Jun Zhang

Department of Computer ScienceUniversity of Kentucky

Lexington, KY 40506‐0633

February 11, 2015

Definite Integral

2

A definite integral has an interval for integration. For a fixed integration interval, the result is a number

An indefinite integral does not have an integration interval. The result of an indefinite integral (antiderivative) is a class of functions

Numerical integration is for computing definite integrals

Fundamental Theorem of Calculus:

20

1sin

dxx

Cxdxx cossin

x

a

ba

aFxFdttF

aFbFdxxF

)()()('

)()()('

Numerical Integration

Partition

4

The definite integral of a function can be viewed as the area under a curve. This point of view lends us means to compute definite integral

Let P be a partition of the interval of [a,b] as

We have n subintervals as [xi,xi+1]. Let mi be the greatest lower bound of (a nonnegative function) f(x) on [xi,xi+1] as

And Mi as the least upper bound on the same subinterval

bxxxxxaP nn 1210

1:)(inf iii xxxxfm

1:)(sup iii xxxxfM

Partition

Lower and Upper Sums

6

The lower sums and upper sums of f corresponding to the given partition P are

If we consider the definite integral of a nonnegative f as the area under the curve, we have

for all partitions P

If f is continuous on [a,b], then the above inequality defines the definite integral. The integral also exists if f is monotone (either increasing or decreasing) on [a,b]

1

01

1

01

);(

);(

n

iiii

n

iiii

xxMPfU

xxmPfL

ba

PfUdxxfPfL );()();(

Upper and Lower Bounds

Riemann‐Integrable Functions

8

If the greatest lower bound equals the least upper bound for all partitions of [a,b], i.e.,

Then f is said to be Riemann‐integrable

Every continuous function defined on a closed and bounded interval of the real line is Riemann‐integrable

We have

where P0, P1,.., Pn,… are a sequence of partitions such that the length of the largest subinterval in Pn converges to 0 as n → ∞

We can construct nested (refined) partitions

),(sup);(inf PfLPfUPP

);(lim)();(lim nn

bann

PfUdxxfPfL

Computation

9

1.) need a procedure to evaluate f(x)

2.) determine a partition (how many subintervals) of the interval [a,b]

3.) compute mi and Mi on each subinterval

4.) compute the sums L(f; P) and U(f; P)

5.) an approximate value is obtained

6.) the error of this approximation is bounded by

ba

PfLPfUdxxf ;;)( 21

PfLPfU ;;21

Trapezoid Rule

10

A strategy that is better than estimating both the upper and the lower bounds of the area beneath the curve is to use trapezoids

The interval [a,b] is first divided into subintervals [xi,xi+1], 0 ≤ i ≤ n – 1. A typical trapezoid has the subinterval [xi,xi+1] as its base, and the two vertical sides are f(xi) and f(xi+1). The area is given by the base times the average height. The basic trapezoid rule for the subinterval [xi,xi+1] is

The total area under the curve is

)()()()( 11211

i

i

i

xxxfxfdxxf i

x

x ii

ba

n

iiAPfTdxxf

1

0);()(

1

011 )]()()[(

21 n

iiiii xfxfxx

Trapezoid Rule

Uniform Spacing

12

The lengths of the subintervals in a partition can be different. For fast computation, a uniform partition of the interval may be advantageous

Let n be the number of subintervals, then h = (b – a)/n is the uniform interval spacing. The nodal points are xi = a + i h, i = 0, 1,…, n. Hence the composite trapezoid rule is

Note that the end point of an interval is the starting point of the next interval. This fact can save almost half of the computation,

ba

n

ini xfxfxfhdxxf

1

10 )]()([

21)()(

1

01 )]()([

2);(

n

iii xfxfhPfT

Trapezoid Rule with Uniform Spacing

Error Analysis (1)

14

If f”(x) exists and is continuous on [a,b], the error of the composite trapezoid rule T is

for some ξ in (a,b)

Proof. We first prove the result for a=0, b= 1 and h=1. That is

This simplified formula will be proved with the help of polynomial interpolation

Define a polynomial of degree one that interpolates f at 0 and 1

ba

ab hOfhTdxxf )()(")( 2212

xfffxp )]0()1([)0()(

)(")1()0()( 121

21

1

0fffdxxf

Error Analysis (2)

15

It follows that

Using the error formula for the polynomial interpolation, we have

Integrate it on both sides,

)]1()0([21

)]0()1([21)0()(

1

0

ff

fffdxxp

1

0

1

0

1

0)1()](["

21)()( dxxxxfdxxpdxxf

)1()]([")()( 21 xxxfxpxf

Error Analysis (3)

16

Using the Mean‐Value Theorem for Integrals

So we have

We then do a change of variable, and let

)("61

)1()]([")1()](["1

0

1

0

f

dxxxsfdxxxxf

2))](([")("))](([')(' ,)(

)( )],([)(

ababtaftgababtaftgdtabdx

tabaxabtaftg

)("121)]1()0([

21)(

1

0fffdxxf

Error Analysis (4)

17

Then using the result for the special case,

This is the error formula for the trapezoid rule with only one subinterval.

Let [a,b] be divided into n equal subintervals by points

)("12

)()]()([2

)("121)]1()0([

21)(

)()(

)]([)()(

3

1

0

1

0

fabbfafab

gggab

dttgab

dtabtafabdxxfb

a

],[ lsubinterva with ,,...,, 110 iin xxxxx

Error Analysis (5)

18

Let h be the interval length

Sum over all subintervals to get the composite trapezoid rule

Note that h=(b-a)/n, we use Intermediate-Value Theorem of Continuous Functions,

1 )("

121)]()([

2)( 3

1i

i

x

x ii fhxfxfhdxxf

)("12

)("112

)("12

21

0

1

0

23

fhabfn

habfh n

i

n

iii

1 )("12

)]()([2

)()(1

0

3

1

1

0

1

0

i

i

x

x i

n

iii

n

i

b

a

n

ifhxfxfhxxfdxxf

Example 1

19

Show that

Need to define

We can expand F(a+h) using Taylor series

Then by the Fundamental Theorem of Calculus, we have F’(t) = f(t).Note that F(a) = 0, F”(t)=f’(t), F”’(t)=f”(t), and so on.

)('"!3

)("2

)(')()(32

aFhaFhahFaFhaF

ta

dxxftF )()(

)(")()(

)(

1223

afhafaf

dxxf

hh

haa

Example 2

20

We have

(1)We can also apply the Taylor series directly on f(t) as

(2)

Adding f(a) on both sides of (2) and multiplying it by h/2, we obtain

(3)

Subtracting (3) from (1), we finally get

)("4

)('2

)()]()([2

32

afhafhahfhafafh

)('"!3

)("2

)(')()(32

afhafhahfafhaf

)("!3

)('2

)()(32

afhafhahfdxxfha

a

)("121)]()([

2)( 3 afhhafafhdxxf

ha

a

Estimate Grid Spacing

21

Example. If the composite trapezoid rule is used to compute

with an error of at most 0.5 × 10-4, what is the uniform grid spacing h?

From the graph of the second derivative (a decreasing function)

We find that

We need

It follows that h ≤ 0.01732. The number of subintervals is n ≥ [1/h] = 58

dxe x 10

2

2)24()(" 2 xexxf

2)0(")(" fxf

42612 105.0)("

12

hfhab

Recursive Trapezoid Idea

22

What can we do if the initial partition of interval is not fine enough?

Recursive Trapezoid Formula

23

Given a parameter n, dividing [a,b] into 2n equally spaced subintervals, we have

Note that n = 2n and h = (b – a)/2n

Notice that R(n,0) can be viewed as dividing each subinterval of R(n – 1,0) into two equal sub‐subintervals. If we already computed R(n – 1,0), how can we compute R(n,0) cheaply?

1

1

1

10

)()(2

)(

)()(2

)();(

n

i

n

ini

bfafhhiafh

xfxfhxfhPfT

12

1)()(

2)()0,(

n

ibfafhhiafhnR

24

Recursive Formula

If R(n – 1,0) is available, R(n,0) can be computed as

For n ≥ 1 using h = (b – a)/2n. Initial starting value is

The trick is to only sum the function values at every other grid points

Proof. Note that

With C = h[f(a) + f(b)]/2 and

12

1

1

1201210

n

k

]h)k(a[fh),n(R),n(R

)]b(f)a(f)[ab(),(R 2100

12

1)()0,(

n

iChiafhnR

12

1

1

2)2(2)0,1(n

jChjafhnR

25

Recursive Formula

Hence, we have

Each term in the first sum that corresponds to an even‐indexed value of Iis cancelled by a term in the second term. The final result has only the odd‐indexed values of I

We can use the recursive trapezoid formula to compute a sequence of approximations to a definite integral using the trapezoid rule, without recomputing the values at points that have already been computed in the previous step

1

1

2

1

12

1

12

1

])12([

)2()()0,1(21)0,(

n

n n

k

i j

hkafh

jhafhihafhnRnR

26

Two Dimensional Integration

For one dimensional numerical integration on [0,1], using uniform space h = 1/n

For two dimensional integration on a unit square

n

ii

n

i

nifA

fniff

ndxxf

0

1

1

1

0)]1(2)0([

21)(

n

i

n

jji

n

jj

n

ii

n

ii

n

ii

nj

nifAA

nj

nifAA

dyynifA

dyynifAdydxyxf

0 0

00

0

10

10

1

10

10

,

,

,

,),(

27

Romberg Algorithm

Recursive composite trapezoid method

For h = 1/2n and n ≥ 1

Using Richardson extrapolation, we can have

For i ≥ j and j ≥ 1. This is the Romberg algorithm, which may yield better approximate values for larger j

)]()()[()0,0( 21 bfafabR

12

1

1

1201210

n

k]h)k(a[fh),n(R),n(R

)]1,1()1,([)1,(),(

141

jiRjiR

jiRjiR

j

28

Deriving Romberg Algorithm

Composite trapezoid rule on 2n-1 subintervals

With h = (b – a)/2n-1 and the coefficients ai depend on f but not on h

After one refinement and replacing n – 1 with n and h with h/2, we have

Subtracting the 1st equation from 4 times the 2nd equation

where for n ≥ 1

66

44

22)0,1()( hahahanRdxxfb

a

6

66414

41612

241)0,(

)(

hahahanR

dxxfba

6616

5444

11 haha),n(Rdx)x(fb

a

)]0,1()0,([)0,()1,( 31 nRnRnRnR

29

More Romberg Algorithm

We could apply the extrapolation idea repeatedly to get

Where

This time, the truncation error is of sixth order

A few steps of extrapolation may generate very accurate approximations

Too many extrapolations may make the computation tedious

884

21664

153)2,()( hahanRdxxf

b

a

)]1,1()1,([)1,()2,( 151 nRnRnRnR

30

General Extrapolation

Extrapolation processes can be applied in more general cases where the error term can be represented as

With 0 < α < β < γ, we show how the first term of the error expansion is annihilated. Let

Replacing h by h/2 yields

Multiplying (2) by 2α

hchbhaE

)1()( hchbhahL

)2(2222 hhhh cbaL

)3(2222 222 hhh cbhaL

31

General Extrapolation – Cont.

Subtracting the previous two equations, we can remove the hα term

We can write the new approximation formula as

This approximation formula raises the order of truncation error from O(hα) to O(hβ) with α < β

Please read the book on p. 217 for a concrete example to show how the approximation accuracy is improved using extrapolation

hchb

hhL

)12()12(

)(2

2)12(

chbhhhL )(

121

2122

32

Basic Simpson’s Rule

Simple trapezoid rule uses two points for approximations. Composite trapezoid rule uses more points.Can we get more accurate approximation with different weights?

33


A three point numerical integration rule using the middle point of the interval is known as the Simpson’s rule with different weights for each point

Using Taylor’s expansion, we can find the error term of this approximation is

For some point ξ in (a,a+2h). This should be compared to the error term of the simple trapezoid rule O(h3)

It is desirable to subdivide the interval adaptively so that refinement is only placed in the area of large fluctuation of function value

ha

ah hafhafafdxxf23 )2()(4)()(

)(90

)4(5

fh

34


Using Taylor series for f(x) at a, we have

If we replace the interval size h by 2h, we obtain

By combining these two expansions, we get

)(!4

1)('"!3

1)("!2

1)(')()( )4(432 afhafhafhahfafhaf

)(!4

2)('"34)("2)('2)()2( )4(4

432 afhaFhafhahfafhaf

)(!4

20)('"2)("4)('6)(6)2()(4)( )4(432 afhafhafhahfafhafhafaf

35


On the other hand, define

We expand F(a+2h) as

Note that F’=f, F(a)=0, F”=f’, F”’=f”, we have

From the previous page, we have

x

adttfxF )()(

)(!5

2)(32)('"

34)("2)('2)()2( )5(5

5)4(432 aFhafhaFhaFhahFaFhaF

)(!45

2)('"32)("

34)('2)(2)( )4(5

54322

afhafhafhafhahfdxxfha

a

)(!43

20)('"32

)("34)('2)(2)]2()(4)([

3)4(54

32

afhafh

afhafhahfhafhafafh

36

Basic Simpson’s Rule (1)

Subtracting the previous two equations, we have

We have the Simpson’s rule as

The error term of the Simpson’s rule is

ha

afhhafhafafhdxxf

2 )4(5

90)]2()(4)([

3)(

)(290

1 )4(5

fab

ha

abfbafafabdxxf

2)](

24)([

6)(

37

Adaptive Simpson’s Algorithm

Reduce the size of the intervals to get more accurate approximations

38


Given an interval [a,b], we can use the basic Simpson’s rule to compute an approximation to the integral as

where the approximation part is

and the error term is

For simplicity, we assume f(4)(x) remains constant on (a,b). Let h = b – a, we have

For the first step approximation with

ba

baEbaSdxxfI ),(),()(

)(2

4)(6

)(),( bfbafafabbaS

)(),( )4(5290

1 fbaE ab

)1()1( ESI

),()1( baSS

39

Adaptive Simpson – (II)

And

We then subdivide the interval [a,b] and apply the basic Simpson’s rule on the subintervals [a,c] and [c,b] respectively. We have a new approximation on [a,b] as the sum of two separate approximations

where c = (a + b)/2 with

and

This is certainly a better approximation since the subintervals are smaller than the original interval

)4(5

)1(

2901 fhE

)2()2( ESI

),(),()2( bcScaSS

)1()4(5

)4(5

)2(

161

22/

901

22/

901 EfhfhE

Adaptive Simpson – (III)

40

Subtracting the two approximations yields

Hence the numerical integration can be

The error term is then computable and can be used for building the adaptive process

If this test shows that the error is larger than ε, the interval [a,b] can be split into two subintervals [a,c] and [c,b] with c = (a + b)/2. The previously described procedure is replaced by ε/2 to make sure that the error sum is smaller than ε

)2()2()1()1()2( 15EEESS

)( )1()2(151)2()2()2( SSSESI

)1()2(

151 SS

41


Refine the intervals at the places where the function changes quickly

42

Adaptive Simpson – (IV)

Numerical integration on subintervals

If we want to have

It is more than enough to have

and

RL

b

c

c

a

b

aIIdx)x(fdx)x(fdx)x(fI

have we , on and on of sum the be Let ][][ )2()2( c,bSa,cSS RL

)1()2(151)1()2(

151

)2()2(

)2()2(

RRLL

RRLL

RLRL

SSSS

SISI

SSIISI

SI

2)1()2(

151 LL SS

2)1()2(

151 RR SS

43


One decision to make is to choose where to refine the interval

44

Computational Procedure

The interval [a,b] is divided into four subintervals of equal length. Two Simpson approximations are computed using two double‐width subintervals and four single‐width subintervals

If |S2-S1| ≤15ε, we have done, and set

Otherwise the interval [a,b] is divided in half and the recursive procedure is applied on the two subintervals [a,c] and [c,b], until either the error tolerance is satisfied or the maximum number of subdivisions is reached

)(2

4)(22

4)(12

)(2

4)(6

2

1

bfbcfcfcafafhS

bfbafafhS

1216151 SSS

45


Which sub‐interval or both to divide for refinement

46

Gaussian Quadrature Formulas

A general numerical integration formula is

It suffices to know the nodes x0, x1,…, xn and the weights A0, A1,…, An. For important special functions, they are listed in some reference books

Suppose a set of nodes are given, how to find the weights. This can be done using Lagrange interpolation polynomial as

With

If p is a good approximate to f, we anticipate is a good approximate to

n

iii xlxfxp

0)()()(

n

joj ji

ii xx

xxxl

1,)(

)()()()( 1100 nnba

xfAxfAxfAdxxf

ba

dxxf )(

ba

dxxp )(

47

Gaussian Quadrature

We integrate over p(x) as

where we can compute

Note that the polynomial interpolation is exact for a polynomial of degree at most n. It follows that the integration will be exact for such polynomials

If the nodes can be chosen carefully, it is possible to increase the order of polynomial with the exact integration remarkably. This was discovered by Karl Gauss

ba ii dxxlA )(

ba

ba

dxxpdxxf )()(

n

iii

ba i

n

ii xfAdxxlxf

10)()()(

48

An Example

Determine a quadrature formula when the interval is [‐2,2] and the nodes are ‐1, 0, and 1.We first need to compute the cardinal functions

The weights are computed by integrating these functions

)1(21

111

0)1(0)(

2

1 00

xxxxxxxx

xlj j

j

38

21

31

21)1(

21)(

2

2

2

2

232

2 00

xxdxxxdxxlA

22

1 11 1

101

)1(01)( xxx

xxxx

xlj j

j

)1(21

010

)1(1)1()(

2

1 22

xxxxxxxx

xlj j

j

49

An Example

Similarly, we have

So the quadrature formula defined on the interval [‐2,2] and using the node ‐1, 0, 1, is

It can be verified that this formula gives exact values for the three functions

38

21

31

21)1(

21)(

2

2

2

2

232

2 22

xxdxxxdxxlA

)1(38)0(

34)1(

38)(

2

2fffdxxf

34

31)1()(

2

2

2

2

322

2 11

xxdxxdxxlA

2 , ,1)( xxxf

50

Gaussian Quadrature Theorem

Let q be a nontrivial polynomial of degree n + 1 such that

Let x0, x1,…, xn be zeroes of q. Then we define the formula

With these xi’s as nodes, the approximation will be exact for all polynomials of degree at most 2n+1. All these nodes lie in the open interval (a,b)

We can first figure out the quadrature for

Then use the transformation t = [2x – (b – a)]/(b – a) for a Gaussian quadrature on the general interval [a,b]

)0(0)( nkdxxqxba

k

n

i

ba iiii

ba

dxxlAxfAdxxf0

)(),()(

n

iii tfAdttf

0

11

)()(

51

Gaussian Theorem Proof

The transformed integral is

Proof of Gaussian Quadrature Theorem:Let f be any polynomial of degree at most (2n + 1). Dividing f by q with quotient p and a remainder r

Both p and r are of degree at most n

By hypothesis, we have

Since xi are roots of q, we have

rqpf

0)()( ba

dxxpxq

)()()()()( iiiii xrxrxqxpxf

ba

dxxf )( dtabtabfab

11 2

121

21 )()()(

52

Gaussian Theorem Proof (II)

Since the degree of r is at most n, the integration is exact

Gaussian Quadrature Theorem guarantees high accuracy numerical integration with just a few nodes. However, finding these nodes is not an easy task. The roots of Legendre polynomials are the nodes for Gaussian quadrature on the interval [-1,1]. With q0(x) = 1, q1(x) = x, we have forn ≥ 2

ba

dxxr )(

n

i

n

iiiii

ba

ba

ba

ba

xfAxrAdxxr

dxxrdxxqxpdxxf

0 0)()()(

)()()()(

)(1)(12)( 21 xqn

nxxqn

nxq nnn

cs537 numerical analysis - computer science …jzhang/cs537/lecture3.pdf · cs537 numerical...

Documents