chapter 12-b - bauer college of business · ch 12 - integral calculus 8/8/2019 1 1 chapter 12-b...

Ch 12 - Integral Calculus 8/8/2019

1

1

Chapter 12-bIntegral Calculus - Extra

Thomas SimpsonIsaac Newton

2

BONUS Introduction to Numerical

Integration


2

30

2

4

6

8

10

12

3 5 7 9 11 13 15

Numerical Integration

Idea: Do an integral in small parts, like the way we presented integration; i.e., a summation.

Numerical methods just try to make it faster and more accurate.

4

• Idea: Weighted sum of function values to approximate integral

)x(fc)x(fc)x(fc

)x(fcdx)x(f

nn1100

i

n

0ii

b

a

x0 x1 xnxn-1x

f(x)

Basic Numerical Integration

• Task: Find appropriate ci’s (weights) and the xi’s (nodes).


3

5

• We want to find integration of functions of various forms of the equation known as the Newton-Cotes integration formulas (“rules”).

• Newton-Cotes formula

Assume the value of f(x) defined on [a,b] is known at equally spaced points xi (i=0,1,...,n), where x0 =a, and xn =b. Then,

where xi = h i + x0, with h (“step size”) = (xn − x0)/n = (b −a)/n.

The ci's are called weights. The xi's are called nodes. The precision of the approximation depends on n.

Note: The N-C rules use nodes equally spaced. But, they do not have to be --unequally spaced nodes are OK too.

n

iii xfcf(x)dx

1

b

a

)(


6

• Weights

In the N-C formulae, they are derived from an approximation required to be equal for a polynomial of order lower or equal to the degree of the polynomials used to approximate the function. In other methods, weights and nodes can be derived jointly.

• Error analysis

The error of the approximation is the difference between the value of the integral and the numerical result:

error = ε = ∫ f(x) dx – Σi ci f(xi)

The errors are frequently approximated using Taylor series for f(x).The error analysis gives a strict upper bound on the error, if the derivatives of f are available.



4

7

• The weights are derived from the Lagrange polynomials L(x). The weight, Li(x), depend only on the xi's (no two xi's are the same); not on the function ƒ.

Recall that L(x) is used for polynomial interpolation of a function f(x), given a set of points (xi, f(xi)).

Example: Interpolate f(x) = 2 x3 over [2, 4], with 3 points: (2, 3, 4). The interpolating Lagrange is:

16 ∗ 54 ∗ 128 ∗

18 52 48

Newton-Cotes Formula

))...()()...((

))...()()...((

)(

)()(

110

110

,0 niiiiii

nii

jinj ji

ji xxxxxxxx

xxxxxxxx

xx

xxxL

8

• Different methods use different polynomials to get the ci's.

• Newton-Cotes Closed Formulae – Use both end points

– Trapezoidal Rule : Linear

– Simpson’s 1/3-Rule : Quadratic

– Simpson’s 3/8-Rule : Cubic

– Boole’s Rule : Fourth-order

• Newton-Cotes Open Formulae – Use only interior points

– midpoint rule

Newton-Cotes Formula


5

9

Trapezoid Rule• Straight-line approximation

The trapezoid rule approximates the region under the graph of the function f(x) as a trapezoid and calculating its area.

)x(f)x(f2

h

)x(fc)x(fc)x(fcdx)x(f

10

1100i

1

0ii

b

a

x0 x1 x

f(x)

L(x)

Approximate integral by the trapezoid’s area.

10

Trapezoid Rule - Derivation

• We use the Lagrange approximation for function f(x) over the interval (xn – x0) (Lagrange interpolation), given by

010 1

0 1 1 0

0 1

( ) ( ) ( )

dx a x , b x , , d ;

h

0( ) (1 ) ( ) ( ) ( )

1

x xx xL x f x f x

x x x x

x alet h b a

b a

x aL f a f b

x b

• Then, we integrate to obtain the trapezoid rule

)()(...)()()()()( 1100 xLxfxLxfxLxfxf nn

• For the case, n = 2, with the interval (x1 – x0):


6

11

Trapezoid Rule - Derivation

• Integrating:

1

0

1 1

0 0

1 12 2

0 0

( ) ( ) ( )

( ) (1 ) ( )

( ) ( ) ( ) ( ) ( )2 2 2

b b

a af x dx L x dx h L d

f a h d f b h d

hf a h f b h f a f b

The weights depend only on h!

• This approximation may be poor. The approximation error is:

ε = ∫ f(x) dx – Σi ci f(xi) = -(b – a)3/(12) f ’’(η), η Є [a, b].

• Thus, if the integrand is convex –i.e., positive second derivative–, the error is negative. That is, the trapezoidal rule overestimates the true value of the integral.

12

Evaluate the integral

• Exact solution

• Graph

926477.5216)1x2(e4

1

e4

1e

2

xdxxe

1

0

x2

4

0

x2x24

0

x2

dxxe4

0

x2

Trapezoid Rule - Example


7

13


• Trapezoidal Rule: Approximation Error

ε = -(b – a)3/(12) f ’’(η) (η is a number between a and b).

Let’s take η =2. Then

ε = -43/12*[2*exp(2*2)+2*exp(2*2)+4*(2)*exp(2*2)] =-3494.282

• Trapezoidal Rule

dxxe4

0

x2

%12.357926.5216

66.23847926.5216

66.23847)e40(2)4(f)0(f2

04dxxeI 84

0

x2

Trapezoid Rule - Example

14

Simpson’s 1/3-Rule (Kepler’s Rule)

• Approximate the function by a parabola

)x(f)x(f4)x(f3

h

)x(fc)x(fc)x(fc)x(fcdx)x(f

210

221100i

2

0ii

b

a

x0 x1x

f(x)

x2h h

L(x)


8

15

1 xx

0 xx

1 xx

h

dxd ,

h

xx ,

2

abh

2

ba x ,bx ,ax let

)x(f)xx)(xx(

)xx)(xx(

)x(f)xx)(xx(

)xx)(xx( )x(f

)xx)(xx(

)xx)(xx()x(L

2

1

0

1

120

21202

10

12101

200

2010

21

)x(f2

)1()x(f)1()x(f

2

)1()(L 21

20

Simpson’s 1/3-Rule - Derivation

• Use a quadratic Lagrange interpolation:

16

1

1

23

2

1

1

3

1

1

1

23

0

1

12

1

0

21

1

10

1

1

b

a

)2

ξ

3

ξ(

2

h)f(x

)3

ξ(ξ)hf(x)

2

ξ

3

ξ(

2

h)f(x

)d ξ1ξ(ξ2

h)f(x)d ξξ1()hf(x

)d ξ1ξ(ξ2

h)f(xdξ)(Lhf(x)dx

)f(x)4f(x)f(x3

hf(x)dx 210

b

a

• Integrate the Lagrange interpolation

• Again, the weights depend only on h!

Simpson’s 1/3-Rule - Derivation

Thomas Simpson (1710 – 1761, England)


9

17

• Approximate by a cubic polynomial

)x(f)x(f3)x(f3)x(f8

h3

)f(xc)f(xc)f(xc)f(xc)x(fcdx)x(f

3210

33221100i

3

0ii

b

a

x0 x1x

f(x)

x2h h

L(x)

x3h

Simpson’s 3/8-Rule

18

)x(f)xx)(xx)(xx(

)xx)(xx)(xx(

)x(f)xx)(xx)(xx(

)xx)(xx)(xx(

)x(f)xx)(xx)(xx(

)xx)(xx)(xx(

)x(f)xx)(xx)(xx(

)xx)(xx)(xx()x(L

3231303

210

2321202

310

1312101

320

0302010

321

)x(f)x(f3)x(f3)x(f8

h33

a-bh ; L(x)dxf(x)dx

3210

b

a

b

a

• Lagrange interpolation

• Integrate to obtain the rule

Simpson’s 3/8-Rule


10

19

Example: Simpson’s Rules


• Simpson’s 1/3-Rule: Approximation Error

ε = -(b – a)5/(2880) f (4)(η) (η is a number between a and b).

Since f (4)(η) >0, the error is negative (overshooting).

Let’s take η=2.5. Then,

ε = -45/(2880) [{exp(2*2.5)*[16*(2.5)+32]} =-3799.3769

• Simpson’s 1/3-Rule

dxxe4

0

x2

4 2

0

4 8

(0 ) 4 (2 ) (4 )3

20 4(2 ) 4 8240 .411

35216 .926 8240 .411

57 .96%5216 .926

x hI xe dx f f f

e e

20

Example: Simpson’s Rules


• Simpson’s 3/8-Rule

• Simpson’s 3/8-Rule: Approximation Error

ε = -(b – a)5/(6480) f (4)(η) (η is a number between a and b).

dxxe4

0

x2

4 2

0

3 4 8(0) 3 ( ) 3 ( ) (4)

8 3 3

3(4/3)0 3(19.18922) 3(552.33933) 11923.832 6819.209

85216.926 6819.209

30.71%5216.926

x hI xe dx f f f f


11

21

Midpoint Rule

• Newton-Cotes Open Formula

where η ∊ [a,b].

)(f24

)ab()

2

ba(f)ab(

)x(f)ab(dx)x(f3

m

b

a

a b x

f(x)

xm

The midpoint rule amounts to compute the area of the rectangle

• Note: This rule does not make any use of the end points.

22

Two-point Newton-Cotes Open Formula

• Approximate by a straight line

)(f108

)ab()x(f)x(f

2

abdx)x(f

3

21

b

a

x0 x1x

f(x)

x2h h x3h


12

23

Three-point Newton-Cotes Open Formula

• Approximate by a parabola

)(f23040

)ab(7

)x(f2)x(f)x(f23

abdx)x(f

5

321

b

a

x0 x1x

f(x)

x2h h x3h h x4

24

Better Numerical Integration

• Composite integration

– Composite Trapezoidal Rule

– Composite Simpson’s Rule

• Richardson Extrapolation

• Romberg integration


13

25

Composite Trapezoid Rule

0

1

2

3

4

5

6

7

3 5 7 9 11 13 15

Two segments

0

1

2

3

4

5

6

7

3 5 7 9 11 13 15

Four segments

0

1

2

3

4

5

6

7

3 5 7 9 11 13 15

Many segments

0

1

2

3

4

5

6

7

3 5 7 9 11 13 15

Three segments

To improve the Trapezoid Rule, first splits the interval of integration [a,b] into N smaller, uniform subintervals, and then applies the trapezoidal rule on each of them.

26


)x(f)x(f2)2f(x)f(x2)f(x2

h

)f(x)f(x2

h)f(x)f(x

2

h)f(x)f(x

2

h

f(x)dxf(x)dxf(x)dxf(x)dx

n1ni10

n1n2110

x

x

x

x

x

x

b

a

n

1n

2

1

1

0

x0 x1x

f(x)

x2h h x3h h x4

n

abh

• Use the Trapezoid Rule in n intervals. Then, add them together.


14

27


• Evaluate the integral dxxeI4

0

x2

%66.2 95.5355

)4(f)75.3(f2)5.3(f2

)5.0(f2)25.0(f2)0(f2

hI25.0h,16n

%50.10 76.5764)4(f)5.3(f2 )3(f2)5.2(f2)2(f2)5.1(f2

)1(f2)5.0(f2)0(f2

hI5.0h,8n

%71.39 79.7288)4(f)3(f2

)2(f2)1(f2)0(f2

hI1h,4n

%75.132 23.12142)4(f)2(f2)0(f2

hI2h,2n

%12.357 66.23847)4(f)0(f2

hI4h,1n

28

Composite Trapezoid Rule with Unequal Segments

• Evaluate the integral

Use the following hi‘s: {h1=2, h2=1, h3=0.5, h4=0.5}

dxxeI4

0

x2

%45.14 58.5971 45.3

2

0.5

5.33e 2

0.532

2

120

2

2

)4()5.3(2

)5.3()3(2

)3()2(2

)2()0(2

)()()()(

87

76644

43

21

4

5.3

5.3

3

3

2

2

0

ee

eeee

ffh

ffh

ffh

ffh

dxxfdxxfdxxfdxxfI


15

29

Composite Simpson’s Rule

x0 x2x

f(x)

x4h h xn-2h xn

n

abh

…...

• Piecewise Quadratic approximations

hx3x1 xn-1

30

Composite Simpson’s Rule


• Using n = 2, h = 2

• Using n = 4, h = 1

dxxeI4

0

x2

%70.8 975.5670

e4)e3(4)e2(2)e(403

1

)4(f)3(f4)2(f2)1(f4)0(f3

hI

8642

%96.57 411.8240e4)e2(403

2

)4(f)2(f4)0(f3

hI

84


16

31

Composite Simpson’s Rule with Unequal Segments


Using h1 = 1.5, h2 = 0.5

dxxeI4

0

x2

%76.3 23.5413

4)5.3(433

5.03)5.1(40

3

5.1

)4()5.3(4)3(3

)3()5.1(4)0(3

)()(

87663

2

1

4

3

3

0

eeeee

fffh

fffh

dxxfdxxfI

32

• Newton-Cotes Formulae

- Nodes: Use evenly-spaced functional values

- Weights: Derived from an approximation required to be equal for a polynomial of order lower or equal to the degree of the polynomials used to approximate the function. Given nodes, best!

- Problem: Can explode for large n (Runge’s phenomenon)

• Q: Can we use more efficient weights and nodes? Yes!

• Gaussian Quadratures

- Gaussian quadrature sets the nodes and the weights in such a way that the approximation is exact when f(.) is a low order polynomial. Best choice for both, nodes and weights!

Gaussian Quadratures


17

33


• In fact, Gaussian quadrature is more general than simple integration, it computes an approximation to the weighted integral:

• Gaussian Quadratures

- Select functional values at non-uniformly distributed points to achieve higher accuracy. The values are not predetermined, but unknowns to be determined.

- Change of variables the interval of integration is [-1,1].

- Gauss-Legendre formulae for nodes and weights

- With n nodes, delivers exact answer if f is (2n-1)th-order polynomial.

n

iii

b

a

xfcf(x)c(x)dx1

)(

34

n

iii xfcf(x)dx

1

1

1

)(

• The Gauss-Legendre quadrature rule is in general stated as:

the ci's are weights, the xi's are the quadrature nodes. The approximation error term, ε, is called the truncation error of integration.

• Goal: Get an exact answer if f is a (2n-1)th-order polynomial. (With n=5, we get an exact answer f is a 9th-order polynomial).

• For Gauss-Legendre quadrature, the nodes are chosen to be zeros of certain Legendre polynomials. They are not trivial to compute.



18

35

n

iicdxf(x)dx

1

1

1

1

1

121• To get right answer for f(x) = 1

• To get right answer for f(x) = x

• To get right answer for f(x) = x2

• To get right answer for f(x) = xj for j=0,...., 2n-1

A system of 2n equations in 2n unknowns (n ci,’s & nΨi,’s).

Gaussian Quadratures – Nodes and Weights

n

iiicxdx

1

1

1

0

n

iiicdxx

1

21

1

2 3/2

n

i

jii

j cdxx1

1

1

36

• By construction we get right answer for

f(x) = 1 (j=0), f(x) = x (j=1), ...., f(x) = xj 1 (j=2n-1),

enough to get the right answer for any polynomial of order 2n-1.

• Example: n=2 A system of 4 equations and 4 unknowns:

Gaussian Quadratures – Nodes and Weights

3

1x

3

1x

1c1c

xcxc0dxx xf

xcxc3

2dxx xf

xcxc0xdx xf

cc2dx1 1f

2

1

2

1

322

31

1

1 133

222

21

1

1 122

221

1

1 1

2

1

1 1


19

37

Change of Interval for Gaussian Quadrature

• Coordinate transformation from [a,b] to [-1,1]

This can be done by an affine transformation on t and a change of variables.

t2t1a b

n

iii

b

axfc

abdx

ababx

abfdttf

1

1

1)(

2)

2()

22()(

btx

atx

dxab

dt

abx

abt

1

12

22

38

Gaussian Quadrature on [-1, 1]

• For n=2, we have four unknowns: c1, c2, x1, x2.

• We have already solved this problem:

c1=1; c2=1; x1,=-1/√3; & x2 ,= 1/√3.

)x(fc)x(fc)x(fc)x(fcdx)x(f nn2211i

1

1

n

1ii

)f(xc)f(xc

f(x)dx :2n

2211

1

1

x2x1-1 1

• Gauss Quadrature General formulation:


20

39


• Approximation formula for n = 2:

)3

1()

3

1()(

1

1ffdxxfI

40


• Now, choose (c1, c2, c3, x1, x2, x3) such that the method yields “exact integral” for f(x) = x0, x1, x2, x3,x4, x5. (Again, (c1, c2, c3, x1, x2, x3) are calculated by assuming the formula gives exact expressions for integrating a fifth order polynomial).

)()()()( :3Case 332211

1

1xfcxfcxfcdxxfn

x3x1-1 1x2


21

41


533

522

511

1

1

55

433

422

411

1

1

44

333

322

311

1

1

33

233

222

211

1

1

22

332211

1

1

321

1

1

0

5

2

0

3

2

0

21

xcxcxcdxxxf

xcxcxcdxxxf

xcxcxcdxxxf

xcxcxcdxxxf

xcxcxcxdxxf

cccxdxf

5/3

0

5/3

9/5

9/8

9/5

3

2

1

3

2

1

x

x

x

c

c

c

42


• Approximation formula for n = 3

)5

3(

9

5)0(

9

8)

5

3(

9

5)(

1

1fffdxxfI


22

43

Example (1): Gaussian Quadrature

Evaluate

First, a coordinate transformation

• Two-point formula (n=2)

33.34%)( 543936.3477376279.3468167657324.9

)3

44()

3

44()

3

1()

3

1()( 3

44

3

441

1

eeffdxxfI

926477.5216dtteI4

0

t2

1

1

1

1

444

0

2 )()44(

2dxdt ;2222

dxxfdxexdtteI

xab

xab

t

xt

44

Example (1): Gaussian Quadrature• Three-point formula (n=3)

• Four-point formula (n=4)

4.79%)( 106689.4967

)142689.8589(9

5)3926001.218(

9

8)221191545.2(

9

5

)6.044(9

5)4(

9

8)6.044(

9

5

)6.0(9

5)0(

9

8)6.0(

9

5)(

6.0446.04

1

1

eee

fffdxxfI

%)37.0( 54375.5197

)339981.0()339981.0(652145.0

)861136.0()861136.0(34785.0)(1

1

ff

ffdxxfI


23

45

Example (2): Gaussian Quadrature

Evaluate 44949742.2

1 64.1

0

2

2

dxeIx

1

1

1

1

)]1(82[.2

164.1

0

2 )(2

82.

2

82.

2

1

82 );1(82.82.82.22

22

dxxfdxedteI

dx.dtxxab

xab

t

xt

First, a coordinate transformation:

46

Example: Gaussian Quadrature

• Three-point formula (n=3)

%)007.0(0.44946544

)0.19271450+0.63509351+9(0.5461465*.32713267

9

5

9

8

9

5

2

82.

)6.0(9

5)0(

9

8)6.0(

9

5

2

82.)(

2

82.

222 )]6.01(82[.2

1)]01(82[.

2

1)]6.01(82[.

2

1

1

1

eee

fffdxxfI

%)065.0( 44978962.)0.43323413+7(0.9417114*0.32713267

2

82.)

3

1()

3

1(

2

82.)(

2

82.22 )]

3

11(82[.

2

1)]

3

11(82[.

2

11

1

eeffdxxfI

• Two-point formula (n=2)


24

47

Monte Carlo Integration

• In our motivation of integrals, we evaluated a one-dimensional integral by a sum of rectangles, using the end points of each interval to measure the height. Some of these rectangles overestimated the area, some underestimated the area.

• Let’s focus on one of those rectangles, say with base [a, b]. We can also use as the height a randomly selected interior point, x1 ∊ [a, b] and estimate the integral, say I(x1). Of course, it may over- or under-estimate the area.

• But, we can randomly select N interiors points and get N estimations of the area. Some points will under-estimate, some points will over-estimate, but, statistical intuition suggests that the average may work.

• In fact, as N increases, the average of the integral converges to the integral.

48

Monte Carlo Integration – Example 1

• Example 1: We want to do MC integration for (exact integral = 5,216.92):

> M <- 200> x <- runif(M,0,4)> All_I <- matrix(0,M,1)> a <- 0> b <- 4> m <- 1> while (m <= M) {+ Int <- (b-a)*(x[m] * exp(2*x[m]))+ m <- m+ 1+ All_I[m] <- Int+ }> IN <- sum(All_I)/M> IN[1] 5489.388

dxxe4

0

x2


25

49


• MC Integration can be applied to any area, like the area of a circle.

• Example 2: We want to estimate the area of a circle with radius, r=2 (exact area = π * r2 = π * 4 =12.56637):

> M <- 100> x <- runif(M,-2,2)> y <- runif(M,-2,2)> #box is area 16.> distance.from.0 <- sqrt(x*x + y*y)> inside.circle <- (distance.from.0 < 2)> area <- 16*sum(inside.circle)/M> area[1] 12.48

50

Monte Carlo Integration – Properties


26

51


• Example 3: Back to the trapezoid example, where we wanted to integrate the following function:

> All_I <- matrix(0,M,1)> a <- 0> b <- 4> m <- 1> while (m <= M) {+ Int <- (b-a)/((b-a)/2)* (x[m] * exp(2*x[m]) + y[m] * exp(2*y[m]))+ m <- m+ 1+ All_I[m] <- Int+ }> IN <- sum(All_I)/M> IN[1] 5448.996

Note: The exact integral is 5,216.93.

dxxe4

0

x2

52

Monte Carlo Integration & Multiple Integrals

• Q: Why use the MC estimator instead of the also very simple determinist quadrature rules?

• Quadrature rules do not extend very well to higher dimension. An approach is to rewrite the problem in terms of one-dimensional integrals. For two or three dimension it may work well, but for more than four dimensions it become imprecise.

• These rules suffer from the curse of dimensionalitily.

• Monte Carlo integration extends well to many dimensions. IT is based on repeated function evaluations, not repeated integrations using one-dimensional methods.

Popular MC algorithm: Markov chain Monte Carlo (MCMC), which include the Metropolis-Hastings algorithm and Gibbs sampling.


27

53

Q: What's the integral of (1/cabin)d(cabin)?A: A natural log cabin!

chapter 12-b - bauer college of business · ch 12 - integral calculus 8/8/2019 1 1 chapter 12-b...

Documents