chapter 12. numerical differentiationfaculty.nps.edu/oayakime/ae2440/slides/chapter 12... · all...

All rights reserved. No part of this publication may be reproduced, distributed, or transmitted, unless for course participation, in any form or by any means, or stored in a database or retrieval system, without the prior written permission of the Publisher and/or Author. Contact the American Institute of Aeronautics and Astronautics, Professional Development Programs, 12700 Sunrise Valley Drive, Suite 200, Reston, VA 20191-5807. 1 out of 24

Engineering Computations and Modeling in MATLAB/Simulink

Chapter 12. Numerical Differentiation



Outline• 12.1 Introduction• 12.2 Two-Point Difference Approximations of the First-Order Derivative• 12.3 Errors Estimates from Taylor Series Expansion• 12.4 More Accurate Multiple-Point Approximations• 12.5 Second- and Higher-Order Derivatives• 12.6 Alternative Ways of Obtaining Approximation Formulas• 12.7 Partial Derivatives• 12.8 Using the MATLAB Functions




MATLAB Functions

diff - Finite differences

gradient - Numerical gradientpolyder - Derivative of a polynomial



Sample Points

( )y f x=

x

y

2 2( , )i ix y− −

x

y

( )y f x=

( , )i ix y

1 1( , )i ix y− −

1 1( , )i ix y+ +

2 2( , )i ix y+ +

1 1i i i ix x x x− +− ≠ −

1 1i i i ix x x x h− +− = − =

hh h h h h h hh

3 3( , )i ix y− −3 3( , )i ix y+ +



The First-Order Derivative:First Shoty

h h hhh h

3ix − 2ix − 1ix − ix 2ix + 3ix +1ix +

3( )if x −2( )if x − 1( )if x −

( )if x

2( )if x + 3( )if x +

1( )if x +

1( ) ( )( ) i ii

f x f xf xh

+ −′ ≈ Forward difference approximation

1( ) ( )( ) i ii

f x f xf xh

−−′ ≈ Backward difference approximation

FDA

BDA



The First-Order Derivative:Second Shoty

h h hhh h

3ix − 2ix − 1ix − ix 2ix + 3ix +1ix +

3( )if x −2( )if x − 1( )if x −

( )if x

2( )if x + 3( )if x +

1( )if x +

1 1( ) ( ) ( ) ( )1( )2

i i i ii

f x f x f x f xf xh h

+ −− − ′ ≈ +

1 1( ) ( )( )2

i ii

f x f xf xh

+ −−′ ≈ centered difference approximation



The First-Order Derivative: Formal Approach

2 3 4( )

1 ( ) ( ) ( ) ...2! 3

( ) ( )!

( )! 4

ivi i i ii i f x f x fh h hf x f x h xf x+ ′′ ′′′= +′+ + + +

2 3 4( )

1 ( ) ( ) ( ) ...2! 3

( ) ( )!

( )! 4

ivi i i ii i f x f x fh h hf x f x h xf x− ′′ ′′′= −′− + + −

Taylor series

12 3

( )( ) ( ) ( ) ...2! 3

( )! 4!

( ) ( ) ivi i

i iii

h h hf x ff x f x fx xxh

f +′ − ′′ ′′′= − − − −

12 3

( )( ) ( ) ( ) ...2! 3

( ) ( ) ( )! 4!

ivi ii

i ii

h h hf x ff x f xh

f xx xf −′ − ′′ ′′′= + − + −

Forward difference approximation

Backward difference approximation

( ) ( )2 ihe h f x′′≅ −

( ) ( )2 ihe h f x′′≅



The First-Order Derivative: Making a Trick

2 3 4( )

1 ( ) ( ) ( ) ...2! 3

( ) ( )!

( )! 4

ivi i i ii i f x f x fh h hf x f x h xf x+ ′′ ′′′= +′+ + + +

2 3 4( )

1 ( ) ( ) ( ) ...2! 3

( ) ( )!

( )! 4

ivi i i ii i f x f x fh h hf x f x h xf x− ′′ ′′′= −′− + + −

Taylor series

1

3

1 (2 ( ) ...) (3

( ) )ii i if x f x h xx fhf+ − ′′′− = + +′

1 12( ) ( ) ( ) ...

2 6( )i

i ii

f x f xh

hx xf f+ −′ − ′′′= − − centered difference approximation

22( ) ( )

6 ihe h f x′′′≅ −

*Note, we used two Taylor series formulas and that is why this approximation of the first-order derivative is called

three-point centered difference approximation (even though the third, central, point does not show up)



The Second-Order Derivative:Keep Talking

2 3 4( )

1 ( )( ) ( ) ( ) ...2! 3

( ( )! 4!

) ii i

vii i i

h h hf x f f xf x f x f xx h+ ′ ′′′= + + + +′′ +

2 3 4( )

1 ( )( ) ( ) ( ) ...2! 3

( ( )! 4!

) ii i

vii i i

h h hf x f f xf x f x f xx h− ′ ′′′= − + − −′′ +

Taylor series

1 1

42 ( )2 2 ( ) .( ) ( ) ( ( ) ..

4)

!i i i iiivf f xhf x f x f x h x+ − ′ +′+ = + +

1 (12

)2

2 ( ) ...1

( )) ( ( ( )2

) ivi

i i ii

f x f x f x xf fh

hx + −− += −′ −′

three-point centered difference approximation for the second-order derivative

22 ( )( ) ( )

12iv

ihe h f x≅ −



Let’s Try Symbolic Math Toolbox

%% Define symbolic variablessyms h f_i f_iplus1 f_prime f _2prime%% Introduce the equationEq = -f_iplus1 + f_i + h*f_prime + h^2*f_2prime/factorial(2);%% Solve the equation for f_primesol = solve(Eq,f_prime);%% Collect coefficients and display the result for f_primer=collect(sol,f_2prime); pretty(r)

1( )( ) () ii

ixh

f f x fx + −≈′ Forward difference approximation( ) ( )

2 ihe h f x′′≅ −

It works!



Exploiting SMT One More Time%% Define symbolic variables

syms h f_i f_iplus1 f_iminus1syms f_prime f_2prime f_3prime f_4prime%% Introduce equations for two pointsEqplus1 = -f_iplus1 + f_i + h*f_prime + h^2*f_2prime/factorial(2) ...+ h^3*f_3prime/factorial(3) + h^4*f_4prime/factorial(4);Eqminus1 = -f_iminus1 + f_i - h*f_prime + h^2*f_2prime/factorial(2) ...- h^3*f_3prime/factorial(3) + h^4*f_4prime/factorial(4);%% Resolve two equations for f_prime and f_2primesol=solve(Eqplus1,Eqminus1,f_prime,f_2prime);%% Collect coefficients and display the result for f_2primer=collect(sol.f_2prime,f_4prime); pretty(r)

12

12( ) (( () ) )i ii

if x f xf x f xh

+ −−=′ +′ centered difference approximation

22 ( )( ) ( )

12iv

ihe h f x≅ −



Why Not to Use More Points?

1

2 3 4 5( ) ( )( ) ( ) ( ) ( ) ( ) ...

2!( ) (

4!)

3! 5!iv v

i i i iii i f x h h h hf x f x h f x f x f x f x+ ′′ ′′′= + + + +′ + +

1

2 3 4 5( ) ( )( ) ( ) ( ) ( ) ( ) ...

2!( ) (

4!)

3! 5!iv v

i i i iii i f x h h h hf x f x h f x f x f x f x− ′′ ′′′= − + − +′ − +Taylor series

3 5( )

1 12 22 ( ) ( ) ...3! 5!

(( ) )) ( vii i ii

h hf x f x fh xf xx f+ − ′′′− = + + +′

2 3 4 5( ) ( )

2(2 ) (2 ) (2 ) (( 2 )2 ( ) ( ) ( ) ( ) ...

2! 3! 4! 5) )(

!( ) iv v

i ii i i ii f x f x f x f xh h h hf x f x hf x+ ′′ ′′′= + + + +′ + +

2 3 4 5( ) ( )

2(2 ) (2 ) (2 ) (( 2 )2 ( ) ( ) ( ) ( ) ...

2! 3! 4! 5) )(

!( ) iv v

i ii i i ii f x f x f x f xh h h hf x f x hf x− ′′ ′′′= − + − +′ − +

2 2

3 5( )2(2 ) 2(2 )4 ( ) ( ) ...

3( ) ( ) )

! 5!(i i i

vi if h hx f x f xh x xf f+ − ′ ′′′− = + + +

2 1 1 2( ) ( )8 (812

( ) ) ( )i i i ii

f x f x f x f xf xh

+ + − −− + − +=′ Five-point* centered difference

approximation for the first-order derivative

44 ( )( ) ( )

30v

ihe h f x≅



Exploiting SMT Again%% Define symbolic variablessyms h f_i f_iminus2 f_iminus1 f_iplus1 f_iplus2syms f_prime f_2prime f_3prime f_4prime f_5prime%% Introduce equations for four pointsEqplus2 = -f_iplus2 + f_i + 2*h*f_prime + (2*h)^2*f_2prime/factorial(2)...

+ (2*h)^3*f_3prime/factorial(3) + (2*h)^4*f_4prime/factorial(4)...+ (2*h)^5*f_5prime/factorial(5);

Eqplus1 = -f_iplus1 + f_i + h*f_prime + h^2*f_2prime/factorial(2)...+ h^3*f_3prime/factorial(3) + h^4*f_4prime/factorial(4)...+ h^5*f_5prime/factorial(5);

Eqminus1 = -f_iminus1 + f_i - h*f_prime + h^2*f_2prime/factorial(2)...- h^3*f_3prime/factorial(3) + h^4*f_4prime/factorial(4)...- h^5*f_5prime/factorial(5);

Eqminus2 = -f_iminus2 + f_i - 2*h*f_prime + (2*h)^2*f_2prime/factorial(2)...- (2*h)^3*f_3prime/factorial(3) + (2*h)^4*f_4prime/factorial(4)...- (2*h)^5*f_5prime/factorial(5);

%% Resolve four equations for f_prime, f_2prime, f_3prime, and f_4primesol=solve(Eqminus2,Eqminus1,Eqplus1,Eqplus2,f_prime,f_2prime,f_3prime,f_4prime);%% Collect coefficients and display the result for f_primer=collect(sol.f_prime,f_5prime); pretty(r)



Summary for the First-Order Derivatives

1( )( ) () ii

ixh

f f x fx + −=′

1( )( () )ii

if x f xf xh

−−=′

2-point forward difference approximation

2-point backward difference approximation

( ) ( )2 ihe h f x′′≅ −

( ) ( )2 ihe h f x′′≅

1 1( (2

) ( ) )ii

iff x fh

x x+ −′ −= 3-point* centered difference approximation

22( ) ( )

6 ihe h f x′′′≅ −

2 14( ) (( ) () 32

)i i ii

f x f x xf fh

x + +′ − + −=

1 2( ) ( ) (3 )42

( ) i i ii

f x f x ff x xh− −+′ −

=



22( ) ( )

3 ihe h f x′′′≅

22( ) ( )

3 ihe h f x′′′≅

2 1 1 2( ) ( )8 (812

( ) ) ( )i i i ii

f x f x f x f xf xh

+ + − −− + − +=′ 5-point* centered difference approximation4

4( ) ( )30

Vi

he h f x≅



Summary for the Second-Order Derivatives

1 12

2( ) (( () ) )i ii

if x f xf x f xh

+ −−=′ +′

2 12

2( ) (( () ) )i ii

if x f x f xf xh

+ +′ − +=′

3-point centered difference approximation


22 ( )( ) ( )

12iv

ihe h f x≅ −

( ) ( )ie h hf x′′′≅ −

1 22

( ) (2 (( ) ) )i i ii

f x f xf xx fh

− −+′ −=′ 3-point backward difference approximation

( ) ( )ie h hf x′′′≅



Higher-Order Derivatives

3 2 13

( ) ( ) (3 3) ) )( (i i ii

if x ff x x f x f xh

+ + +− + −′′ =′

1 2 33

( ) ( ) ( )( ) 3 ( )3i ii

i if x f x f x f xf xh

− − −′ + −′ −=′



( )3( ) ( )2

ivie h hf x≅ −

3( ) ( )2 ie h hf x′′′≅

2 1 1 23

2( ) ( ) ( ) ( )( ) 22i

i i i if x f x ff x f xh

x + + − −′ − + −=′′ 5-point* centered difference approximation

2 2 ( )1( ) ( )4

vie h h f x≅ −

2 1( 1) 24

( ) ( ) ( ) ( )( 4 (4 6 ))i i ivi

i i if x f x f xf x f x f xh

+ + − −− + − +=

5-point centered difference approximation( )1( ) ( )6

viie h hf x≅ −



Partial Derivatives

( , ) ( , )( , ) i x i i ix i i

x

f x h y f x yf x yh

+ −≈

( , ) ( , )( , ) i i i x ix i i

x

f x y f x h yf x yh

− −≈

Forward difference approximation

Backward difference approximation

( , ) ( , )( , )2

i x i i x ix i i

x

f x h y f x h yf x yh

+ − −≈ Centered difference approximation

( , )ii

x x xy y

f f x yx =

=

∂=∂

( , )z f x y=

2

2 2

( , ) 2 ( , ) ( , )( , ) i x i i i i x ixx

x

f x h y f x y f x h yf f x yx h

+ − + −∂= ≈∂

2

( , )

( , ) ( , ) ( , ) ( , )

4 4

xy

i x i y i x i y i x i y i x i y

x y x y

f f x yx y

f x h y h f x h y h f x h y h f x h y hh h h h

∂= ≈∂ ∂

+ + − − + + − − − −+

First-Order Derivatives

Second-Order Derivatives

2

2 2

( , ) 2 ( , ) ( , )( , ) i i y i i i i y

yyy

f x y h f x y f x y hf f x y

y h+ − + −∂

= ≈∂



The MATLAB diff Function

1( )( () )ii

if x f xf xh

−−=′

1x 2x 3x 4x 5x

2 1x x− 3 2x x− 4 3x x− 5 4x x−

3 2 12x x x− + 4 3 22x x x− + 5 4 32x x x− +

4 3 2 13 3x x x x− + − 5 4 3 23 3x x x x− + −

5 4 3 2 14 6 4x x x x x− + − +

diff(x)

x

diff(x,2)

diff(x,3)

diff(x,4)

4 3( 1) 24

( ) ( ) ( ) ( ) ( )4 4( ) 6iv i i i ii

if x f x f x f x f xf xh

+ + + +− + − +=

12

12( ) (( () ) )i ii

if x f xf x f xh

+ −−=′ +′

33

2 1( ) ( ) (3 3) ) )( (i i ii

if x ff x x f x f xh

+ + +− + −′′ =′

2-point forward/backward difference approximation for f’ (Slide 14)

3-point forward/centered/backward difference approximation for f’’ (Slide 15)

4-point forward/backward difference approximation for f’’’ (Slide 16)

5-point centered difference approximation for f(iv) (Slide 16)



Using the diff Function to Compute Derivatives

x=linspace(0,pi,50); h=x(2)-x(1);y=sin(2*x).*cos(x);plot(x,y,'o-.'), holdplot(x(2:end),diff(y)/h,'+m')gridlegend('Function','1^{st}-order derivative','Location','n')

x=linspace(0,pi,50); h=x(2)-x(1);y=sin(2*x).*cos(x);plot(x,y,'o-.'), holdplot(x(2:end),diff(y)/h,'+m')plot(x(2:end-1),diff(y,2)/h^2,'pg'), gridlegend('Function','1^{st}-order derivative',...

'2^{nd}-order derivative','Location','n')

12

12( ) (( () ) )i ii

if x f xf x f xh

+ −−=′ +′

1( )( () )ii

if x f xf xh

−−=′ Two-point backward difference approximation

Three-point centered difference approximation



A Couple of Comments

x=linspace(0,pi,50); h=x(2)-x(1);y=sin(2*x).*cos(x);plot(x,y,'bo-.'), holdplot(x(1:end-1),diff(y)/h,'+m')gridlegend('Function','1^{st}-order derivative','Location','n')

1( )( ) () ii

ixh

f f x fx + −=′

x=linspace(0,pi,50) +0.1*rand(1,50);y=sin(2*x).*cos(x);plot(x,y,'bo-.'), holdplot(x(1:end-1),diff(y)./diff(x),'+m')gridlegend('Function','1^{st}-order derivative','Location','n')

1

1

( ) (( ) )i

i i

i i

f x ff x xx x+

+

′ −=

−

Two-point forward difference approximation

Variable step-size formula for the two-point forward difference approximation



Using the Gradient FunctionFx=gradient(A)[Fx,Fy]=gradient(A)[Fx,Fy,Fz,...]=gradient(A)[...] = gradient(A,h)[...] = gradient(A,h1,h2,...)

F F FFx y z

∂ ∂ ∂∇ = + + +

∂ ∂ ∂i j k

Output argument(s)Input argument A is:

a vector a matrix an array

Fx A vector

Fx,Fy Two matrices

Fx,Fy,Fz,… Multiple arrays

>> y=[1 3 6 7 3 4 6]y = 1 3 6 7 3 4 6>> gradient(y)ans = 2.0000 2.5000 2.0000 -1.5000 -1.5000 1.5000 2.0000>> gradient(y,10)ans = 0.2000 0.2500 0.2000 -0.1500 -0.1500 0.1500 0.2000>> gradient(y,y)ans = 1 1 1 1 1 1 1>> gradient(y,0.1*([0:7]+rand(1,8)))ans = 14.248 21.016 26.974 -20.493 -12.843 10.560 52.542



Visualizing 2-D Vector Field and 3-D Surface Normals

v = -2:0.2:2;[x,y] = meshgrid(v);z = x .* exp(-x.^2 - y.^2);[px,py] = gradient(z,0.2,0.2);colormap jetcontour(v,v,z), hold onquiver(v,v,px,py), hold offaxis equal

v = -pi/2:pi/16:pi/2;[x,y] = meshgrid(v);z = sin(x).*cos(y);[px,py] = gradient(z,0.1,0.1);colormap('Spring')contourf(v,v,z), hold onquiver(v,v,px,py), hold offaxis equal

v =-2:0.2:2;[x,y] = meshgrid(v);z = x .* exp(-x.^2 - y.^2);[U,V,W] = surfnorm(x,y,z);colormap bonesurf(v,v,z), hold onquiver3(v,v,z,U,V,W,'r')axis equal

Note, while surfnorm plots a surface with surf and displays its surfacenormals as radiating vectors, quiver3 displays vectors withcomponents (u,v,w) at the points (x,y,z)

quiver displays vectors with components (u,v) at the points (x,y)



Using the polyder Function

r=[1,2,4,6];a=poly(r);b=polyder(a);x=linspace(min(r),max(r),30);plot(x,polyval(a,x),'bo-.'), hold onplot(r,polyval(a,r),'Marker','p','MarkerFaceColor','r','MarkerSize',15)plot(x,polyval(b,x),'+m'), hold off, gridlegend('Polynomial','Roots','Derivative','Location','North')

Function Descriptionconv computes a product of two polynomialsdeconv performs a division of two polynomials

(returns the quotient and remainder)poly creates a polynomial with specified rootspolyder calculates the derivative of polynomials

analyticallypolyeig solves polynomial eigenvalue problempolyfit produces polynomial curve fittingpolyint integrates polynomial analyticallypolyval evaluates polynomial at certain pointspolyvalm evaluates a polynomial in a matrix senseresidue converts between partial fraction

expansion and polynomial coefficientsroots finds polynomial rootspoly2sym converts a vector of polynomial

coefficients to a symbolic polynomialsym2poly converts polynomial coefficients to a

vector of polynomial coefficients



The End of Chapter 12

Questions?

chapter 12. numerical differentiationfaculty.nps.edu/oayakime/ae2440/slides/chapter 12... · all...

Documents