The Taylor series method for ordinarydifferential equations

Karsten Ahnert1,2

Mario Mulansky2

1 Ambrosys GmbH, Potsdam2 Institut für Physik und Astronomie, Universität Potsdam

December, 8, 2011

ambrosys

1

Outline

1 Solving ODEs

2 The method

3 Implementation

4 Conclusion

2

Solving Ordinary differential equations numerically

Find a numerical solution of the initial value problem for an ODE

x = f (x , t) , x(t = 0) = x0

Example: Explicit Euler

x(t + ∆t) = x(t) + ∆t f (x(t), t) +O(∆t2)

General scheme of order s

x(t) 7→ x(t + ∆t) , or

x(t + ∆t) = Ftx(t) +O(∆ts+1)

3

Numerical methods – Overview

Methods:Steppers: x(t) 7→ x(t + ∆t)Methods with embedded error estimationAdaptive step size controlDense output

Examples:Explicit Runge Kutta methodsImplicit methods for stiff systemsSymplectic methods for Hamiltonian systemsMultistep methodsTaylor series method

4

Software for ordinary differential equations

GNU Scientific library – gsl, CNumerical recipes, C and C++www.odeint.com, C++odeint, Pythonapache.common.math, Java

5

Taylor series method

x = f (x)

Taylor series of the solution

x(t + ∆t) = x(t) + ∆t x(t) +∆t2

2!x(t) +

∆t3

3!x (3)(t) + . . .

Auto Differentiation to calculate x(t), x(t), x (3)(t), . . .Applications: Problems with high accuracy

astrophyical application, chaotic dynamical systemsArbitrary precision typesInterval arithmetics

6

Software for the Taylor method

Most software packages use generatorsATSMCC, ATOMFT – Fortran generatorGeorge F. Corliss and Y. F. Chang. ACM Trans. Math. Software, 8(2):114-144, 1982.

Y. F. Chang and George F. Corliss, Comput. Math. Appl., 28:209-233, 1994

Taylor – C GeneratorÁngel Jorba and Maorong Zou, Experiment. Math. Volume 14, Issue 1 (2005), 99-117.

TIDES – arbitrary precision, Mathematica generatorRodríguez, M. et. al, TIDES: A free software based on the Taylor series method, 2011

Operator overloadingAdol-CcppAD

Expression templatesTaylor

7

1 Solving ODEs

2 The method

3 Implementation

4 Conclusion

8

Notation

Taylor series of the ODE

x(t + ∆t) = x(t) + ∆t x(t) +∆t2

2!x(t) +

∆t3

3!x (3)(t) + . . .

Introduce the reduced derivatives Xi

Fi =1i!

(f(x(t)

))(i), Xi =

1i!

x (i)(t) , Xi+1 =1

i + 1Fi

Taylor series

x(t + ∆t) = X0 + ∆tX1 + ∆t2X2 + ∆t3X3 + . . .

9

Algebraic operations – Expression trees

Example: Lorenz system

x = σ(y − x) y = Rx − y − xz z = −bz + xy

Expression trees

σ

xy

R x

x zy x yzR

10

The algorithm – Evaluation of the expression tree

Recursive determination of the Taylor coefficients1. Initialize X0 = x(t)2. Calculate X1 = F0(X0)

3. Calculate X2 = 12F1(X0,X1)

4. Calculate X3 = 13F2(X0,X1,X2)

. . .Calculate Xs = 1

s Fs−1(X0,X1, . . . ,Xs−1)

Finally x(t + ∆t) = X0 + ∆tX1 + ∆t2X2 + . . .

In every iteration the expression tree is evaluated!

11

The algorithm – Evaluation of the expression tree

Recursive determination of the Taylor coefficients1. Initialize X0 = x(t)2. Calculate X1 = F0(X0)

3. Calculate X2 = 12F1(X0,X1)

4. Calculate X3 = 13F2(X0,X1,X2)

. . .Calculate Xs = 1

s Fs−1(X0,X1, . . . ,Xs−1)

Finally x(t + ∆t) = X0 + ∆tX1 + ∆t2X2 + . . .

In every iteration the expression tree is evaluated!

11

Algebraic operations

At every iteration the nodes in the expression tree have to beevaluated

Formulas for iteration i :

Constants: Ci = cδi,0

Dependend variable x : Xi

Summation s = l + r : Si = Li + Ri

Multiplication m = l × r : Mi =i∑

j=0LjRi−j

Division d = l/r : Di = 1/R0(Li −i−1∑j=0

DjRi−j)

Formulas for special functions exist: exp, log, cos, sin, . . .

12

Algebraic operations – Formulas

At every iteration the nodes in the expression tree have to beevaluated. Formulas for iteration i :

CConstants: Ci = cδi,0

xDependend variable: x Xi

Summation s = l + r : Si = Li + Ri

Multiplication m = l × r : Mi =i∑

j=0LjRi−j

Division d = l/r : Di = 1/R0(Li −i−1∑j=0

DjRi−j )

f(x)

Formulas for specialfunctions exist

13

Nice side effects

Step size controlError estimate: err = Xs

v = || errkεrel+εabs|xk | ||

∆t = v−1/s

No acceptance or rejection step is required

Dense output is trivially presentx(t + τ) = X0 + τX1 + τ2X2 + τ3X3 + . . . , 0 < τ < ∆t

Methods for order estimation exist

14

1 Solving ODEs

2 The method

3 Implementation

4 Conclusion

15

Implementation

Taylor

Downloadhttps://github.com/headmyshoulder/taylor

Taylor will be integrated into odeint

Implements some odeint - stepper

Modern C++Heavy use of the C++ template systemsExpression templates for expression treesTemplate meta-programming

16

Implementation

Taylor

Downloadhttps://github.com/headmyshoulder/taylor

Taylor will be integrated into odeint

Implements some odeint - stepper

Modern C++Heavy use of the C++ template systemsExpression templates for expression treesTemplate meta-programming

16

Implementation

Taylor

Downloadhttps://github.com/headmyshoulder/taylor

Taylor will be integrated into odeint

Implements some odeint - stepper

Modern C++Heavy use of the C++ template systemsExpression templates for expression treesTemplate meta-programming

16

C++ Templates

Here, C++ templates will be used to create the expressiontree – Expression TemplatesTemplate Metaprogramming to evaluate the expressiontemplatesIt basically means using the template engine to generate aprogram from which the compiler creates then the binaryC++ compilers always use the template engine (noadditional compile step required)

Template engine and templates are a functionalprogramming languageTemplates form a Turing-complete programming language−→ You can solve any problem with the template engine

17

C++ Templates

Here, C++ templates will be used to create the expressiontree – Expression TemplatesTemplate Metaprogramming to evaluate the expressiontemplatesIt basically means using the template engine to generate aprogram from which the compiler creates then the binaryC++ compilers always use the template engine (noadditional compile step required)

Template engine and templates are a functionalprogramming languageTemplates form a Turing-complete programming language−→ You can solve any problem with the template engine

17

Expression templates

template< class L , class R > struct binary_expression{

binary_expression( string name , L l , R r )...L m_l;R m_r;

};

struct terminal_expression{

terminal_expression( string name ) ...};

const terminal_expression arg1( "arg1" ) , arg2( "arg2" );

template< class L , class R >binary_expression< L , R > operator+( L l , R r ){

return binary_expression< L , R >( "Plus" , l , r );}...

template< class Expr > void print( Expr expr ) { ... }

print( arg1 );print( arg1 + ( arg2 + arg1 - arg2 ) );

18

Expression templates

Expression template are constructed during compile-timeStrong optimization – no performance lossLazynessApplications: Linear algebra systems, AD, (E)DSL

Example MTL4mtl4::dense_matrix< double > m1( n , n ) , m2( n , n ) , m3( n , n ) ;

/ / do s o m e t h i n g u s e f u l w i t h m1 , m2 , m3

mtl4::dense_matrix< double > result = m1 + 5.0 * m2 + 0.5 * m3 ;

Last line creates an expression template which is evaluated tofor( int i=0 ; i<n ; ++i )

for( int j=0 ; j<n ; ++j )result( i , j ) = m1( i , j ) + 5.0 * m2( i , j ) + 0.5 * m3( i , j );

19

First example – Lorenz system

taylor_direct_fixed_order< 25 , 3 > stepper;state_type x = {{ 10.0 , 10.0 , 10.0 }} ;double t = 0.0;double dt = 1.0;while( t < 50000.0 ){

stepper.try_step(fusion::make_vector(

sigma * ( arg2 - arg1 ) ,R * arg1 - arg2 - arg1 * arg3 ,arg1 * arg2 - b * arg3) , x , t , dt );

cout << t << "\t" << x << "\t" << dt << endl;}

ODE is a compile time sequence of expression templatesThe expression template is defined with boost::proto

No preprocessing step is necessary!

20

Expression templates and ODEs

Boost.Proto (C++ library)Creation, manipulation and evaluation of the syntax treeGrammar = Allowed expression + (optional) Transformation

Taylor library uses Proto as front end:The ODE is a set of Proto expression templatesODE is transformed into a custom expression template

struct tree_generator :proto::or_<

variable_generator< proto::_ > ,constant_generator ,plus_generator< tree_generator > ,minus_generator< tree_generator > ,multiplies_generator< tree_generator > ,divides_generator< tree_generator > ,unary_generator< tree_generator >

> { };

21

Expression templates and ODEs

Boost.Proto (C++ library)Creation, manipulation and evaluation of the syntax treeGrammar = Allowed expression + (optional) Transformation

Taylor library uses Proto as front end:The ODE is a set of Proto expression templatesODE is transformed into a custom expression template

struct tree_generator :proto::or_<

variable_generator< proto::_ > ,constant_generator ,plus_generator< tree_generator > ,minus_generator< tree_generator > ,multiplies_generator< tree_generator > ,divides_generator< tree_generator > ,unary_generator< tree_generator >

> { };

21

Expression templates and ODEs

Evaluation of the custom template – iteration several timesof the templateThe nodes implement the rules for the algebraicexpressionsOptimzation of the expression tree

R

R ×

Example: The plus nodetemplate< class Left , class Right >struct plus_node : binary_node< Left , Right >{

plus_node( const Left &left , const Right &right ): binary_node< Left , Right >( left , right ) { }

template< class State , class Derivs >Value operator()( const State &x , const Derivs &derivs , size_t which ){

return m_left( x , derivs , which ) + m_right( x , derivs , which );}

};

22

The interface

Interface for easy implementation of arbitrary ODEs existtaylor_direct_fixed_order< 25 , 3 > stepper;stepper.try_step( sys , x , t , dt );

sys represents the ODE, Example:fusion::make_vector(

sigma * ( arg2 - arg1 ) ,R * arg1 - arg2 - arg1 * arg3 ,arg1 * arg2 - b * arg3 )

x is the state of the ODE is in-place transformed

t,dt are the time and the step size

23

Results

Performance comparison against full Fortran code

The Lorenz system as test systemBenchmarking: a Fortran code with a non-ADimplementationBoth codes have the same performance, run-timedeviation is less 20%Exact result depends strongly on the used compiler (gcc4.5, gcc 4.6, gfortran, ...)

24

Comparison against other mehtods

10-12

10-9

10-6

10-3

ε rel

10-2

10-1

100

101

102

t Co

mp

Runge Kutta Cash Karp 54

Taylor s=15

Taylor s=25

Taylor has good performance for high precisionOutperforms the classical Runge-Kutta steppers

25

Conclusion

Taylor – A C++ library for the Taylor series method ofordinary differential equationsUses expression templates as basis for the automaticdifferentiationFastUses modern C++ methodsTemplate Metaprogramming is the main programmingtechnique

26

Outlook

The library is not completeImplementation of special functionsImplementation of stencils for lattice equationsImplementation of variable orderImplementation of dense output functionalityPortability layer for arbitrary precision typesIntegration into odeint

27

Resources

Download and developmenthttps://www.github.com/headmyshoulder/taylor

Odeintodeint.com

Contributions and feedbackare highly welcome

28