Computational AstrophysicsODEs

Peter Hauschildtyeti@hs.uni-hamburg.de

Hamburger SternwarteGojenbergsweg 112

21029 Hamburg

26. Juli 2018

1 / 74

Topics

I general stuffI Euler & Runge-Kutta methodsI Extrapolation methodsI stiff equations & implicit methodsI predictor-corrector methods

2 / 74

general stuff

I ODEs occur quite often in astrophysics:I hydrostatic equation in atmospheresI formal solution of radiation transport along geodesicI stellar structure equationsI orbits

I → need workhorse solvers to deal with themI PDEs are more common (next chapter)

3 / 74

general stuff

I these methods apply to all ODEsI higher order ODEs are transformed to systems of first

oder ODEs, e.g.,

d2ydx2 + q(x)

dydx

= r(x)

transforms to

dy1

dx= y2(x)

dy2

dx= r(x)− q(x)y2(x)

with y(x) = y1(x) and dy/dx = y2(x)

4 / 74

general stuff

I → need methods to solve a set of ODEs

dyi

dx= fi(x , y(1...i))

I equations of more than one independent variable→PDEs

I require far more complex methodsI → next chapter

5 / 74

boundary conditions

I need BCs to solve a ODE problem numericallyI analytical solution→ integration constantsI type of BCs determine numerical approach, e.g.:

1. initial value problem: all yi are given at one startingpoint xs

2. two-point boundary value problem: some yi are givenat xs, the others at some point xf

I first: initial value problems

6 / 74

Euler method

I consider only ODE with i = 1 for simplicity!I index yn → value of y(x) at x = xn, i.e., the nth point

going from xs to xf

I simplest approach: EulerI chose stepsize h for x then write

yn+1 = yn + hf (xn, yn)

I uses information only at the beginning of an intervalI expansion in power series→I first order accuracy:I error is O(h2)

7 / 74

Euler method

8 / 74

Euler method

I not recommended for practical use:I other methods have better accuracy for same step

sizeI instability problems (see below)

9 / 74

Runge-Kutta method

I use Euler to take trial step to xn + h/2I use Euler estimate to approximate x and y at

midpointI use these to compute full stepI → midpoint methodI → second order Runge-Kutta method

10 / 74

midpoint method

11 / 74

Runge-Kutta method

I formally:

k1 = hf (xn, yn)

k2 = hf (xn + h/2, yn + k1/2)

yn+1 = yn + k2 + O(h3)

I → second order accurate!I needs 2 evals of f () for stepsize hI can be extendedI by using several estimates at start/mid/endpointsI accuracy can be increased

12 / 74

4th order RK method

13 / 74

Runge-Kutta method

I formally:

k1 = hf (xn, yn)

k2 = hf (xn + h/2, yn + k1/2)

k3 = hf (xn + h/2, yn + k2/2)

k4 = hf (xn + h, yn + k3)

yn+1 = yn + k1/6 + k3/3 + k4/6 + O(h5)

I → fourth order accurateI better than 2nd order if h is at least 2 times larger for

same quality

14 / 74

Runge-Kutta method

I formally:I this is quite often the caseI certainly not alwaysI practical applications:I 4th order RK generally the best option

15 / 74

step size control

I very often f () changes significantly over integrationinterval

I → to ensure homogeneous quality we need to adapth

I needs to be really small in domains where f () hasnasty features

I h can be larger for smooth, slowly varying functions

16 / 74

step size control

I → general ODE solver needs to track accuracy toobtain best performance for given quality

I step-doubling:I take each step twice as h and two h/2 stepsI → total of 3 RK steps with 4 function evals eachI starting point shared→ 11 evalsI → overhead is 11/8 ≈ 1.38

(we reach accuracy for step size h/2!)

17 / 74

step size control

I indicate truncation error ∆→ difference between thetwo estimates

I use this information to adapt h automaticallyI method is 4th order→I ∆ scales as h5 →

h0 = h1

(∆0

∆1

)0.2

I given a target truncation error ∆0, use this toestimate h0 from current ∆1 and h1

I if ∆1 > ∆0 → decrease hI otherwise increase it

18 / 74

Fehlberg method

I RK of M > 4th order→ up to M + 2 evalsI 4th order most efficient?I Fehlberg→I 5th order method with 6 evalsI these 6 points can also be used to build a 4th order

formulaI → combine them to compute truncation error with

only 6 evals!I embedded RK-Fehlberg method

19 / 74

embedded RKF method

20 / 74

embedded RKF method

21 / 74

modified midpoint method

I go through a (large) step H with h/n sub-stepsI use an algorithm of the form

z0 = y(x)

z1 = z0 + hf (x , z0)

zm+1 = zm−1 + 2hf (x + mh, zm)

y(z + H) =12

[zn + zn−1 + hf (x + H, zn)]

I same as midpoint method but first and last pointsI → modified midpoint method

22 / 74

modified midpoint method

I truncation error→

yn − y(x + H) =∑

i

αih2i

I only even powers in hI → combining steps will deliver two orders better

accuracy per doublingI usually adaptive RKF’s are betterI but modified midpoint is very useful for . . .

23 / 74

Richardson extrapolation

24 / 74

Bulirsch-Stoer method

I original Richardson→ power seriesI limited convergence radius!I Bulirsch-Stoer:

I use modified midpoint methodI use rational functions in h2

I → can take rather large steps H

25 / 74

Bulirsch-Stoer method

I how to subdivide steps best?I H/nj

I Bulirsch-Stoer: nj = 2nj−2

I more efficient: nj = 2j (Deuflhard)

26 / 74

Bulirsch-Stoer method

I jmax is not determined a prioriI extrapolation delivers error limitsI → use them to limit stepsI variation: polynomial extrapolation

27 / 74

Bulirsch-Stoer method

I method works extremely well for well behaved ODEsI → smooth functionsI it does not work well for

I function evals through table lookup or interpolationI discontinuous functionsI internal singular pointsI steep changes of f () throughout interval

28 / 74

Stiff equations

I occurs for sets of ODE that have very different scalesin x

I explicit (forward) methods:I new value yn+1 computed explicitly from old value yn

I that is a real problem:

29 / 74

Stiff equations

I considery ′ = cy

I Euler formula for stepsize h→

yn+1 = yn + hy ′n = (1 + ch)yn

I if h > 2/c → disaster because yn →∞ for n→∞I → method is unstable if h > 2/cI explicit methods require stepsizes smaller than the

smallest scale of the problem!

30 / 74

instability visualization

31 / 74

Stiff equations

I that can be bad:

u′ = 998u + 1998vv ′ = −999u − 1999v

I initial values:

u(0) = 1v(0) = 0

32 / 74

Stiff equations

I analytic solution:

u = 2 exp(−x)− exp(−1000x)

v = − exp(−x) + exp(−1000x)

I explicit methods→I stepsize h ' 2/c = 1/1000 is required for stabilityI see IDL example

33 / 74

implicit methods

I evaluate RHS at the new y point!I e.g., backward Euler schemeI consider

y ′ = cy

I Euler formula for stepsize h→

yn+1 = yn + hy ′n+1

I thereforeyn+1 =

yn

1− chI → absolutely stable!

34 / 74

implicit methods

I can take arbitrary stepsI of course, accuracy will be lost if h too largeI absolute stability only for linear ODEsI but implicit methods are always more stable

35 / 74

implicit methods

I generalization to systems of ODEsI example: linear, constant coefficients

y′ = −C · y

with positive definite matrix CI explicit differencing→

yn+1 = (1− Ch) · yn

36 / 74

implicit methods

I An → 0 for n→∞ only if the largest eigenvalue of Ais < 1

I → explicit method stable only if

h <2

λmax

where λmax is the largest eigenvalue of CI implicit differencing→

yn+1 = (1 + Ch)−1 · y

37 / 74

implicit methods

I λ: eigenvalues of C→ eigenvalues of (1 + Ch)−1 are(1 + λh)−1

I →< 1 for all hI → stable!I price: must solve matrix equations at each step

38 / 74

implicit methods

I extension to nonlinear ODEs:

y′ = f(y)

I implicit differencing→

yn+1 = yn + hf(yn+1)

I non-linear equations!

39 / 74

implicit methods

I use Newton’s method→ linearization

yn+1 = yn + h[1− h

∂f∂y

]−1

· f(yn)

I if h is small→ use only one Newton iterationI → semi-implicit method

40 / 74

stiff equations

I generalizations of RK→ Kaps-Rentrop methodsI generalizations of Bulirsch-Stoer→ Bader/DeuflhardI Gear’s methods (frequently used!)

41 / 74

predictor-corrector methods

I write the solution of the ODE as

yn+1 = yn +

∫ xn+1

xn

f (t , y) dt

I approximate f (t , y) by a polynomial through a numberof previous points xn, xn−1, xn−2

I → multistep methodI result of

∫has the form

yn+1 = yn+h(β0f (xn+1, yn+1)+β1f (xn, yn)+β2f (xn−1, yn−1)+· · · )

42 / 74

predictor-corrector methods

I β0 = 0→ explicit, otherwise implicitI how to solve?I → Newton’s method→ multistep solver for stiff

problems

43 / 74

predictor-corrector methods

I → functional iterationI → predictor-corrector methodsI idea: use explicit method to get estimate yn+1

I → predictor stepI next: use estimate to correct yn+1

I → corrector stepI can be iteratedI if number of iterations is fixed at the beginningI → explicit method (why?)

44 / 74

Adams-Bashforth-Moulton method

I overall good stability (explicit!)I 3rd order version:I predictor step: Adams-Bashforth

yn+1 = yn +h12(23y ′n − 16y ′n−1 + 5y ′n−2

)+ O(h4)

I corrector step: Adams-Moulton

yn+1 = yn +h12(5y ′n+1 + 8y ′n − y ′n−1

)+ O(h4)

45 / 74

predictor-corrector methods

I have starting/stopping problems (use RK)I stepsize control hardI → use canned routines whenever possibleI historically very frequently usedI very good for very smooth functions that are

expensive to evaluate

46 / 74

2 point BCs

I consider ODE where value(s) are given at start xs

and end xf points of intervalI numerical solution has to fulfill solution at both pointsI more complicated variations existI many can be reduced to the above problem

47 / 74

2 point BCs

I example 1: Eigenvalue problemI consider

dyi

dx= fi(x , yi , λ)

I N + 1 BCs→ solution only for specific values of λ(eigenvalues)

48 / 74

2 point BCs

I eigenvalue λ is constant→I add equation of the form

yN+1 = λ

I withdyN+1

dx= 0

49 / 74

2 point BCs

I example 2: free boundary problemI xs is given, but xf needs to be determined from given

values of yi

I → N + 1 conditions

50 / 74

2 point BCs

I add a constant ‘independent’ variable

yN+1 = xf − xs

dyN+1

dx= 0

I and use transformed independent variable 0 ≤ t ≤ 1with

x − xs = tyN+1

I solve dyi/dt in [0,1]

51 / 74

2 point BCs

I two classes of methods:1. shooting2. relaxation

52 / 74

shooting

53 / 74

relaxation

54 / 74

pure shooting

I let n1,n2 be the number of conditions posed atx1 = xs and x2 = xf

I → at x1 we can chose n2 y ’s at willI consider this a vector V of dimension n2 so that

yi(x1) = yi(x1; V)

I we can then integrate the ODE to x2

55 / 74

pure shooting

I define an error vector F that measures the deviationof the computed n2 BCs

I this vector is a function of V!I goal:

~F (V) = 0

I → system of (non-linear) equations!I → e.g., use Newton method

56 / 74

pure shooting

I Jacobian forJδV = −F

can (usually!) not be computed analyticallyI → approximate by numerical Jacobian, e.g.,

∂Fi

∂Vj≈ Fi(Vj + ∆Vj)− Fi(Vj)

∆Vj

57 / 74

pure shooting

I caution: ∆Vj determination!I shooting requires n2 + 1 integrations of N ODEs per

iterationI linear ODEs→ only one cycle required

58 / 74

shooting to a fitting point

I pure shooting can cause troubleI e.g., for some estimates the solution may be totally offI or very sensitive to guessesI Newton method may not converge etc.I related problems: singular points at x1 and/or x2

I shooting to a fitting point:I start from x1 and x2 and integrate to an intermediate

point inside [x1, x2]

I → N independent variables

59 / 74

shooting to a fitting point

I at the fitting point xm we have

yi(xm; V1) = yi(xm,V2)

I V1: the n2 guesses at x1

I V2: the n1 guesses at x2

I → use N dimensional Newton method to find solution

60 / 74

relaxation methods

I basic idea:I convert the ODE into an FDE by discretizationI FDE = finite difference equationI needs a ’mesh’ of points over the integration interval

61 / 74

relaxation methods

I example: write ODE

dydx

= g(x , y)

I possible FDE discretization:

yk − yk−1 − (xk − xk−1)g(xk−1/2, yk−1/2) = 0

I k mesh points xk and results yk

I xk−1/2 = (xk + xk−1)/2 etc.

62 / 74

relaxation methods

I → many possible ways to discretize!!I N equations with M mesh points→ N ×M unknownsI start with a guess and iterate to improveI → relax to the solutionI classical iteration scheme: Newton (did you guess?)I produces matrix equation of the order NMI than can be huge!I but matrix has special form→ solution easier

63 / 74

relaxation methods

I consider problem from above (shooting)I define mesh of M x points x1 to xM

I first and last correspond to intervalI yk : vector of the N dependent vars at point xk

I FDE may look like

Ek (yk ,yk−1) = yk−yk−1−(xk−xk−1)gk (xk−1/2,yk−1/2) = 0

I backward differenceI forward differencing also possible!

64 / 74

relaxation methods

I N equations for 2N variables (points k and k − 1)I M − 1 of those→ (M − 1)N equationsI remaining N from the BCs at x1 and xM !I first BC has n1 non-zero components:

E1 = B(x1,y1)

I best for later: last n1 are non-zeroI similar: at xM we have the first n2 non-zeros:

EM+1 = C(xM ,yM)

65 / 74

relaxation methods

I in general: non-linear systemI generate MN guesses yj,k and use Newton’s method

to compute corrections ∆yj,k viaI interior points:

Ek (yk ,yk−1) +∑

n

∂Ek

∂yn,k−1∆yn,k−1 +

∑n

∂Ek

∂yn,k∆yn,k = 0

I similar on the boundariesI example: N = 5 variables, M = 4 mesh points, 3 BCs

at first point, 2 at the last point

66 / 74

relaxation matrix

67 / 74

relaxation methods

I block matrix (band structure)I can be solved efficiently!I → iterate until corrections are small enough for your

requirementsI use analytic change of variables to deal with difficult

y ’sI example: implicit non-linear EOSs etc.

68 / 74

mesh point allocation

I first idea: distribute points uniformly over intervalI then double number of pointsI → estimate truncation errorI dynamically allocate mesh points?I consider x as a dependent variableI define q so that q = 1 is the first mesh point and

q = M the last

69 / 74

mesh point allocation

I ∆q = 1 is the difference between two mesh pointsI transform to q →

dydq

= gdxdq

I FDE version→

yk − yk−1 −12

[(g

dxdq

)k

+

(g

dxdq

)k−1

]= 0

I important: dx/dq goes with g

70 / 74

mesh point allocation

I dq/dx is mesh point densityI this density is related to the change in y(x)

I make up formula that is proportional to dq/dxI guarantee that integral of mesh density is M?

71 / 74

mesh point allocation

I → use a linear function Q(q) to adjust value ofintegral as required:

dQdq

= ΨdΨ

dq= 0

I Ψ is an intermediate variable that is there to alloweasy prescription of actual mesh density

72 / 74

mesh point allocation

I for exampledQdx

=dQdq

dqdx≡ φ(x)

I writedxdq

=Ψ

φ(x)

I this adds 3 ODEs and we need to prescribe φ(x)

73 / 74

mesh point allocation

I example: uniform spacing

dQdx

= φ(x) =1∆

I more complicated:

dQdx

= φ(x) =1∆

+

∣∣∣∣dy/dxyδ

∣∣∣∣I → log change in y of δ as ’attractive’ as a change in x

of ∆

74 / 74