lecture6: truncationerror,consistency,and...

Lecture 6:Truncation Error, Consistency, and

Convergence

J.K. [email protected]

WI3097TUDelft Institute of Applied Mathematics

Delft University of Technology

8 October 2012

() Truncation Error, Consistency, and Convergence 8 October 2012 1 / 54

Outline

1 ReviewSingle Step MethodsStability

IVP StabilityAmplification factor

2 Local Truncation Error & Consistency

3 Global truncation error & Convergence


Single-step methodsSummary

We have 4 schemes to solve

y ′(t) = f (t, y(t))

y(t0) = y0.

Forward Euler: wn+1 = wn + hf (tn,wn)

Backward Euler: wn+1 = wn + hf (tn+1,wn+1)

Trapezoidal method:

wn+1 = wn +h2 (f (tn,wn) + f (tn+1,wn+1))

Modified Euler:

wn+1 = wn + hf (tn,wn)

wn+1 = wn +h2 (f (tn,wn) + f (tn+1, wn+1))


Stability

Lax Equivalence Theorem: For a linear scheme,

Stability + Consistency = Convergence

Stability is how much small perturbations affect the solution.For this we have to look at

Stability of the initial value problem (IVP)Stability of the numerical method


StabilityIVP Stability

For the stability of the initial value problem (IVP), we consider

ε(t) = y(t)︸︷︷︸perturbed problem

−y(t)

There are three possibilities:1 It is unstable if limt→∞ |ε(t)| → ∞2 It is stable if limt→∞ |ε(t)| <∞ (finite)3 It is unconditionally stable if limt→∞ |ε(t)| → 0


StabilityIVP Stability

For the test equation

y ′ = λyy(0) = y0, (1)

limt→∞ |ε(t)| → 0 equates to λ < 0.


StabilityNumerical Stability

For numerical stability, we must consider the amplificationfactor, Q(hλ) in

wn+1 = Q(hλ)wn = (Q(hλ))n+1y0.

The possiblities are:1 The method is unstable if limh→0 |Q(hλ)| > 1.2 The method is stable if limh→0 |Q(hλ)| ≤ 1.

We use these conditions to choose an appropriate bound on h.


StabilityAmplification factor

wn+1 = Q(hλ)wn

The amplification factor for our methods is given by:

Forward Euler: Q(hλ) = 1 + hλ, stable for 0 < h < 2|λ| .

Backward Euler: Q(hλ) = 11−hλ , stable for 0 < h.

Trapezoidal method: Q(hλ) =1+ h

2λ

1− h2λ, stable for 0 < h.

modified Euler: Q(hλ) = 1 + hλ+ 12(hλ)2, stable for

0 < h < 2|λ| .


Local Truncation Error

Last time we asked: How good are these methods?

For this, we need convergence.But, convergence is difficult to prove directly.

The Lax equivalence theorem says that we can instead provestability and consistency.


Stability

Last time, we covered stability.

Stability is given by the amplification factor,

wn+1 = Q(hλ)wn.

For this, we choose h so that |Q(hλ)| ≤ 1.



For consistency, we will need the local truncation error. Butfirst we need a few definitions:

yn+1 = exact solution at time tn+1.

zn+1 = Q(hλ)yn = approximation applied to the exactsolution at time tn.wn+1 = (Q(hλ))n+1y0 = approximation applied n + 1times.



Definition ( Local truncation error)The local truncation error at time step n + 1 is given by

τn+1(h) =yn+1 − zn+1

h .

Notice, it is the difference between the exact solution and theapproximation applied to the exact solution at time tn.


Consistency

Definition (Consistent)A numerical method is consistent if

limh→0

τn+1(h) = 0.

Consistent ↔We can replicate the results.Solution of the numerical method converges to the exactsolution of the IVP.yn+1 − zn+1 → 0 faster than h→ 0.

By definition, this means that we need to examine thetruncation error.


Local truncation error

Consider again the test equation:

y ′ = λy ,y(0) = y0

which has solution y(t) = y0eλt .

The solution at tn+1 is

yn+1 = y(tn+1) = y0eλtn+1 = eλ(n+1)hy0 = eλhyn = yn+1.



The numerical method is given by

wn+1 = Q(hλ)wn.

Applied to the exact solution it is

zn+1 = Q(hλ)yn.

The local truncation error is then

τn+1(h) =yn+1 − zn+1

h =eλhyn − Q(hλ)yn

h =(eλh − Q(hλ))yn

h .



This means that the truncation error is determined by

eλh − Q(λh).

For this, we need two things:1 The Taylor series for eλh

eλh = 1+hλ+12!

(hλ)2+13!

(hλ)3+· · · =r−1∑i=0

1i! (hλ)i+O(hr )

2 Q(λh), which is given by the numerical method.


Local truncation errorForward Euler

The amplification factor for Forward Euler is given by

Q(hλ) = 1 + hλ.

Therefore, the local truncation error is

τn+1(h) =(eλh − Q(λh))yn

h

=1h

(1 + hλ+

12!

(hλ)2 +O(h3)

)︸︷︷︸

Taylor series for eλh

− (1 + hλ)︸︷︷︸Amplification factor

yn=

1h

( 12!

(hλ)2 +O(h3)

)yn

=

( 12!hλ2 +O(h2)

)yn ⇒ τn+1(h) = O(h)


Forward Euler

Euler forward,wn+1 = wn + hF (tn,wn),

Is an explicit method.Has amplification factor Q(hλ) = 1 + hλ.Is stable for 0 < h < 2

|λ| .

Has local truncation error of order one.Stability + Consistency = Convergence. It is a first ordermethod.


Consistency

Let us do the same for the other methods.

Convergence is difficult to prove.Lax theorem says that we can instead prove stability andconsistency.Stability is given by ensuring the amplification factor is lessthan one.

|Q(hλ)| ≤ 1.

For consistency, we need the local truncation error tovanish as the step size becomes smaller.

limh→0

τn+1(h) = limh→0

1h (yn+1 − zn+1).


Local truncation errorModified Euler

For the modified Euler, the amplification factor is given by

Q(hλ) = 1 + hλ+(hλ)2

2 .

It is stable for 0 < h < 2|λ| .

For consistency, we examine the local truncation error:


h

=1h

[(1 + hλ+

12!

(hλ)2 +13!

(hλ)3 +O(h4)

)..

..−(1 + hλ+

(hλ)2

2

)]yn

=1h

( 13!

(hλ)3 +O(h4)

)yn

=(13!h2λ3 +O(h3))yn ⇒ τn+1(h) = O(h2)


Modified Euler

Modified Euler is given by

wn+1 = wn + hF (tn,wn),

wn+1 = wn +h2 (F (tn,wn) + F (tn+1, wn+1)) (2)

Is an explicit method.Has amplification factor Q(hλ) = 1 + hλ+ (hλ)2

2 .

Is stable for 0 < h < 2|λ| .

Has local truncation error of order two.Stability + Consistency = Convergence. It is a secondorder method.


Local truncation errorImplicit methods

For implicit methods we generally have to work a bit harder.

Amplification factor is not in polynomial form.Have to put in polynomial form.⇒ Requires Taylor expansion.


Local truncation errorBackward Euler

For Backward Euler, the amplification factor is given by

Q(hλ) =1

1− hλ.

Before we plug this into the formula for the local truncationerror, we must perform a Taylor expansion:

11− hλ = 1 + hλ+ (hλ)2 + (hλ)3 +O(h4),

when |hλ| < 1.


Local truncation errorBackward Euler

Plugging in 11−hλ = 1 + hλ+ (hλ)2 + (hλ)3 +O(h4) into the

formula for local truncation error gives


h

=1h

[(1 + hλ+

12!

(hλ)2 +O(h3)

)..

..− (1 + hλ+ (hλ)2 +O(h3))]yn

=1h

(−12(hλ)2 +O(h3)

)yn

=

(−12hλ

2 +O(h2)

)yn ⇒ τn+1(h) = O(h)


Backward Euler

Euler backward,

wn+1 = wn + hF (tn+1,wn+1),

Is an implicit method.Has amplification factor Q(hλ) = 1

1−hλ .

Is stable for h > 0.Has local truncation error of order one.Stability + Consistency = Convergence. It is a first ordermethod.


Local truncation errorTrapezoidal method

For the Trapezoidal method, the amplification factor,

Q(hλ) =1 + hλ

21− hλ

2

also has to be expanded into a Taylor series:

1 + hλ2

1− hλ2

=

(1 +

hλ2

)(1 +

hλ2 + (

hλ2 )2 + (

hλ2 )3 +O(h4)

)= 1 + hλ+

12(hλ)2 +

14(hλ)3 +O(h4)

when∣∣∣hλ2 ∣∣∣ < 1.


Local truncation errorTrapezoidal method

Plugging in 1+ hλ2

1− hλ2

= 1 + hλ+ 12(hλ)2 + 1

4(hλ)3 +O(h4) into theformula for local truncation error gives


h

=1h

[(1 + hλ+

12!

(hλ)2 + +13!

(hλ)3 +O(h4)

)..

..− (1 + hλ+12(hλ)2 +

14(hλ)3 +O(h4))

]yn

=1h

(−16(hλ)3 +O(h4)

)yn

=

(−16h

2λ3 +O(h3)

)yn ⇒ τn+1(h) = O(h2)


Trapezoidal method

Trapezoidal method,

wn+1 = wn +h2 (F (tn,wn) + F (tn+1,wn+1)),

Uses the average of Euler forward and Euler backward.Is an implicit method.

Has amplification factor Q(hλ) =1+ hλ

21− hλ

2.

Is stable for h > 0.Has local truncation error of order two.Stability + Consistency = Convergence. It is a secondorder method.


Local truncation errorTo find the local truncation error

τn+1(h) =1h (yn+1 − zn+1)

where zn+1 = Q(hλ)yn.1 Write the exact solution in terms of yn : yn+1 = ynehλ.

τn+1(h) =ynh (ehλ − Q(hλ))

2 Expand ehλ in a Taylor series.3 Expand the amplification factor, Q(hλ) in a Taylor series

(if necessary).4 Simplify the quantity ehλ − Q(hλ)

5 Multiply by ynh .


Local truncation errorObserve: All methods can be written as

wn+1 = Q(hλ)wn.

Explicit methods can be written as

zn+1 = yn + hΦ(zn, tn, h).

This gives a truncation error of:

τn+1(h) =1h [yn+1 − zn+1] =

1h [yn+1 − (yn + hΦ(zn, tn, h))]

=1h (yn+1 − yn)︸︷︷︸

All explicit methods

− Φ(zn, tn, h)︸︷︷︸Depends upon method



For explicit methods, the local truncation error can be writtenas

τn+1(h) =1h (yn+1 − yn)− Φ(zn, tn, h).

We can simplify this further. Expand yn+1 around yn :

yn+1 = yn + h (yn)′︸︷︷︸=f (tn,yn)

+h22 (yn)′′︸︷︷︸

=(f (tn,yn))′=(ft+ffy )n

+O(h3)

= yn + hf (tn, yn) +h22 (ft + ffy )n +O(h3)



Then

τn+1(h) =1h (yn+1 − yn)− Φ(zn, tn, h)

=1h

((yn + hf (tn, yn) +

h22 (ft + ffy )n +O(h3))− yn

)− Φ(zn, tn, h)

= f (tn, yn) +h2 (ft + ffy )n +O(h2)− Φ(zn, tn, h)

Euler Forward: Φ(zn, tn, h) = f (tn, yn)

Modified Euler: Φ(zn, tn, h) = f (tn, yn) + h(ft + ffy )


Local truncation errorFinding Φ(zn, tn, h) for modified Euler:

zn+1 = yn + hf (tn, yn)

zn+1 = yn +h2 (f (tn, yn) + f (tn+1, zn+1))

We need to expand f (tn+1, zn+1) in a Taylor series around(tn, yn).

f (tn+1, zn+1) =f (tn, yn) + ft(tn, yn) (tn+1 − tn)︸︷︷︸=h

+fy (tn, yn) (zn+1 − yn)︸︷︷︸=hf (tn,yn)

+(ftt(tn, yn)(tn+1 − tn)2 ..

..+ 2fty (tn+1 − tn)(zn+1 − yn) + fyy (zn+1 − yn)2)

+O(h3)

=f (tn, yn) + h(ft(tn, yn) + f (tn, yn)fy (tn, yn)) +O(h2)


Global truncation error

The local truncation error,

τn+1(h) =1h (yn+1 − zn+1),

says how well the method performs over one time-step. It tellsus whether the method is consistent.

What is the cumulative effect?



For that we need:

Definition (Global truncation error)The global truncation error of a scheme is defined as

en+1 = yn+1 − wn+1,

where yn+1 = y(tn+1) is the exact solution at tn+1 andwn+1 = (Q(hλ))n+1y0 is the approximation applied to the initialcondition n + 1 times.

This is the cumulative effect of the errors from theapproximation over the entire interval.



Definition (Convergence)A numerical method is convergent if

limh→0

en+1 = 0.

The global truncation error goes to zero as we increase thenumber of subintervals.The local & global truncation errors are of the same order.


Convergence

We now have the three elements in the Lax EquivalenceTheorem: For a linear scheme,


Proof is in the book...


Higher Order Methods

The methods we have discussed are all 1st or 2nd ordermethods. Sometimes, we would like a higher order method.

When we require high accuracy.When we must solve the equation over long periods intime.


Higher Order Methods4th−order Runge-Kutta (RK4)

One higher-order method is the family of Runge-Kutta methods.

The 4th−order Runge-Kutta scheme is:

k1 = hf (tn,wn)

k2 = hf (tn +h2 ,wn +

k12 )

k3 = hf (tn +h2 ,wn +

k22 )

k4 = hf (tn + h,wn + k3)

RK4 is the average of these slopes

wn+1 = wn +16 (k1 + 2(k2 + k3) + k4)


Higher Order Methods4th−order Runge-Kutta (RK4)

RK4:Uses an average of different slopes.This allow us to avoid taking higher derivativesThis is a 4th−order method.The amplification factor is

Q(hλ) = 1 + hλ+12(hλ)2 +

13!

(hλ)3 +14!

(hλ)4

The stability region is 0 < h < 2.8|λ|

.


Example

ExampleConsider the numerical scheme given by

u? = un + βhf (tn, un)

un+1 = u? + (1− β)hf (tn + βh, u?)

1 Show the truncation error is O(h) for each value of β.Show that there does not exist a β for which the scheme isO(h2).

2 What is the amplification factor?3 Consider the nonlinear equation y ′ = 2y − 4y2 and takeβ = 1

2 . Determine the maximal step size such that themethod is stable in the neighbourhood of y = 1

2 .


ExampleTruncation error

The numerical method applied to the exact solution, yn, isgiven by

z? = yn + βhf (tn, yn)

zn+1 = z? + (1− β)hf (tn + βh, z?)

We first need to expand f (tn + βh, z?) in a Taylor series around(tn, yn) :

f (tn + βh, z?) =f (tn, yn) + (ft)n(tn + βh − tn) + (fy )n(z? − yn)

+12β

2h2(ftt)n(tn + βh − tn)2 +12(fyy )n(z? − yn)2

+ βh(fty )n(tn + βh − tn)(z? − yn) +O(h3)



Substituting z? = yn + βhf (tn, yn) into the expansion, we have

f (tn + βh, z?) = f (tn, yn) + βh((ft)n + (f fy )n) +O(h2).

Now, we can consider zn+1 :

zn+1 =z? + (1− β)hf (tn + βh, z?)

=yn + βhf (tn, yn)

+ (1− β)h(f (tn, yn) + βh((ft)n + (f fy )n) +O(h2))

=yn + βhf (tn, yn)

+ h(f (tn, yn) + βh(ft + f fy )n +O(h2))

−βh(f (tn, yn) + βh(ft + f fy )n +O(h2))



Therefore we have that

zn+1 = yn+hf (tn, yn)+βh2(ft+f fy )n+(βh)2(ft+f fy )n+O(h3)

We also need

yn+1 = yn + h (yn)′︸︷︷︸=f (tn,yn)

+12h

2 (yn)′︸︷︷︸=(ft+ffy )n

′ +O(h3)

= yn + hf (tn, yn) +12h

2(ft + ffy )n +O(h3) = yn+1



Which gives the local truncation error as

τn+1(h) =1h (yn+1 − zn+1)

=1h

((yn + hf (tn, yn) +

12h

2(ft + ffy )n +O(h3)) ..

..− (yn + hf (tn, yn) + βh2(ft + f fy )n + . . .))

=12h(ft + ffy )n − βh(ft + f fy )n + . . .

=O(h)

The local truncation error is O(h) for any value of β.


ExampleAmplification factor

To determine the amplification factor, we apply the numericalmethod to the test equation,

y ′ = λy , y(0) = y0, λ < 0.

This gives

u? = un + βhλunun+1 = u? + (1− β)hλu?


ExampleAmplification factor

Simplifying,

un+1 = u? + (1− β)hλu?

= (un + βhλun) + (1− β)hλ(un + βhλun)

= (1 +��*βhλ+ hλ+ β(hλ)2 −��*βhλ− β2(hλ)2)un

= (1 + hλ+ β(1− β)(hλ)2)︸︷︷︸Q(hλ)

un

The Amplification factor is given by

Q(hλ) = (1 + hλ+ β(1− β)(hλ)2)


ExampleNonlinear equation

The nonlinear equation is

y ′ = f (y) = 2y − 4y2.

Notice, this is zero at y = 0 and y = 12 . If β = 1

2 , we want todetermine the maximum step-size, h, for which the scheme isstable. We proceed as follows

First, we linearize f (y) around y = 12 .



The linearization is given by

L(y) = f (12) + f ′(12)

(y − 1

2

).

This givesL(y) = 1− 2y .

Therefore, to consider the stablity of the nonlinear equation

y ′ = 2y − 4y2.

it is enough to consider the stability of the linear equation

y ′ = −2y

This is just our test equation with λ = −2.



For stability, we consider the amplification factor with β = 12

and λ = −2 :

Q(hλ) =1 + hλ+ β(1− β)(hλ)2

=1−2h +12

(1− 1

2

)(−2h)2

=1− 2h + h2

The method is stable if

|Q(hλ)| = |1− 2h + h2| ≤ 1.



|Q(hλ)| = |1− 2h + h2| ≤ 1

This means

−1 ≤ 1− 2h + h2 ≤ 1−1 ≤ (h − 1)2 ≤ 1−2 ≤ h(h − 2) ≤ 0

Recall: h > 0. Thus:

0 < h ≤ 2 for stability.


Summary table

FE BE ME TM

Amplification factor 1 + hλ 1− hλ 1 + hλ+ (hλ)22

1+ hλ2

1+ hλ2

Type method Explicit Implicit Explicit ImplicitStability range 0 < h < 2

|λ| h > 0 0 < h < 2|λ| h > 0

Method order One One Two TwoΦ(zn, tn, h) fn fn + h(ft + ffy )n

FE = Forward Euler; BE = Backward Euler; ME ModifiedEuler; TM = Trapezoidal method


SummaryLax Equivalence Theorem: For a linear scheme,


Stability: given by amplification factor, choose h such that

|Q(hλ)| < 1.

Consistency: given by the local truncation error:

τn+1 =1h (yn+1 − zn+1).

Convergence: given by the global truncation error:

en+1 = yn+1 − wn+1.


Material addressed

1 ReviewSingle Step MethodsStability

IVP StabilityAmplification factor

2 Local Truncation Error &Consistency

3 Global truncation error &Convergence

Material in book:Chapter 6, Sections 1-7

Useful exercises: 1-5


lecture6: truncationerror,consistency,and...

Documents