Lecture 318.086
Info• PSet 1 will be posted this afternoon/evening (due in 2 weeks)
• Until Feb. 22: Decide for a computational project (50% grade) and submit proposal to me by email (content: see next slide)
• March 28: Submit short (max 1page) written mid-term project report
• Until Friday, May 6: Submit written final project report
• May 5, 10, 12: Presentation (in class): 12 minutes + 3 minutes questions/discussion
• Content of project: Free choice, as long as related to course contents/computational aspect(can also go beyond!). No recycling of existing/old projects!Implementation/testing/comparison of methods is ok
Term project• When submitting your project proposal (Feb. 22), use this template:
• Project title
• Project background: Does it relate to your work in another field (e.g. your thesis)? If yes, briefly outline the questions and goals of your work in the other field.
• Questions and Goals: Briefly describe the questions you wish to investigate in your project. What are your expectations?
• Plan: Which language do you plan to program in? Do you intend to use special software? Does your project work relate to the work of other people at MIT?
Plan for today• Runge-Kutta time integration
• Basic time integration algorithm
• 1-way wave equation
FD schemes for ODEs• Equation: u’ = f(u) with u(0) given
• First order accuracy (p=1): Euler methods
• Second order (p=2): global error O(Δt2), local error O(Δt3):
Trapezoidal implicit
Adams-Bashford explicit
Backward differences explicit
Runge-Kutta 2 explicit
Un+1 � Un
�t=
1
2[f(Un+1, tn+1) + f(Un, tn)]
Un+1 � Un
�t=
3
2f(Un, tn)�
1
2f(Un�1, tn�1)
Un+1 � Un
�t=
1
2[f(Un, tn) + f(Un +�tf(Un, tn), tn+1)]
Lecture: Stability & Accuracy
6.2 Finite Difference Methods 467
3un+1 - 4 u n + un-1 Backward differences/BDF2 2At = f (Un+l, tn+l) . (19)
What about stability? The trapezoidal method (15) is stable even for stiff equa- tions, when a is very negative: l - $a At (left side) will be larger than 1 + l a At (right side). (19) is even more stable and accurate. The Adams method (17) will be stable if At is small enough, but there is always a limit on At for explicit systems.
Here is a quick way to find the stability limit -a& 5 C in (17) when a is real. The limit occurs when the growth factor is exactly G = -1. Set Un+l = - 1 and Un = 1 and Un-l = - 1 i n (17 ) . Solve for a when f (u , t ) = au:
-2 3 Stability limit in (17) - - - 1 - a + - a gives a n t = -1. So C = 1 . (20)
a t 2 2
We now have three second-order methods (15)-(17)-(19), all definitely useful. The reader might suggest including both Un-l and fn-1 to increase the accuracy to third order. Sadly, this method is violently unstable (Problem 5). We may extend (19) by older values of U in backward differences, or extend (17) by older f (U). But including both U and f (U) for extreme accuracy produces instability for all At.
Multistep Methods: Explicit and Implicit
By using p earlier values of U , the accuracy can be increased to order p. Backward Euler has p = 1, and BDF2 in (19) has p = 2. Each VU is U(t) - U(t - At):
1 1 Backward differences (V + Z ~ 2 + . . + -VP) Un+1 = At f (Un+1, tn+l) . (21) P
MATLAB's stiff code odel5s varies from p = 1 to p = 5 depending on the local error. The alternative is to use older values of f (U, t) instead of U. Explicit comes first:
Adams- Bashforth Un+1 - Un = At(b1 fn + . . . + bp fn-,+I) (22) The table shows the numbers b up to p = 4, starting with Euler for p = 1.
The fourth-order method is often a good choice, although astronomers go above p = 8.
order of accuracy p = l p = 2 p = 3 p = 4
b l b2 b3 b4
1 312 -112
23/12 -16112 5/12 55/24 -59124 37/24 -9124
limit on -aAt for stability
2 1
611 1 3/10
constant c in error DE
5/ 12 112
318 2511720
FD schemes for ODEs• Runge-Kutta is actually a family of methods!
• The most famous is Runge-Kutta-4 (RK4), which has p=4:
RK4 explicit
(ode45 in matlab) with
k1 =1
2f(Un, tn)
k2 =1
2f(Un +�tk1, tn+1/2)
k3 =1
2f(Un +�tk2, tn+1/2)
k4 =1
2f(Un + 2�tk3, tn+1)
Un+1 � Un
�t=
1
3[k1 + 2k2 + 2k3 + k4]
Stability diagrams• So far, we obtained stability conditions such as (RK2):
• It makes sense (see lecture) to consider complex z, and draw the stability region in the complex plane
|1 + z +1
2z2| 1
6.2 Finite Difference Methods 469
Adams-Bashforth
5ii Adams-Moulton
Backward Differences Runae-Kutta
Figure 6.3: Stability regions IGI 5 1 for popular methods (stabi1ity.m on the cse site). Explicit methods are stable for a At inside the curves; implicit methods outside.
Runge-Kutta Methods
If evaluations of f (u, t ) are not too expensive, Runge-Kutta methods are highly competitive (and self-starting). Runge-Kutta of orders 4 and 5 is the basis for ode45. These are compound one-step methods, using Euler's Un + At fn inside f :
Simplified Runge- Kutta un+l-un 1 At = -[fn + f (Un + Atfn , tn+~)] . 2 (25)
You see the compounding of f . For u ' = au the growth factor G captures (At)2:
Comparing with the exact growth ea At, this confirms second-order accuracy. Stability hits a limit at aAt = -2 where G = 1. Now let a At = r be complex:
Stability limit for RK2 1 [GI = Il+x+-x21 = 1 for z = a + i b 2
The stability limit is a closed curve in the complex plane through x = a At = -2. Figure 6.3 shows all the numbers x (eigenvalues in the matrix case) at which IGI 5 1.
6.2 Finite Difference Methods 469
Adams-Bashforth
5ii Adams-Moulton
Backward Differences Runae-Kutta
Figure 6.3: Stability regions IGI 5 1 for popular methods (stabi1ity.m on the cse site). Explicit methods are stable for a At inside the curves; implicit methods outside.
Runge-Kutta Methods
If evaluations of f (u, t ) are not too expensive, Runge-Kutta methods are highly competitive (and self-starting). Runge-Kutta of orders 4 and 5 is the basis for ode45. These are compound one-step methods, using Euler's Un + At fn inside f :
Simplified Runge- Kutta un+l-un 1 At = -[fn + f (Un + Atfn , tn+~)] . 2 (25)
You see the compounding of f . For u ' = au the growth factor G captures (At)2:
Comparing with the exact growth ea At, this confirms second-order accuracy. Stability hits a limit at aAt = -2 where G = 1. Now let a At = r be complex:
Stability limit for RK2 1 [GI = Il+x+-x21 = 1 for z = a + i b 2
The stability limit is a closed curve in the complex plane through x = a At = -2. Figure 6.3 shows all the numbers x (eigenvalues in the matrix case) at which IGI 5 1.
(z = a Δt for u’=au)
Basic ODE time integration• Given: RHS f(u,t), initial conditions u(0)
• Solve u’(t) = f(u,t):
Nsteps=1000; %Perform 1000 integration steps dt = 0.1;u = zeros(Nsteps+1); % Preallocate space for solution u % Set initial conditions: u(1)=u0; t = 0;
% Forward Euler: for i=1:1000
u(i+1) = u(i) + dt*f(u(i),t); t=t+dt;
end% Plot solution u etc.
If you use integrator that requires U from past: Start doing a single Forward Euler step to get the first 2 values of U, then use your integrator of choice!
1-way wave equation
• So far we only considered ODE’s: u’ = f(u,t) and we only considered methods for the time integration
• Now: PDEs! Simplest example: 1-way wave equation:
• We need to use “good” time integration and good approximation of spatial derivatives.
ut
= cux
1-way wave equation• Properties of :u
t
= cux
• Conservative equation (modes neither grow nor shrink)
lecture
• c is the speed by which the wave travels
• wave does not change its form
• Solution to initial condition : u(x, 0) = e
ikx
u(x, t) = e
ik(x+ct)
lecture
Spatial finite difference (FD) schemes• Common FD schemes for 1D ux:
u(x)
x
Forward diff, O(Δx):
Backw. diff, O(Δx):i
Central diff, O(Δx2):
Lecture: accuracyBut this is only spatial accuracy! How does it affect overall accuracy?
FD for the 1-way wave equation
Upwind/forward
differences
Lax-Wendroff
Lax-Friedrich
• Now: How to do time and space discretization in optimal way?
U(x, t+�t)� U(x, t)
�t
= c
U(x+�x, t)� U(x, t)
�x
U(x, t+�t)� U(x, t)
�t
= c
U(x+�x, t)� U(x��x, t)
2�x
+�t
2c
2U(x+�x, t)� 2U(x, t) + U(x��x, t)
�x
2
U(x, t+�t)� 12 (U(x+�x, t) + U(x��x, t))
�t
= c
U(x+�x, t)� U(x��x, t)
2�x
• Multiplying by △t gives various terms on RHS with factor r = c
�t
�x
CFL condition for stability: r≤1 (lecture)
until here
say next time why r<1 is stability criterium
Courant-Friedrichs-Lewy condition
• The CFL condition is necessary but not sufficient for stability! r = c
�t
�x
1
• Sufficient criteria for stability: Neumann stability analysis
• Idea: Remember, wave equation has solution in terms of exponentials: => We found |G|=1 analytically, but how does this look for discrete approximations? => Will tell us stability for each mode k separately
u(x, t) = G(k, t)eikx
Neumann stab. analysis for upwing and Lax-Friedrichs: Lecture
0 r 1• We find: Upwind:Lax-Friedrichs: �1 r 1
Lecture 418.086
Last time
• 1-way wave equation (also called advection equation)
• CFL criterium for stability
• von Neumann stability analysis
• solution in terms of single modes:
r = c
�t
�x
1
ut
= cux
u(x, t) = G(k, t)eikx
• |G|=1 analytically• |G|<1 for stability of numerical scheme
FD for 1-way wave eq: summary
Upwind differences
Lax-Wendroff
Lax-Friedrich
U(x, t+�t)� U(x, t)
�t
= c
U(x+�x, t)� U(x, t)
�x
U(x, t+�t)� U(x, t)
�t
= c
U(x+�x, t)� U(x��x, t)
2�x
+�t
2c
2U(x+�x, t)� 2U(x, t) + U(x��x, t)
�x
2
U(x, t+�t)� 12 (U(x+�x, t) + U(x��x, t))
�t
= c
U(x+�x, t)� U(x��x, t)
2�x
O(Δt)
O(Δt2)
O(Δt) !
0 r 1
�1 r 1
�1 r 1
r = c
�t
�x
Lecture
Notation• We have seen different schemes for 1-way wave equation.
The schemes are often called stencils and depicted like this:
480 Chapter 6 Initial Value Problems
Lax-Friedrichs G matches the next term in the exact growth factor only if r2 = 1:
GLF = cos kAx + ir sin kAx = 1 + irkAx - AX)^ + . - - (22) 1 2 2 Gexact = e = 1 + irk Ax + 2i r (k AX)^ + - - -
In the exceptional cases r = 1 and r = -1, G agrees with Gexact. Staying exactly on the characteristic line, UJ,,+1 matches the true u ( j A x , t + At) . For r2 < 1, Lax- Friedrichs has an important advantage and disadvantage:
U,,,+1 is a positive combination of old values. But accuracy is only first-order.
4. Lax-Wendroff is stable for -1 < r 5 1. The LW difference equation combines U,,, and Uj-l,n and Uj+1,, to compute U,,n+l:
Lax- Wendroff 1 zk Ax 1 -zkAx G = (1 - r2) + -(r2 + r)e + -(r2 - r)e . 2 2 (23)
This is G = 1 - r2 + r2 cos kAx + ir sin kAx. At the dangerous frequency kAx = T ,
the growth factor is 1 - 2r2. That stays above -1 if r2 5 1. Problem 5 shows that IGI < 1 for every kAx. Lax-Wendroff is stable when-
ever the CFL condition r 2 < - 1 is satisfied. The wave can go either way (or both ways) since c can be negative. LW is the most accurate of these five methods.
upwind wrong way centered Lax-Friedrichs Lax-Wendroff stable unstable unstable stable stable i f r L 1 all At all At if Irl 5 1 if Irl 5 1
5. The centered methods of maximum accuracy are stable for -1 < r < 1. Lax-Wendroff uses three values at time level n for accuracy two. For every even p = 2q, there are p+ 1 coefficients a_,, . . . , a, so that the difference equation U,,n+l = C amU,+,,, has accuracy p. Matching G = C ameimkAx with the exact factor e - - eikrAx gives p + 1 equations C amm& rj for the a's.
Using all those values, the CFL condition allows the possibility of stability out to -q 5 c A t lAx 5 q. The actual requirement is -1 5 r 5 1.
Lax-Wendroff
Lax-Friedrich
U(x, t+�t)� U(x, t)
�t
= c
U(x+�x, t)� U(x��x, t)
2�x
+�t
2c
2U(x+�x, t)� 2U(x, t) + U(x��x, t)
�x
2
U(x, t+�t)� 12 (U(x+�x, t) + U(x��x, t))
�t
= c
U(x+�x, t)� U(x��x, t)
2�x
Real-world accuracyUpwind:
R. J. LeVeque — AMath 585–6 Notes 183
0 1 2 3 4 5 6−0.5
0
0.5
1Upwind solution at time 4
0 1 2 3 4 5 6−0.5
0
0.5
1Lax−Wendroff solution at time 4
Figure 13.4: Numerical solution using upwind (diffusive) and Lax-Wendroff (dispersive) methods.
13.6.2 Lax-Wendroff
If the same procedure is followed for the Lax-Wendroff method, we find that all O(k) terms drop out ofthe modified equation, as is expected since this method is second order accurate on the advection equa-tion. The modified equation obtained by retaining the O(k2) term and then replacing time derivativesby spatial derivatives is
vt + avx +16ah2
!1 −
"ak
h
#2$
vxxx = 0. (13.37)
The Lax-Wendroff method produces a third order accurate solution to this equation. This equationhas a very different character from (13.35). The vxxx term leads to dispersive behavior rather thandiffusion. This is clearly seen in Figure 13.4, where the Un
j computed with Lax-Wendroff are comparedto the true solution of the advection equation. The magnitude of the error is smaller than with theupwind method for a given set of k and h, since it is a higher order method, but the dispersive termleads to an oscillating solution and also a shift in the location of the main peak, a phase error.
The group velocity for wave number ξ under Lax-Wendroff is
cg = a − 12ah2
!1 −
"ak
h
#2$ξ2
which is less than a for all wave numbers. (The concept of group velocity is explained in Section 13.7.)As a result the numerical result can be expected to develop a train of oscillations behind the peak, withthe high wave numbers lagging farthest behind the correct location.
If we retain one more term in the modified equation for Lax-Wendroff, we would find that the U nj
are fourth order accurate solutions to an equation of the form
vt + avx +16ah2
!1 −
"ak
h
#2$
vxxx = −ϵvxxxx, (13.38)
Lax-Wendroff:
R. J. LeVeque — AMath 585–6 Notes 183
0 1 2 3 4 5 6−0.5
0
0.5
1Upwind solution at time 4
0 1 2 3 4 5 6−0.5
0
0.5
1Lax−Wendroff solution at time 4
Figure 13.4: Numerical solution using upwind (diffusive) and Lax-Wendroff (dispersive) methods.
13.6.2 Lax-Wendroff
If the same procedure is followed for the Lax-Wendroff method, we find that all O(k) terms drop out ofthe modified equation, as is expected since this method is second order accurate on the advection equa-tion. The modified equation obtained by retaining the O(k2) term and then replacing time derivativesby spatial derivatives is
vt + avx +16ah2
!1 −
"ak
h
#2$
vxxx = 0. (13.37)
The Lax-Wendroff method produces a third order accurate solution to this equation. This equationhas a very different character from (13.35). The vxxx term leads to dispersive behavior rather thandiffusion. This is clearly seen in Figure 13.4, where the Un
j computed with Lax-Wendroff are comparedto the true solution of the advection equation. The magnitude of the error is smaller than with theupwind method for a given set of k and h, since it is a higher order method, but the dispersive termleads to an oscillating solution and also a shift in the location of the main peak, a phase error.
The group velocity for wave number ξ under Lax-Wendroff is
cg = a − 12ah2
!1 −
"ak
h
#2$ξ2
which is less than a for all wave numbers. (The concept of group velocity is explained in Section 13.7.)As a result the numerical result can be expected to develop a train of oscillations behind the peak, withthe high wave numbers lagging farthest behind the correct location.
If we retain one more term in the modified equation for Lax-Wendroff, we would find that the U nj
are fourth order accurate solutions to an equation of the form
vt + avx +16ah2
!1 −
"ak
h
#2$
vxxx = −ϵvxxxx, (13.38)
Real-world accuracy478 Chapter 6 Initial Value Problems
1 2 -
1 .
08. Upwind 06.
04.
02.
0
Figure 6.6: Three approximations to a sharp signal show smearing and oscillation.
For an ideal difference equation, we only want dissipation very close to the shock. This can avoid oscillation without losing accuracy. A lot of thought has gone into high resolution met hods (Section 6.6), to capture shock waves cleanly.
Stability of the Four Finite Difference Methods
Accuracy requires G to stay close to the true eicbAt. Stability requires G to stay inside the unit circle. If IGI > 1 at frequency k, the solution Gnezkx will blow up.
We now check IGI < 1, in the four methods. CFL was only a necessary condition ! 1. Forward differences in space and time: AU/At = cAU/Ax (upwind). Equation (11) was G = 1 - r + reikAx. If the Courant number is 0 < r 5 1, then 1 - r and r will be positive. The triangle inequality gives IGI < 1:
Stability for 0 5 r < 1 zk Ax - IGl 5 11-rI+Ire I = 1 - r + r = l . (17)
This sufficient condition 0 5 c At/ Ax < 1 agrees with the CFL necessary condition U ( x , nAt) depends on the initial values between x and x + nAx. That domain of dependence must include the point x + c n At. (Otherwise, changing the initial value at the point x + c n A t would change the true solution u but not the approximation U.) Then 0 < c nAt < nAx means that 0 5 r 5 1.
Figure 6.7 shows G in the stable case r = and the unstable case r = % (when At is too large). As k varies, and eAAX goes around a unit circle, the complex number G = 1 - r + reikAx goes in a circle of radius r. The center is 1 - r. Always G = 1 at zero frequency (constant solution, no growth).
2. Forward difference in time, centered difference in space. This combination is never stable ! The shorthand U,,, will stand for U(jAx, nAt):
step-like initial condition:
Can we understand this behavior?
Modified equations
• see lecture notes
Real-world accuracyUpwind:
R. J. LeVeque — AMath 585–6 Notes 183
0 1 2 3 4 5 6−0.5
0
0.5
1Upwind solution at time 4
0 1 2 3 4 5 6−0.5
0
0.5
1Lax−Wendroff solution at time 4
Figure 13.4: Numerical solution using upwind (diffusive) and Lax-Wendroff (dispersive) methods.
13.6.2 Lax-Wendroff
If the same procedure is followed for the Lax-Wendroff method, we find that all O(k) terms drop out ofthe modified equation, as is expected since this method is second order accurate on the advection equa-tion. The modified equation obtained by retaining the O(k2) term and then replacing time derivativesby spatial derivatives is
vt + avx +16ah2
!1 −
"ak
h
#2$
vxxx = 0. (13.37)
The Lax-Wendroff method produces a third order accurate solution to this equation. This equationhas a very different character from (13.35). The vxxx term leads to dispersive behavior rather thandiffusion. This is clearly seen in Figure 13.4, where the Un
j computed with Lax-Wendroff are comparedto the true solution of the advection equation. The magnitude of the error is smaller than with theupwind method for a given set of k and h, since it is a higher order method, but the dispersive termleads to an oscillating solution and also a shift in the location of the main peak, a phase error.
The group velocity for wave number ξ under Lax-Wendroff is
cg = a − 12ah2
!1 −
"ak
h
#2$ξ2
which is less than a for all wave numbers. (The concept of group velocity is explained in Section 13.7.)As a result the numerical result can be expected to develop a train of oscillations behind the peak, withthe high wave numbers lagging farthest behind the correct location.
If we retain one more term in the modified equation for Lax-Wendroff, we would find that the U nj
are fourth order accurate solutions to an equation of the form
vt + avx +16ah2
!1 −
"ak
h
#2$
vxxx = −ϵvxxxx, (13.38)
Smeared out due to diffusive term in modified equations!
Real-world accuracyLax-Wendroff:
R. J. LeVeque — AMath 585–6 Notes 183
0 1 2 3 4 5 6−0.5
0
0.5
1Upwind solution at time 4
0 1 2 3 4 5 6−0.5
0
0.5
1Lax−Wendroff solution at time 4
Figure 13.4: Numerical solution using upwind (diffusive) and Lax-Wendroff (dispersive) methods.
13.6.2 Lax-Wendroff
If the same procedure is followed for the Lax-Wendroff method, we find that all O(k) terms drop out ofthe modified equation, as is expected since this method is second order accurate on the advection equa-tion. The modified equation obtained by retaining the O(k2) term and then replacing time derivativesby spatial derivatives is
vt + avx +16ah2
!1 −
"ak
h
#2$
vxxx = 0. (13.37)
The Lax-Wendroff method produces a third order accurate solution to this equation. This equationhas a very different character from (13.35). The vxxx term leads to dispersive behavior rather thandiffusion. This is clearly seen in Figure 13.4, where the Un
j computed with Lax-Wendroff are comparedto the true solution of the advection equation. The magnitude of the error is smaller than with theupwind method for a given set of k and h, since it is a higher order method, but the dispersive termleads to an oscillating solution and also a shift in the location of the main peak, a phase error.
The group velocity for wave number ξ under Lax-Wendroff is
cg = a − 12ah2
!1 −
"ak
h
#2$ξ2
which is less than a for all wave numbers. (The concept of group velocity is explained in Section 13.7.)As a result the numerical result can be expected to develop a train of oscillations behind the peak, withthe high wave numbers lagging farthest behind the correct location.
If we retain one more term in the modified equation for Lax-Wendroff, we would find that the U nj
are fourth order accurate solutions to an equation of the form
vt + avx +16ah2
!1 −
"ak
h
#2$
vxxx = −ϵvxxxx, (13.38)
Dispersion due to term vxxx in modified equations
High freq. modes travel slower (=> on the left of the envelope)
Phase vs. group velocity• Remember from physics:
R. J. LeVeque — AMath 585–6 Notes 187
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 0
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 0.4
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 0.8
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 1.2
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 1.6
Figure 13.6: The oscillatory wave packet satisfies the dispersive equation ut + aux + buxxx = 0. Alsoshown is a black dot, translating at the phase velocity cp(ξ0) and a Gaussian that is translating at thegroup velocity cg(ξ0).
phase velocity
group velocity
Dispersion in LW schemeR. J. LeVeque — AMath 585–6 Notes 187
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 0
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 0.4
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 0.8
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 1.2
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 1.6
Figure 13.6: The oscillatory wave packet satisfies the dispersive equation ut + aux + buxxx = 0. Alsoshown is a black dot, translating at the phase velocity cp(ξ0) and a Gaussian that is translating at thegroup velocity cg(ξ0).
R. J. LeVeque — AMath 585–6 Notes 187
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 0
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 0.4
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 0.8
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 1.2
−3 −2 −1 0 1 2 3−1
−0.5
0
0.5
1
time = 1.6
Figure 13.6: The oscillatory wave packet satisfies the dispersive equation ut + aux + buxxx = 0. Alsoshown is a black dot, translating at the phase velocity cp(ξ0) and a Gaussian that is translating at thegroup velocity cg(ξ0).