18.336|numerical methods for partial di erential equations ...plamen/18.336/math336.pdf1.3. von...
TRANSCRIPT
18.336—Numerical Methods for Partial Differential Equations,
Spring 2005
Plamen Koev
September 6, 2012
2
Contents
1 Hyperbolic PDEs 51.1 Consistency, Stability, Well-posedness, and Convergence . . . . . . . . . . . . . . . . . . . . . 51.2 Fourier Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.3 Von Neumann Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.4 The Leap-Frog Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.5 Dissipation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.6 Dispersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.7 Group Velocity and Propagation of Wave Packets . . . . . . . . . . . . . . . . . . . . . . . . . 151.8 Summary of Schemes for the Wave Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2 Parabolic equations 192.1 The Heat Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.2 The Du Fort–Frankel Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.3 The Convection-Diffusion Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.4 Summary of Schemes for the Heat Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3 Systems of PDEs in Higher Dimensions 273.1 The Equation ut +Aux = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.2 The Equation ut +Aux = Bu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.3 The Equation ut +Aux +Buy = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.4 The equation ut = b1uxx + b2uyy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.5 ADI methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.6 Boundary conditions for ADI methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4 Elliptic Equations 354.1 Steady-State Heat Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.2 Numerical methods for uxx + uyy = f . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.3 Jacobi, Gauss–Seidel, and SOR(ω) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3
4 CONTENTS
Chapter 1
Hyperbolic PDEs
We consider the hyperbolic equationut + aux = 0
for t ≥ 0 and initial condition u(0, x) = u0(x). The unique solution to this problem is given by
u(t, x) = u0(x− at),i.e., the solution is wave traveling right if a > 0 and left if a < 0.
We will show (a) how to generate schemes for its numerical solution, (b) verify that these schemes area good approximation to the differential equation (i.e., are consistent) and (c) that the numerical solutionconverges to the solution to the differential equation.
The idea in using finite differences to solve a PDE is to select a grid in time and space (with meshlengthsk and h, respectively) and to approximate the values u(mk, nh) for integer m,n. All all that follows u willdenote the exact solution to the PDE and
v(time)(space) = vnm ≈ u(mh, kn)
will denote the approximate finite difference solution.We will approximate derivatives of a function f as follows:
δ+f(x) =f(x+ h)− f(x)
hforward difference
δ−f(x) =f(x)− f(x− h)
hbackward difference
δ0f(x) =f(x+ h)− f(x− h)
2hcentered difference
For a grid function v = (. . . , v−2, v−1, v0, v1, v2, . . .) we have:
δ+vm =vm+1 − vm
hforward difference
δ−vm =vm − vm−1
hbackward difference
δ0vm =vm+1 − vm−1
2hcentered difference
1.1 Consistency, Stability, Well-posedness, and Convergence
Definition 1 (Consistency). We say that a finite difference scheme Pk,hv = f is consistent with the PDEPu = f of order (r, s) if for any smooth function φ
Pφ− Pk,hφ = O(kr, hs) (1.1)
5
6 CHAPTER 1. HYPERBOLIC PDES
To verify consistency expand φ in Taylor series and make sure (1.1) holds.
Definition 2 (L2 norm). For a function w = (. . . , w−2, w−1, w0, w1, w2, . . .) on a grid with step size h:
‖w‖ =
(h
∞∑m=−∞
|wm|2)1/2
For a function f on the real line:
‖f‖ =(∫ ∞−∞|f(x)|2dx
)1/2
Definition 3 (Stability). A finite one-step difference scheme Pk,hvnm = 0 for a first-order PDE is stable ifthere exist numbers k0 > 0 and h0 > 0 such that for any T > 0 there exists a constant CT such that
‖vn‖ ≤ CT ‖v0‖
for 0 ≤ nk ≤ T, 0 < h ≤ h0, 0 < k ≤ k0.
Definition 4 (Well-posedness). The initial value problem for the first-order PDE Pu = 0 is well-posed iffor any time T ≥ 0, there is a constant CT , such that any solution u(t, x) satisfies
‖u(t, x)‖ ≤ CT ‖u(0, x)‖, for 0 ≤ t ≤ T.
Definition 5 (Convergence). A one-step finite difference scheme approximating a PDE is convergent if forany solution to the PDE, u(t, x), and solution to the finite difference scheme vnm, such that v0
m → u(0, x) asmh→ x, we have vnm → u(t, x) as (nk,mh)→ (t, x) (as h, k → 0).
Theorem 1 (Lax). A consistent finite difference scheme for a PDE for which the initial value problem iswell-posed is convergent if and only if it is stable.
Theorem 2 (The Courant–Friedrichs–Lewy Condition). A necessary condition for stability of the explicitscheme for the hyperbolic equation ut + aux = 0:
vn+1m = αvnm−1 + βvnm + γvnm+1
with k/h = λ held constant is|aλ| ≤ 1.
Proof. The solution is u(t, x) = u0(x − at) and u(1, 0) = u0(−a). The finite difference scheme vn0 dependsonly on v0
m for |m| ≤ n. Therefore |hn| ≥ | − a|. Since kn = 1, we have n = 1/k and |n/k| ≥ |a| or|aλ| ≤ 1.
1.2 Fourier Analysis
Fourier Transform and Inversion formula
• For u defined on R
u(ω) =1√2π
∫ ∞−∞
e−iωxu(x)dx, u(x) =1√2π
∫ ∞−∞
eiωxu(ω)dω
• For a grid function v = (. . . , v−2, v−1, v0, v1, v2, . . .) with grid spacing h (here ξ ∈ [−π/h, π/h])
v(ξ) =1√2π
∞∑m=−∞
e−imhξvmh, vm =1√2π
∫ π/h
−π/heimhξ v(ξ)dξ
Theorem 3 (Parseval).‖u(x)‖ = ‖u(ω)‖, ‖v‖ = ‖v‖.
(where ‖v‖2 =∫ π/h−π/h |v(ξ)|2dξ.)
1.3. VON NEUMANN ANALYSIS 7
1.3 Von Neumann Analysis
Provides an uniform way of verifying if a finite difference scheme is stable.
Example 1. Let’s study the forward-time backward-space scheme
vn+1m − vnm
k+ a
vnm − vnm−1
h= 0.
Rewrite asvn+1m = (1− aλ)vnm + aλvnm−1,
where λ = k/h. Use the Fourier inversion formula
vnm =1√2π
∫ π/h
−π/heimhξ vn(ξ)dξ
and substitute to obtain
1√2π
∫ π/h
−π/heimhξ vn+1(ξ)︸ ︷︷ ︸
∗
dξ =1√2π
∫ π/h
−π/heimhξ [(1− aλ) + aλe−ihξ]vn(ξ)︸ ︷︷ ︸
∗∗
dξ.
The Fourier transform is unique, so (*) must equal (**):
vn+1(ξ) = [(1− aλ) + aλe−ihξ]vn(ξ).
Denote g(hξ) ≡ (1− aλ) + aλe−ihξ, called amplification factor. We have
vn(ξ) = g(hξ)vn−1(ξ) = . . . =(g(hξ)
)nv0(ξ).
Now
‖vn‖2 =∫ π/h
−π/h|vn(ξ)|2dξ =
∫ π/h
−π/h|g(hξ)|2n|v0(ξ)|2dξ.
Therefore ‖vn‖ ≤ ‖v0‖ (i.e., the scheme is stable), if |g(hξ)| ≤ 1. Write θ = hξ and evaluate |g(θ)|2
|g(θ)|2 = |(1− aλ) + aλe−ihξ|2
= (1− aλ+ aλ cos θ)2 + a2λ2 sin2 θ
= (1− 2aλ sin2 θ2 )2 + 4a2λ2 sin2 θ
2 cos2 θ2
= 1− 4aλ(1− aλ) sin2 θ2 .
Thus |g(θ)| ≤ 1 if 0 ≤ aλ ≤ 1. Then ‖vn‖ ≤ ‖v0‖ and the scheme is stable if 0 ≤ aλ ≤ 1.
Theorem 4 (Stability condition). A one-step finite difference scheme is stable if and only if there existpositive constants K,h0, k0 such that
|g(θ, k, h)| ≤ 1 +Kk
for all θ, 0 < k ≤ k0, 0 < h ≤ h0. If g is independent of k, then the condition is
|g(θ, k, h)| ≤ 1.
Proof. Assume g(θ, k, h) ≤ 1 +Kk for some K.
‖vn‖2 =∫ π/h
−π/h|g|2n|v0(ξ)|2dξ ≤ (1 +Kk)2n
∫ π/h
−π/h|v0(ξ)|2dξ ≤ (1 +Kk)2n‖v0‖2
8 CHAPTER 1. HYPERBOLIC PDES
Now nk ≤ T and (1 +Kk)n ≤ (1 +Kk)T/k ≤ eKT , meaning ‖vn‖ ≤ eKT ‖v0‖ and the scheme is stable.Conversely, assume that for any C there exists an interval [θ1, θ2] such that |g| ≥ 1+Ck for θ ∈ [θ1, θ2], h ∈
(0, h0], and k ∈ (0, k0]. Let
v0(ξ) =
0 if hξ 6∈ [θ1, θ2],√h(θ2 − θ1)−1 if hξ ∈ [θ1, θ2].
Now ‖v0‖ = 1 and
‖vn‖2 =∫ π/h
−π/h|g|2n|v0(ξ)|2dξ =
∫ θ2/h
θ1/h
|g|2n h
θ2 − θ1dξ ≥ (1 + Ck)2n ≥ 1
2e2TC‖v0‖2
for n near T/k. Therefore the scheme is unstable if C can be arbitrarily large. If g is independent of h andk, then |g| ≤ 1 +Kk must hold for any 0 < k ≤ k0, therefore |g| ≤ 1.
In practice to analyze a finite difference scheme we do not write integrals. Instead we replace vnm bygneimθ and solve for g.
Example 2. Forward-time centered space
vn+1m − vnm
k+ a
vnm+1 − vnm−1
2h= 0.
0 =gn+1eimθ − gneimθ
k+ a
gnei(m+1)θ − gneim−1θ
2h= gneimθ
(g − 1k
+ aeiθ − e−iθ
2h
).
So g(θ) = 1− iaλ sin θ, with λ = k/h. If λ is constant, then |g(θ)|2 = 1 + a2λ2 sin2 θ > 1 and the scheme isunstable.
Example 3. The Lax–Wendroff scheme:
vn+1m = vnm −
aλ
2(vnm+1 − vnm−1) +
a2λ2
2(vnm+1 − 2vnm + vnm−1),
so the amplification factor is:
g(θ) = 1− aλ
2(eiθ − e−iθ) +
a2λ2
2(eiθ − 2 + e−iθ)
= 1− iaλ sin θ − a2λ2(1− cos θ)= 1− 2a2λ2 sin2 θ
2 − iaλ sin θ
Thus
|g(θ)|2 = (1− 2a2λ2 sin2 θ2 )2 +
(2aλ sin θ
2 cos θ2)2
= 1− 4a2λ2(1− a2λ2) sin4 θ2
The scheme is stable only when |g(θ)| ≤ 1, i.e., when |aλ| ≤ 1.
Example 4. For the Crank–Nicolson scheme
vn+1m = vnm −
aλ
4(vn+1m+1 − v
n+1m−1 + vnm+1 − vnm−1)
we obtain
g(θ) =1− 1
2 iaλ sin θ1 + 1
2 iaλ sin θthus |g(θ)|2 =
1 + ( 12aλ sin θ)2
1 + ( 12aλ sin θ)2
= 1
so this scheme is unconditionally stable.
1.4. THE LEAP-FROG SCHEME 9
1.4 The Leap-Frog Scheme
In this section we prove that Leap-frog is stable if and only if |aλ| < 1. The scheme is
vn+1m − vn−1
m
2k+ a
vnm+1 − vnm−1
2h= 0, i.e., vn+1
m = vn−1m + aλ(vnm+1 − vnm−1).
Write δ0vm = vm+1−vm−12h . Then
[vn+1m
vnm
]=[−2kaδ0 1
1 0
]·[
vnmvn−1m
].
Fourier transform for vectors = Fourier transform in each component:
[vn+1m
vnm
]=
1√2π
∫ π/h
−π/heimhξ
[vn+1(ξ)vn(ξ)
]dξ,[
vn+1(ξ)vn(ξ)
]=
1√2π
∞∑m=−∞
e−imhξ[vn+1m
vnm
]h
and Parseval for vectors (| · | means the 2-norm for vectors or matrices so we can tell it apart from theL2-norm)
∥∥∥∥[ vn+1
vn
]∥∥∥∥2
= h
∞∑m=−∞
∣∣∣∣[ vn+1m
vnm
]∣∣∣∣2 =∫ π/h
−π/h
∣∣∣∣[ vn+1(ξ)vn(ξ)
]∣∣∣∣2 dξ =∥∥∥∥[ vn+1(ξ)
vn(ξ)
]∥∥∥∥2
.
Now Fourier of Leap-Frog:
1√2π
∫ π/h
−π/heimhξ
[vn+1(ξ)vn(ξ)
]dξ =
[−2kaδ0 1
1 0
]1√2π
∫ π/h
−π/heimhξ
[vn(ξ)vn−1(ξ)
]dξ
=1√2π
∫ π/h
−π/h
[−2kaδ0eimhξ vn(ξ) + eimhξ vn−1(ξ)
eimhξ vn(ξ)
]dξ
=1√2π
∫ π/h
−π/heimhξ
[−2ka e
ihξ−e−ihξ2h 1
1 0
] [vn(ξ)vn−1(ξ)
]dξ
=1√2π
∫ π/h
−π/heimhξ
[−2iaλ sin(hξ) 1
1 0
] [vn(ξ)vn−1(ξ)
]dξ
Therefore
[vn+1(ξ)vn(ξ)
]=[−2iaλ sin(hξ) 1
1 0
]︸ ︷︷ ︸
G(hξ)
[vn(ξ)vn−1(ξ)
]= G ·
[vn(ξ)vn−1(ξ)
]= . . . = Gn ·
[v1(ξ)v0(ξ)
].
10 CHAPTER 1. HYPERBOLIC PDES
Parseval now gives ∥∥∥∥[ vn+1
vn
]∥∥∥∥2
=∥∥∥∥[ vn+1(ξ)
vn(ξ)
]∥∥∥∥2
=∫ π/h
−π/h
∣∣∣∣(G(hξ))n ·[v1(ξ)v0(ξ)
]∣∣∣∣2 dξ≤
∫ π/h
−π/h|G(hξ)|n ·
∣∣∣∣[ v1(ξ)v0(ξ)
]∣∣∣∣2 dξ≤ max
|hξ|≤π|G(hξ)|2n
∥∥∥∥[ v1(ξ)v0(ξ)
]∥∥∥∥2
= max|hξ|≤π
|G(hξ)|2n∥∥∥∥[ v1
v0
]∥∥∥∥2
.
Remains to see when the 2-norm of G is bounded. Jordan form: G = TΛT−1 and Gn = TΛnT−1.Characteristic polynomial:
g2 + 2iaλ sin(hξ)g − 1 = 0
Λn bounded only if the roots (not to be confused with λ): |λ1,2| ≤ 1, but λ1λ2 = −1, so we must have|λ1| = |λ2| = 1. Eigenvalues (denote s ≡ sin(hξ) for short):
λ1,2 = −iaλs±√
1− (aλs)2.
If |aλ| > 1, then there exists ξ, s.t. |aλs| > 1 and both λ1 and λ2 are purely imaginary and distinct, so oneof them will be > 1 and the other < 1 in magnitude. So we must have |aλ| ≤ 1.
When |aλ| ≤ 1 we have |λ1,2|2 = (aλs)2 +1− (aλs)2 = 1. Therefore both λ1 and λ2 are on the unit circle.If λ1 6= λ2, then Jordan form of G is (exercise):
G =[
1 1−λ2 −λ1
] [λ1
λ2
] [ −λ1 −1λ2 1
]−λ1 + λ2
Exercise: |A| ≤ ‖A‖F = (∑i,j |aij |2)1/2 (Frobenius norm).
Now
|Gn| ≤ |TΛnT−1| ≤ |T | · 1 · |T−1| ≤ ‖T‖F ‖T−1‖F ≤√
4 ·√
4| − 2
√1− |aλ|2|
≤ 2√1− |aλ|2
is nicely bounded. Going back to Parseval∥∥∥∥[ vn+1
vn
]∥∥∥∥ ≤ √2
(1− |aλ|2)1/4
∥∥∥∥[ v1
v0
]∥∥∥∥and Leap-frog is stable.
Next case: λ1 = λ2. It occurs when sin(hξ) = ±1 and |aλ| = 1. Assume aλs = 1, (the −1 case isanalogous).
G =[−2i 1
1 0
]=[
1 1i 0
] [−i −i
−i
] [0 −i1 i
]= TΛT−1
A little unusual to write a Jordan block with −i in position (1, 2) but legal and, in this case, convenient.
Gn = T (−i)n[
1 10 1
]nT−1 = (−i)nT
[1 n0 1
]T−1,
1.5. DISSIPATION 11
i.e., G(±π/2) will blow up. You’d think that there may be cancellation, but no:
n =∣∣∣∣[ 1 n
0 1
]∣∣∣∣ = |inT−1GnT | ≤ |T | · |Gn| · |T |
and both |T | and |T−1| are nicely bounded by (say) 2, so |Gn| ≥ n/4 = T/(4k)→∞ as k → 0.The stability condition is therefore |aλ| < 1.
1.5 Dissipation
We would expect the wave equation to propagate the initial condition with a constant speed a, including allfrequencies that make up that initial condition.
Unfortunately the discrete nature of our data means that instead of the initial condition u0(x) we havea discrete version of it—vn0 .
The initial condition u0(x) is a superposition (in theory) of an infinite number of frequencies (thinkFourier expansion), whereas vn0 only inherits the frequencies ξ ∈ [−π/h, π/h]. All higher frequencies areignored by our discrete initial condition. Recall that the Fourier transform vn(ξ) of vn is only defined forξ ∈ [−π/h, π/h].
Obviously different frequencies are treated differently and we would like to get a better understanding ofthat treatment. Example is the best way to go here. Consider Lax–Friedrichs:
vn+1m − 1
2 (vnm+1 + vnm−1)k
+ avnm+1 − vnm−1
2h= 0,
equivalently,vn+1m = 1−aλ
2 vnm+1 + 1+aλ2 vnm−1.
Von Neumann analysis implies
g(hξ) = cos(hξ)− iaλ sin(hξ),|g(hξ)|2 = cos2(hξ) + (aλ)2 sin2(hξ).
Let θ = hξ as usual.We see that θ = 0 and θ = π are not dampened, but all other θ are. Let’s observe closely. Pick aλ to be
(say) 1/2. Then
vn+1m =
14vnm+1 +
34vnm−1
• θ = π/2. Then eimhξ = eimθ = eimπ/2 = . . . , 1, 0,−1, 0, 1, 0,−1, 0, 1, . . .
n = 4 1/16n = 3 1/8 0 −1/8n = 2 1/4 0 −1/4 0 1/4n = 1 1/2 0 −1/2 0 1/2 0 −1/2n = 0 1 0 −1 0 1 0 −1 0 1
• θ = π, we have eimhξ = eimθ = eimπ = . . . , 1,−1, 1,−1, 1, . . . = (−1)m.We can verify that vnm = (−1)m+n is a solution to Lax–Friedrichs, so θ = π is not dampened at all.
We don’t really expect good results for wildly oscillating solutions, so we can expect that the higherfrequencies will not be well-represented in our calculation. However it is unacceptable for higher frequenciesto be less dampened than the middle-range ones.
Another example. Look at Lax–Wendroff: |g|2 = 1− 4a2λ2(1− a2λ2) sin4 θ2 ≤ 1− const · sin4 θ
2 .This is very important—says that all frequencies, except ξ = 0 (then θ = 0) are decreasing and the
highest frequencies are suppressed the most. This is exactly what we want and will call schemes that havethis property dissipative.
12 CHAPTER 1. HYPERBOLIC PDES
Definition 6 (Dissipative Scheme). A scheme is dissipative of order 2r if
|g(θ)| ≤ 1− c · sin2r θ2 .
The reason we like dissipative schemes is that if we are not doing a good job with the high frequenciesanyway, why not kill them.
Remark 1. A dissipative scheme is always stable.
How can we make a non-dissipative scheme dissipative? This calls for another example. Crank–Nicolson,which is second order accurate.
vn+1m − vnm
k+ a
vn+1m+1 − v
n+1m−1 + vnm+1 − vnm−1
4h= 0
So adding a fourth derivative in there will not affect the order of accuracy of the approximation, since fourthderivatives get ignored anyway. When we do the Fourier analysis the fourth derivative will bring a sin4 θ
2 .
vn+1m − vnm
k+ a
vn+1m+1 − v
n+1m−1 + vnm+1 − vnm−1
4h+ Cε
vnm−2 − 4vnm−1 + 6vnm − 4vnm+1 + vnm+2
h4= 0
Now select C appropriately so that the fourth derivative only brings sin4 θ2 into the picture, without any
weird powers of k and h: C = h4
16k . Then after some simplification
g(θ) =1− ε sin4 θ
2 − iaλ2 sin θ
1 + iaλ2 sin θ
implying
|g(θ)|2 =1 +
(aλ2 sin θ
)2 − 2ε sin4 θ2 + ε2 sin8 θ
2
1 +(aλ2 sin θ
)2
= 1−ε sin4 θ
2 −
>0 for ε<1︷ ︸︸ ︷ε sin4 θ
2 (1− ε sin4 θ2 )
1 +(aλ2 sin θ
)2≤ 1−
ε sin4 θ2
1 +(aλ2 sin θ
)2 .If we now restrict |aλ| (say) ≤ 10, then 1 +
(aλ2 sin θ
)2 ≤ 26, and
|g|2 ≤ 1− ε
26sin4 θ
2 .
We want a bound on |g|, not |g|2:
|g|2 ≤ 1− ε
26sin4 θ
2 ≤ |g|2 ≤ 1− ε
26sin4 θ
2 +( ε
52
)2
sin8 θ2 =
(1− ε
52sin4 θ
2
)2
,
so|g| ≤ 1− ε
52sin4 θ
2 .
The scheme is all of a sudden dissipative of order 4 (since 2r = 4). Although Crank–Nicolson is stable forall aλ we cannot make it dissipative without restricting aλ.
The exact same trick works for Leap-frog.
1.6. DISPERSION 13
1.6 Dispersion
In this section we investigate whether in the numerical solution of ut + aux = 0 different frequencies travelwith the same speed a as they should. They, of course, do not and we will see that in fact travel with speedα(hξ) ≈ a.
Look for a solution tout + aux = 0, u(0, x) = f(x)
(which has a unique solution u(t, x) = f(x− at)) using separation of variables
u(t, x) = g(t)eixξ,
assuming u(0, x) = g(0)eixξ = eixξ =periodic wave (here we assume g(0) = 1). Then
ut + aux = g′(t)eixξ + ag(t)iξeixξ = (g′(t) + aiξg(t))eixξ = 0.
Since |eixξ| = 1 we have g′(t) + aiξg(t) = 0, which implies g(t) = e−iatξg(0). Insert back and get
u(t, x) = g(0)e−iatξeixξ = ei(x−at)ξ,
since g(0) = 1.Therefore the initial condition is translated with speed a for all ξ.
Example 5. Same Fourier analysis can be used for other equation also to study the speed of differentfrequency waves, e.g.,
ut + aux + uxxx = 0.
For u(t, x) = g(t)eixξ we get
ut + aux + uxxx = (g′ + iξ(a− ξ2)g)eixξ = 0,
thusg′ + iξ(a− ξ2)g = 0,⇒ g(t) = e−iξ(a−ξ
2)tg(0),
so the solution isu(t, x) = ei(x−(a−ξ2)t)ξg(0).
now the speed of the waves depends on ξ.
Definition 7 (Dispersion). The phenomenon of waves with different frequencies moving with different speedsis called dispersion.
Return now to the solution of the difference equation. Take Lax–Friedrichs:
vn+1m =
12
(vnm+1 + vnm−1)− aλ
2(vnm+1 − vnm−1).
Separation of variables: vnm = gneimhξ and substitute above to get
g = cos(hξ)− iaλ sin(hξ),
so the solution isvnm = (cos(hξ)− iaλ sin(hξ))neimhξ,
which looks nothing like ei(x−at)ξ. Let
g(hξ) ≡ ρe−iω = ρ cosω − ρi sinω = cos(hξ)− iaλ sin(hξ).
14 CHAPTER 1. HYPERBOLIC PDES
Therefore tanω = aλ tan(hξ), ρ2 = cos2(hξ) + a2λ2 sin2(hξ). For |hξ| ≤ π/2 we have
vnm = (ρe−iω)neimhξ = ρneimhξ−iωn = ρnei(mh−ωn/ξ)ξ = ρnei(x−ωnξ
tnk )ξ = ρnei(x−
ωkξ t)ξ = ρnei(x−α(hξ)t)ξ,
where x = mh, t = nk and
α(hξ) ≡ ω
kξ=
arctan(aλ tan(hξ))λhξ
(Recall tan ε ≈ ε and arctan ε ≈ ε so α ≈ arctan(aλ tan(hξ))λhξ ≈ a.)
We havevnm = |g(hξ)|n · ei(x−α(hξ)t)ξ.
Definition 8 (Phase speed). The quantity α(hξ) is called phase speed, and is the speed at which waves offrequency ξ are propagated by the difference scheme.
Once again, waves with different frequencies travel with different speeds. Thus we say that the scheme isdispersive. We want the scheme to be dispersive as little as possible (i.e., α(hξ) ≈ a), so that the numericalsolution looks like the exact solution.
Time to study the Taylor series for α(hξ) to obtain a better estimate of the closeness to a.
tan z = z +13z3 +O(z5)
arctan z = z − 13z3 +O(z5)
Let z = hξ
α(z) =arctan(aλ tan z)
λz
=aλ tan zλz
− (aλ tan z)3
3λz+ . . .
= a · z + z3/3 + . . .
z− a3λ3
λ
z3
3z+ . . .
= a
(1 + (1− a2λ2)
(hξ)2
3+ . . .
)So, if ξ is given and h is small, then the wave speed is slightly higher than a, and the high frequencies travelfastest. Let’s look at some special cases.
Take hξ = π/2. Then ω = π/2 and ρ = |aλ|, so
vnm = |aλ|neimhξ · e−inπ/2 = |aλ|nei(xξ−πn/2) = |aλ|nei(x−t/λ)ξ
(since nπ/2 = (t/k)(hξ) = tξ/λ). So the speeds can be quite different. Exact = a; Computed = 1/λ = aaλ ,
so it is not a good idea to take aλ small. The closer to the stability limit (i.e., the closer to 1) the better.
1.7. GROUP VELOCITY AND PROPAGATION OF WAVE PACKETS 15
1.7 Group Velocity and Propagation of Wave Packets
Consider the numerical solution to ut + aux = 0 with initial data u(0, x) = eiξ0xp(x), where p(x) decaysrapidly at ∞. The function u(0, x) is called a wave packet—see Figure 1.1.
−1 −0.5 0 0.5 1 1.5 2−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Figure 1.1: Wave packet cos 5π cos2 πx2 on [−1, 1]
The exact solution is u(t, x) = eiξ0(x−at)p(x− at).
Proposition 1. A finite difference scheme will have a solution that is approximately
v∗(t, x) = eiξ0(x−α(hξ0)t)p(x− γ(hξ0)t),
where α(hξ0) is the phase speed, and γ(hξ0) is the group velocity, given by
γ(θ) = α(θ) + θα′(θ).
The rest of this section is devoted to proving Proposition 1 and may be skipped on a first reading.Consider a class of one-step schemes with the property
vn(ξ) = g(hξ)vn−1(ξ) = . . . = (g(hξ))nv0(ξ).
In addition, for simplicity, assume |g(hξ)| = 1. The numerical method will give
vnm =1√2π
∫ π/h
−π/heiξxm(g(hξ))nv0(ξ)dξ.
On the other sideu(0,mh) =
1√2π
∫ ∞−∞
eimhξu(0, ξ)dξ.
Split up the interval (−∞,∞) =⋃∞l=−∞[l 2πh −
πh , l
2πh + π
h ) to obtain
u(0,mh) =1√2π
∑l
∫ l 2πh +πh
l 2πh −πh
eimhξu(0, ξ)dξ.
Set ξ = l 2πh + ξ′, meaning dξ = dξ′ and
u(0,mh) =1√2π
∑l
∫ πh
−πheimh(l 2πh +ξ′)u(0, l
2πh
+ ξ′)dξ′
=1√2π
∫ πh
−πheimhξ
′∞∑
l=−∞
u(0, ξ′ + l2πh
)dξ′.
16 CHAPTER 1. HYPERBOLIC PDES
This gives a formula for
v0(ξ) =∞∑
l=−∞
u(0, ξ + l2πh
).
If u is smooth, then its Fourier transform decays rapidly, and only the middle l = 0 term really matters
v0(ξ) ∼ u(0, ξ),
with the error bounded by h to some high power depending on the smoothness of u(0, x).Consider
u(0, ξ) =1√2π
∫ ∞−∞
e−ixξu(0, x)dx
=1√2π
∫ ∞−∞
e−ixξeiξ0xp(x)dx
=1√2π
∫ ∞−∞
e−ix(ξ−ξ0)p(x)dx
= p(ξ − ξ0)
Let’s recall how we handle the phase speed
g(hξ) = |g(hξ)|e−iω = e−iω ⇒ (g(hξ))n = e−iωn = e−i ωnkk
nhξξ
= e−iωλhξ tξ = e−iα(hξ)tξ.
We can return to
vnm =1√2π
∫ π/h
−π/heiξxm · e−iα(hξ)tnξ · v0(ξ)dξ.
Exercise 1. Verify that eiξxm , v0(ξ), g(hξ), and ω are periodic functions of ξ with period 2πh .
Since v0(ξ) ∼ u(0, ξ) = p(ξ − ξ0), we change the variables φ = ξ − ξ0. Then ξ = φ+ ξ0, and
vnm =1√2π
∫ π/h+ξ0
−π/h+ξ0
ei(φ+ξ0)xm · e−iα(h(φ+ξ0))tn(φ+ξ0) · v0(φ+ ξ0)dφ.
Since all functions are periodic with period 2πh we can shift the interval of integration back and get
vnm =1√2π
∫ π/h
−π/hei(φ+ξ0)xm · e−iα(h(φ+ξ0))tn(φ+ξ0) · v0(φ+ ξ0)dφ.
This begins to look right. Factor out the phase speed
vnm = eiξ0(xm−α(hξ0)tn) · 1√2π
∫ π/h
−π/heiφxm · e−i[(φ+ξ0)α(hφ+hξ0)−ξ0α(hξ0)]tn · v0(φ+ ξ0)dφ.
This begins to look like a Fourier transform
∼ eiξ0(x−α(hξ0)t) · 1√2π
∫ π/h
−π/heiφ“x− (φ+ξ0)α(hφ+hξ0)−ξ0α(hξ0)
φ t”· v0(φ+ ξ0)dφ.
Since v0(ξ) ∼ p(ξ− ξ0) we have v0(φ+ ξ0) ∼ p(φ+ ξ0 − ξ0) = p(φ). The next step is to replace v0(φ+ ξ)by p(φ). Also since p(φ) goes to zero rapidly as φ→∞ we may as well let the integral go to infinity. Hence
vnm ∼ eiξ0(x−α(hξ0)t) · 1√2π
∫ ∞−∞
eiφ(x−γt) · p(φ)dφ = eiξ0(x−α(hξ0)t) · p(x− γt).
1.7. GROUP VELOCITY AND PROPAGATION OF WAVE PACKETS 17
The last step is wrong because γ depends on φ, but it does tell us where we are going. We have
γ ≡ (φ+ ξ0)α(hφ+ hξ0)− ξ0α(hξ0)φ
=(hφ+ ξ0)α(hφ+ hξ0)− hξ0α(hξ0)
hφ
=(θ + θ0)α(θ + θ0)− θ0α(θ0)
θ
=G(θ + θ0)−G(θ0)
θ
= G′(θ0) + θ2G′′(θ∗),
where θ ≡ hφ, θ0 ≡ hξ0 and G(θ) ≡ θα(θ) and θ∗ is between θ0 and θ. The beauty of the above expressionis that G′(θ0) does not depend on φ, but only on hξ0 = θ0.
Let’s go back to
vnm ∼ eiξ0(xm−α(hξ0)tn) · 1√2π
∫ π/h
−π/heiφ(xm−G′(hξ0))tn · e−iφ
hφ2 G′′(θ∗)tp(φ)dφ.
The idea is to replace e−iφhφ2 G′′(θ∗)t by one. By doing so we are making an error bounded by∫ ∞
−∞
∣∣∣e−iφhφ2 G′′(θ∗)t − 1∣∣∣ · |p(φ)|dφ.
We will now show that this error is at most O(h).Let’s first bound |p(φ)|
p(φ) =1√2π
∫ ∞−∞
e−iφxp(x)dx
φ4p(φ) =1√2π
∫ ∞−∞
(−iφ)4e−iφxp(x)dx;
=1√2π
∫ ∞−∞
∂4
∂x4(e−iφx) · p(x)dx
=1√2π
∫ ∞−∞
e−iφx · p′′′′(x)dx
Thus
|p(φ)| ≤ 1√2π·∫∞−∞ |p
′′′′(x)|dx|φ|4
=C
|φ|4.
Also |eiz − 1|2 = 4 sin2 z2 ≤ |z|
2, so |eiz − 1| ≤ |z|. By using this estimate we have∫ ∞−∞
∣∣∣e−iφhφ2 G′′(θ∗)t − 1∣∣∣ · |p(φ)|dφ ≤
∫ ∞−∞
∣∣∣∣hφ2 12G′′(θ∗)t
∣∣∣∣ · |p(φ)|dφ
≤ h · const ·∫ ∞−∞|φ2p(φ)|dφ
≤ h · const ·∫ ∞−∞
∣∣∣∣ 1φ2
∣∣∣∣ dφ≤ h · const.
If we work in L2 we can bound the error by h2 in norm, but not pointwise.Either way we have shown that
vnm = eiξ0(x−α(hξ0)t)p(x−G′(hξ)t) + small terms.
18 CHAPTER 1. HYPERBOLIC PDES
Definition 9. The quantity γ(θ) = G′(θ) = α(θ) + θ · α′(θ) is called group velocity.
We have α(θ) → a as h → 0. So the phase speed is different from the group velocity, but both tend tothe correct speed a as h→ 0. Otherwise the numerical method will not converge.
1.8 Summary of Schemes for the Wave Equation ut + aux = 0
Notation: λ = kh , θ = hξ.
Name Scheme g(θ) Stable dissipative α(θ)/aForward timeforward space
vn+1m −vnm
k + avnm+1−v
nm
h = 0
Forward timebackward space
vn+1m −vnm
k + avnm−v
nm−1h = 0 1− aλ+ aλe−iθ 0 ≤ aλ ≤ 1
order 2, if0 < aλ < 1
Forward-timecentered space
vn+1m −vnm
k + avnm+1−v
nm−1
2h = 0 1− iaλ sin θ No No
Backward-timecentered space
vnm−vn−1mk + a
vnm+1−vnm−1
2h = 0 1− θ26 (1 + 2a2λ2)
Lax–Wendroffvn+1m =vnm − aλ
2 (vnm+1 − vnm−1)
+ a2λ22 (vnm+1 − 2vnm + vnm−1)
1− 2a2λ2 sin2 θ2
−iaλ sin θ|aλ| ≤ 1 order 4 1− 1
6 θ2(1− a2λ2)
Lax–Friedrichs
vn+1m − 1
2 (vnm+1+vnm−1)
k
+amnm+1−v
nm−1
2h = 0cos θ − iaλ sin θ |aλ| ≤ 1 No 1 + (1− a2λ2) θ
23
Leap-Frogvn+1m −vn−1
m2k + a
vnm+1−vnm−1
2h =0−iaλ sin θ
±p
1− (aλ sin θ)2|aλ| < 1 No
atan aλ sin θ√1−(aλ sin θ)2
a
Crank–Nicolson
vn+1m −vnm
k +
avn+1m+1−v
n+1m−1+vnm+1−v
nm−1
4h = 0
1− 12 iaλ sin θ
1+ 12 iaλ sin θ
Always No 1− θ26 (1 + 1
2a2λ2)
Chapter 2
Parabolic equations
2.1 The Heat Equation
ut = buxx
Schemes:
• Lax–Friedrichsvn+1m − 1
2 (vnm+1 + vnm−1)k
= bvnm+1 − 2vnm + vnm−1
h2
• Lax–Wendroff
u(t+ k) = u+ kut +k2
2utt
= u+ kbuxx +k2b2
2uxxxx
vn+1m = vnm + kb
vnm+1 − 2vnm + vnm−1
h2+k2b2
2·vnm+2 − 4vnm+1 + 6vnm − 4vnm−1 + vnm−2
h4
• Forward in time, centered in space
vn+1m − vnm
k= b
vnm+1 − 2vnm + vnm−1
h2
• Backward in time, centered in space
vn+1m − vnm
k= b
vn+1m+1 − 2vn+1
m + vn+1m−1
h2
• Leap-Frogvn+1m − vn−1
m
2k= b
vnm+1 − 2vnm + vnm−1
h2
• Du Fort–Frankelvn+1m − vn−1
m
2k= b
vnm+1 − (vn+1m + vn−1
m ) + vnm−1
h2
• Crank–Nicolson
vn+1m − vnm
k=b
2
(vn+1m+1 − 2vn+1
m + vn+1m−1
h2+vnm+1 − 2vnm + vnm−1
h2
).
19
20 CHAPTER 2. PARABOLIC EQUATIONS
For parabolic equations it appears natural to have a “new” λ, which we will call µ.
Definition 10 (µ).
µ ≡ k
h2.
Von Neumann analysis works the same way: vnm = gneimhξ.
Example 6. Forward in time, centered in space.
g(hξ) = 1− 4bµ sin2 hξ2 .
Stability requires |g| ≤ 1, i.e.,
0 ≤ 4bµ sin2 hξ2 ≤ 2 for all |hξ| ≤ π, meaning bµ ≤ 1
2 .
The scheme is dissipative of order 2 as long as bµ is strictly less than 12 (check!). For bµ = 1
2 we haveg = 1 − 2 sin2 hξ
2 and the frequency ξ = πh is not damped at all: v0
m = eimhξ = eimπ = (−1)m remainsunchanged by the scheme.
Definition 11. Let
‖u(t, x)‖x ≡(∫ ∞−∞|u(t, x)|2dx
) 12
mean the L2 norm of u(t, x) with respect to x for a fixed t.
Remark 2. Let u be a solution to ut = buxx. Then the overall energy E(t) ≡ ‖u(t, x)‖2x decreases with time
‖u(t, x)‖x ≤ ‖u(s, x)‖x, when t ≥ s
and the solution becomes smoother with time
‖ux(t, x)‖2x ≤1
2bt‖u(0, x)‖2.
The dissipative schemes possess the same qualities
‖vn+1‖ = ‖vn+1‖ = ‖g(hξ)vn‖ ≤ ‖vn‖ = ‖vn‖
and‖δ+vn‖2 ≤
4µCt‖v0‖2,
where δ+vm ≡ vm+1−vmh , and C is a constant.
Proof. We will show that E′(t) ≤ 0 meaning E(t) is decreasing:
E′(t) =∫ ∞−∞
2uutdx =∫ ∞−∞
2buuxxdx = 2buux∣∣∣∞−∞− 2b
∫ ∞−∞
u2xdx = −2b‖ux‖2x ≤ 0,
because u(t, x)→ 0 as x→ ±∞ for ‖u(t, x)‖x to exist.The above implies (after integrating from 0 to t):
E(t)− E(0) = −2b∫ t
0
‖ux(τ, x)‖2xdτ ⇒ E(0) ≥ 2b∫ t
0
‖ux(τ, x)‖2xdτ
The derivative ux = ∂∂xu(t, x) is also a solution to ut = buxx because (ux)t = (ut)x = buxxx = b(ux)xx,
therefore ‖ux(t, x)‖x ≤ ‖ux(s, x)‖x for t ≥ s. Now we get
E(0) ≥ 2b∫ t
0
‖ux(τ, x)‖2xdτ ≥ 2bt‖ux(t, x)‖2x,
2.1. THE HEAT EQUATION 21
meaning
‖ux(t, x)‖2x ≤1
2bt‖u(0, x)‖2x,
i.e., the solution get smoother and smoother as t→∞.Now repeat the same analysis for a difference scheme that is dissipative of order (say) 2:
vn+1(ξ) = g(hξ)vn(ξ), where |g(hξ)| ≤ 1− C sin2 hξ2 ,
which implies
‖vn+1(ξ)‖2 ≤∥∥∥(1− C sin2 hξ
2
)vn(ξ)
∥∥∥2
and (after some major reworking):
‖vn+1‖2 + C∥∥∥sin hξ
2 · vn∥∥∥2
≤ ‖vn‖2
(these are, of course, the discrete L2 norms). Now comes the big moment,
sin(hξ/2) · vn(ξ) =eihξ/2 − e−ihξ/2
2i· vn(ξ) =
e−ihξ/2
2i(eihξ − 1)vn(ξ).
Next, observe that
δ+vnm ≡
1h
(vnm+1 − vnm) =1√2π
∫ π/h
−π/heimhξ
eihξ − 1h
vn(ξ)dξ.
On the other side,
δ+vnm =
1√2π
∫ π/h
−π/heimhξ δ+vn(ξ)dξ,
soeihξ − 1
hvn(ξ) = δ+vn(ξ),
and inserting we get‖ sin(hξ/2) · vn‖ = 1
2‖(eihξ − 1)vn‖ = 1
2 · h · ‖δ+vn‖.
We can therefore simplify our inequality
‖vn+1‖2 + Ch2
4‖δ+vn‖2 ≤ ‖vn‖2.
Parseval says “hats = no hats”, so
‖vn+1‖2 +Ck
4µ‖δ+vn‖2 ≤ ‖vn‖2, (2.1)
where, again, µ = k/h2. In particular,‖vn+1‖ ≤ ‖vn‖.
Next, we prove that ‖δ+vn+1‖ ≤ ‖δ+vn‖. The only property we used was that vnm was solution to
vn+1m − vnm
k= b
vnm+1 − 2vnm + vnm−1
h2.
Now tryvn+1m+1 − vnm+1
k= b
vnm+2 − 2vnm+1 + vnmh2
.
22 CHAPTER 2. PARABOLIC EQUATIONS
Subtract first from second, divide by h and get
vn+1m+1−v
n+1m
h − vnm+1−vnm
h
k= b
vnm+2−vnm+1
h − 2 vnm+1−v
nm
h + vnm−vnm−1h
h2.
Or in a simpler language
(δ+v)n+1m − (δ+vn)m
k= b
(δ+v)nm+1 − 2(δ+v)nm + (δ+v)nm−1
h2,
meaning ‖δ+vn+1‖ ≤ ‖δ+vn‖.The inequality (2.1) works for all time steps
‖vn+1‖2 +Ck
4µ‖δ+vn‖2 ≤ ‖vn‖2
‖vn‖2 +Ck
4µ‖δ+vn−1‖2 ≤ ‖vn−1‖2
...
‖v1‖2 +Ck
4µ‖δ+v0‖2 ≤ ‖v0‖2,
which we sum up and cancel common terms to obtain
‖vn+1‖2 +Ck
4µ
n∑k=0
‖δ+vk‖2 ≤ ‖v0‖2 ⇒
‖vn+1‖2 +Ck(n+ 1)
4µ‖δ+vn+1‖2 ≤ ‖v0‖2 ⇒
‖δ+vn+1‖2 ≤ 4µCt‖v0‖2.
This means that the numerical solution will smooth out—as long as the scheme is dissipative.
2.2. THE DU FORT–FRANKEL SCHEME 23
2.2 The Du Fort–Frankel Scheme
This is an example of an explicit and unconditionally stable scheme for ut = buxx.The problem with schemes like forward time, centered space is that they are stable for bµ = bk/h2 ≤ 1
2 ,which puts a terrible restriction k ≤ h2
2b on the timestep. The Du Fort–Frankel scheme,
vn+1m − vn−1
m = 2bµ(vnm+1 − (vn+1m + vn−1
m ) + vnm−1),
is a slight modification of the unstable Leap–Frog scheme. We rewrite the Du Fort–Frankel scheme as
(1 + 2bµ)vn+1m − (1− 2bµ)vn−1
m = 2bµ(vnm+1 + vnm−1).
To study the stability, we substitute vnm = gneimhξ to get
(1 + 2bµ)g2 − (1− 2bµ) = 2bµ(eihξ + e−ihξ)g,
which implies
g± =2bµ cos(hξ)±
√1− 4b2µ2 sin2(hξ)
1 + 2bµ.
The scheme is not dissipative since g−(π) = −1. To determine stability we consider two cases:
• 1− 4b2µ2 ≥ 0⇒ |g±| ≤ 2bµ| cos(hξ)|+√
11+2bµ ≤ 2bµ+1
1+2bµ = 1.
• 1− 4b2µ2 < 0⇒ |g±|2 = (2bµ cos(hξ))2+4b2µ2 cos2(hξ)−11+2bµ = 4b2µ2−1
(1+2bµ)2 = 2bµ−11+2bµ ≤ 1.
In addition, we do not want double roots on the unit circle. Double root occurs when 1 − 4b2µ2 = 0, butthen |g±| ≤ 2bµ| cos(hξ)|
1+2bµ < 1.So we have stability for any value of µ. But how is that possible? The catch is in the consistency. In
order for the scheme to be consistent we must have k/h→ 0, as we will now demonstrate.Rewrite Du Fort–Frankel as
vn+1m − vn−1
m
2k= b
vnm+1 − 2vnm + vnm−1
h2− bv
n+1m + vn−1
m − 2vnmh2
then expand in Taylor to see that it approximates
ut +k2
6uttt = b
(uxx +
h2
12uxxxx
)− b(k2
h2utt +
k4
12h2utttt
).
Now think numerically. For hyperbolic systems we could (at best) hope for kh ≈ 1. However, if we use Du
Fort–Frankel with such a timestep, k = h, the solution will not converge to the solution of ut = buxx, butinstead to the solution of butt+ut = buxx (i.e., the solution to a wave equation). This was not the purpose ofthe exercise. So the scheme will only converge to the solution of ut = buxx if k
h → 0. Even so the truncationerror will be dominated by b k
2
h2utt, which is not small unless kh2 is constant, but then we are back where we
started—with the same restrictions as the ones for forward in time centered in space.We, of course have two explicit schemes—backward in time, centered in space (which is O(k + h2) and
dissipative) and Crank–Nicolson (which is O(k2 + h2) and not dissipative if kh is constant.).
24 CHAPTER 2. PARABOLIC EQUATIONS
2.3 The Convection-Diffusion Equation: ut + aux = buxx
If a = 0, we have the heat equation, and if b = 0, we have the wave equation. Define a new function w suchthat w(t, x− at) = u(t, x), i.e., w(t, x) = u(t, x+ at). Then
bwxx = buxx = ut + aux = wt − awx + awx = wt.
So u is simply the solution to the heat equation translated with speed a. The problem occurs when theviscosity coefficient b is very small compared to a. Then the obvious numerical methods trick you. We havea choice of
vn+1m − vnm
k+ a
vnm+1 − vnm−1
2h= b
vnm+1 − 2vnm + vnm−1
h2, (2.2)
vn+1m − vnm
k+ a
vnm+1 − vnmh
= bvnm+1 − 2vnm + vnm−1
h2,
vn+1m − vnm
k+ a
vnm − vnm−1
h= b
vnm+1 − 2vnm + vnm−1
h2. (2.3)
We expand in Taylor series to see that the orders of accuracy are
O(k + h2), O(k + h), and O(k + h),
respectively, which speaks strongly in favor of (2.2).The heat equation has one nice property. The maximum at a later time is less than the maximum at an
earlier time. We will copy that. Rewrite (2.2) as
vn+1m =
(bµ− aλ
2
)vnm+1 + (1− 2bµ)vnm +
(bµ+
aλ
2
)vnm−1
= bµ
(1− aλ
2bµ
)vnm+1 + (1− 2bµ)vnm + bµ
(1 +
aλ
2bµ
)vnm−1.
Let α ≡ aλ2bµ = ah
2b . Now if all the coefficients are positive (i.e., |α| = | aλ2bµ | < 1), then
|vn+1m | ≤ bµ(1− α) max
m|vnm|+ (1− 2bµ) max
m|vnm|+ bµ(1 + α) max
m|vnm|
≤ [bµ(1− α) + (1− 2bµ) + bµ(1 + α)] maxm|vnm|
≤ maxm|vnm|.
The maximum is a decreasing function of time if |α| ≤ 1, i.e., if
|a|b· h
2≤ 1, which is the same as h ≤ 2b
|a|. (2.4)
This will, of course, be satisfied eventually as h→ 0, but who can wait that long? Say a = 10, b = 10−2 ⇒h ≈ 10−3 is needed. And remember, for stability we must have 1/2 ≥ bµ = 10−2 · k/(10−3)2. This impliesk ≈ 10−4/2, which is terribly small.
Now look at (2.3) instead:
vn+1m − vnm + aλ(vnm − vnm−1) = bµ(vnm+1 − 2vnm + vnm−1),
which we rewrite as
vn+1m = (bµ+ aλ)vnm−1 + (1− 2bµ− aλ)vnm + bµvnm+1
= bµ(1 +aλ
bµ)vnm−1 + (1− 2bµ(1 +
aλ
2bµ))vnm + bµvnm+1
= bµ(1 + 2α)vnm−1 + (1− 2bµ(1 + α))vnm + bµvnm+1.
2.4. SUMMARY OF SCHEMES FOR THE HEAT EQUATION 25
Say a > 0, so α > 0. Now the requirement for max-norm stability becomes
2bµ(1 + α) < 1 or 2bµ+ aλ < 1,
which is a lot less restrictive than (2.4). We can pick h to be 10−2 instead of 10−3, i.e., 10 times larger. Wetry k = 10−3
5 , h = 10−2, a = 10, b = 10−2. Then
2bk
h2+ a
k
h= 2 · 10−2 ·
10−3
5
(10−2)2=
250
+15< 1.
So we increased the timestep by a factor of 10. But at what price?We can rewrite (2.3) as
vn+1m − vnm
k+ a
vnm+1 − vnm−1
2h=(b+
ah
2
)vnm+1 − 2vnm + vnm−1
h2.
We introduced an artificial viscosity ah2 = bα. This artificial viscosity comes from our numerical method.
Therefore solving ut + aux = b(1 +α)uxx using (2.2) is equivalent to solving ut + aux = buxx using (2.3).
2.4 Summary of Schemes for the Heat Equation ut = buxx
Notation: µ ≡ kh2 , θ = hξ.
Name Scheme g(hξ) Stable Dissipative
Lax–Friedrichsvn+1m − 1
2 (vnm+1+vnm−1)k
= bvnm+1−2vnm+vnm−1
h2
Lax–Wendroffvn+1m = vnm + kb
vnm+1−2vnm+vnm−1h2
+ k2b22 ·
vnm+2−4vnm+1+6vnm−4vnm−1+vnm−2h4
Forward timecentered space
vn+1m −vnm
k= b
vnm+1−2vnm+vnm−1h2 1 − 4bµ sin2 hξ
2 bµ ≤ 12 2, if bµ < 1
2
Backward timecentered space
vn+1m −vnm
k= b
vn+1m+1−2vn+1
m +vn+1m−1
h2 Always if µ ≥ c, c > 0
Leap-Frogvn+1m −vn−1
m2k = b
vnm+1−2vnm+vnm−1h2 No
Du Fort–Frankelvn+1m −vn−1
m2k = b
vnm+1−(vn+1m +vn−1
m )+vnm−1h2
2bµ cos θ±q
1−4b2µ2 sin2 θ1+2bµ Always No
Crank–Nicolson
vn+1m −vnm
k= b
2
0@ vn+1m+1−2vn+1
m +vn+1m−1
h2
+vnm+1−2vnm+vnm−1
h2
! Always order 2
26 CHAPTER 2. PARABOLIC EQUATIONS
Chapter 3
Systems of PDEs in HigherDimensions
3.1 The Equation ut + Aux = 0
We consider ut + Aux = 0, where u is a d-vector of functions and A and B are d-by-d matrices. Initialconditions u(0, x) = f(x) (all d-vectors). If A is symmetric A = QΛQT , where QTQ = I, then
ut +QΛQTux = 0⇒ QTut + ΛQTux = 0⇒ wt + Λwx = 0,
where w = QTu. The matrix Λ is diagonal, so the problem falls apart to d independent equations.
(wi)t + λi(wi)x = 0.
The solution is wi(t, x) = gi(x− λit), where g(x) = QT f(x). In general
u(t, x) = Qw(t, x) = Q · (g1(x− λ1t), . . . , gd(x− λdt))T .
How do we generate and analyze a numerical scheme?Take Lax–Wendroff:
u(t+ k) = u+ kut +k2
2utt = u− kAux +
k2
2A2uxx.
Therefore
vn+1m = vnm − kA
vnm+1 − vnm−1
2h+k2
2A2 v
nm+1 − 2vnm + vnm−1
h2
= vnm −λ
2A(vnm+1 − vnm−1) +
λ2
2A2(vnm+1 − 2vnm + vnm−1).
Here vnm is a d-vector. We can use the same Fourier analysis as before. Fourier transform on a vector meansFourier transform in each component, therefore, as before
vn+1(ξ) =[I − iλA sin(hξ)− 2λ2A2 sin2 hξ
2
]vn(ξ) = G(hξ)vn(ξ) = . . . = (G(hξ))nv0(ξ)
We call G the amplification matrix. Stability will mean
‖vn+1‖ ≤ (1 + Ck)‖vn‖, or ‖vn+1‖ ≤ Const · ‖v0‖,
27
28 CHAPTER 3. SYSTEMS OF PDES IN HIGHER DIMENSIONS
when C does not depend on h, k. Let’s analyze (G(hξ))n. Assume A = TΛT−1. Then (after some verysimple math)
Gn(hξ) = T
[I − iλΛ sin(hξ)− 2λ2Λ2 sin2 hξ
2
]nT−1.
Therefore |Gn(hξ)| is bounded if [I−iλΛ sin(hξ)−2λ2Λ2 sin2 hξ2 ]n is. The latter is a diagonal matrix, meaning
each element on the diagonal needs to be bounded. We have reduced the problem to the one-dimensionalcase (which implies λ|λj(A)| < 1) , therefore the stability condition is
λ · max1≤j≤d
|λj(A)| ≤ 1.
3.2 The Equation ut + Aux = Bu
Take Lax–Wendroff for ut +Aux = 0 and just add the lower order term
vn+1m = vnm −
λ
2A(vnm+1 − vnm−1) +
λ2
2A2(vnm+1 − 2vnm + vnm−1) + kBvnm
The amplification factor for the scheme becomes
vn+1(ξ) = (G+ kB)vn(ξ),
so we end up with the question when |(G + kB)n| is bounded by a constant, where |Gn(hξ)| ≤ C. We useStrang’s idea:
(G+ kB)n = Gn + k(Gn−1B +Gn−2BG+ . . . BGn−1
)+ k2
(Gn−2B2 +Gn−3BGB +Bn−4BG2B + . . .+B2Gn−2
)+ . . .
+ kn(G0BG0B . . .G0BG0).
Now estimate upward using |Gn| ≤ C to obtain
|(G+ kB)n| ≤ C +(n
1
)kC2|B|+
(n
2
)k2C3|B|2 + . . .
≤ C
(1 +
(n
1
)kC · |B|+
(n
2
)k2C2 · |B|2 + . . .
)≤ C(1 + k|B|C)n ≤ Cenk|B|C ≤ CeT |B|C
for all nk ≤ T .Therefore the condition for stability is once again |Gn| ≤ C.
3.3 The Equation ut + Aux + Buy = 0
The equation ut + Aux + Buy = 0, where A and B are d-by-d matrices, is a baby problem for an under-lying three dimensional problem, which linearized and reduced leads to the above equation, namely Euler’sequation in fluid mechanics. For a numerical scheme pick
vn+1ml − v
n−1ml
2k+A
vnm+1,l − vnm−1,l
2h+B
vnm,l+1 − vnm,l−1
2h= 0.
Using Fourier transforms in two dimensions we get
vn+1(ξ1, ξ2)− vn(ξ1, ξ2) = 2iλ[A sin(hξ1) +B sin(hξ2)]vn(ξ1, ξ2),
3.3. THE EQUATION UT +AUX +BUY = 0 29
which is the same as[vn+1
vn
]=[
2iλ(A sin θ1 +B sin θ2) II 0
] [vn
vn−1
]= G(θ1, θ2)
[vn
vn−1
].
So we have to study |Gn(θ1, θ2)| and see when it is bounded. Depending on the assumptions one makes thiscan be easy or hard. If we make the simplifying assumptions that A and B are simultaneously diagonalizable,A = PΛ′P−1, B = PΛ′′P−1. Then
A sin(hξ1) +B sin(hξ2) = P diag(λ′i sin(hξ1) + λ′′i sin(hξ2))P−1
and we can write
G =[P
P
] [2iλ(Λ′ sin θ1 + Λ′′ sin θ2) I
I 0
] [P−1
P−1
],
so the whole analysis breaks down to the analysis of the scalar case. Without repeating the previous analysiswe want
|λ(λ′i sin θ1 + λ′′i sin θ2)| < 1− ε
for all θ1 = hξ1, θ2 = hξ2 and i = 1, 2, . . . , d. Our condition for stability is then
λmaxi
(|λ′i|+ |λ′′i |) < 1− ε.
In general if the assumption is that A and B are only symmetric, then the stability criterion is the same,but the proof is a bit more complicated.
30 CHAPTER 3. SYSTEMS OF PDES IN HIGHER DIMENSIONS
3.4 The equation ut = b1uxx + b2uyy
Introduce a notation
A1u = b1∂2
∂x2u, A2u = b2
∂2
∂y2u.
The discrete versions of these operators are
A1hvnml = b1
vnm+1,l − 2vnml + vnm−1,l
h2and A2hv
nml = b2
vnm,l+1 − 2vnml + vnm,l−1
h2,
where u(t, x, y) = u(nk,mh, lh) ∼ vnml (assume ∆x = ∆y = h and ∆t = k). Next we attack ut =b1uxx + b2uyy = A1u+A2u using the Crank–Nicolson idea
vn+1ml − vnml
k= (A1h +A2h)
vn+1ml + vnml
2,
which is equivalent to (I − k
2(A1h +A2h)
)vn+1 =
(I +
k
2(A1h +A2h)
)vn,
and also equivalent to(I − k
2A1h
)(I − k
2A2h
)vn+1 =
(I +
k
2A1h
)(I +
k
2A2h
)vn +A1hA2h
k2
4(vn+1 − vn)︸ ︷︷ ︸
(∗)
The expression (*) is O(k3) (since vn+1−vn = O(k)), so dropping it does not affect the order of the accuracyof the scheme.
Our scheme thus becomes(I − k
2A1h
)(I − k
2A2h
)vn+1 =
(I +
k
2A1h
)(I +
k
2A2h
)vn.
3.5. ADI METHODS 31
3.5 ADI methods
Solve the above as (I − k
2A1h
)vn+1/2 =
(I +
k
2A2h
)vn(
I − k
2A2h
)vn+1 =
(I +
k
2A1h
)vn+1/2
This is the Peaceman–Rachford Algorithm, which is an ADI method—alternating direction implicitmethod. Meaning that the two-dimensional problem has been reduced to two one-dimensional implicitproblems by factoring the scheme.
Let’s now perform stability analysis of this scheme.(I − k
2A2h
)vn+1 =
(I +
k
2A1h
)vn+1/2(
1− b2k
2h2(eihξ2 − 2 + e−ihξ2)
)vn+1 =
(1 + b1
k
2h2(eihξ1 − 2 + e−ihξ1)
)vn+1/2
(1 + 2µb2 sin2(hξ2/2)
)vn+1 =
(1− 2µb1 sin2(hξ1/2)
)vn+1/2
vn+1 =1− 2µb1 sin2(hξ1/2)1 + 2µb2 sin2(hξ2/2)
vn+1/2.
Similarly vn+1/2 =1− 2µb2 sin2(hξ2/2)1 + 2µb1 sin2(hξ1/2)
vn
Meaning vn+1 =1− 2µb1 sin2(hξ1/2)1 + 2µb1 sin2(hξ1/2)
· 1− 2µb2 sin2(hξ2/2)1 + 2µb2 sin2(hξ2/2)
· vn.
Since∣∣∣ 1−x1+x
∣∣∣ ≤ 1 for any x ≥ 0, we conclude |vn+1| ≤ |vn|.Now using Parseval ∑
m,l
|vn+1ml |
2 · h2 =∫ π/h
−π/h
∫ π/h
−π/h|vn+1(ξ1, ξ2)|2dξ1dξ2
≤∫ π/h
−π/h
∫ π/h
−π/h|vn(ξ1, ξ2)|2dξ1dξ2
=∑m,l
|vnml|2 · h2.
Therefore we have stability for all values of m and l, and the order of accuracy of the scheme is O(k2+h2).Thus we can take k = h and the scheme is efficient and accurate.
The Douglas–Rachford method starts with the backward-time, central-space scheme for ut = A1u+A2u
(I − kA1h − kA2h)vn+1ml = vnml,
to obtain (after dropping an O(k3) term)
(I − kA1h)(I − kA2h)vn+1ml = (I + k2A1hA2h)vnml.
The method is
(I − kA1h) vn+1/2 = (I + kA2h) vn
(I − kA2h) vn+1 = vn+1/2 − kA2hvn.
32 CHAPTER 3. SYSTEMS OF PDES IN HIGHER DIMENSIONS
The Mitchell–Fairweather method is second-order accurate in time and fourth-order accurate in space.Recall the operator δ2:
δ2f(x) ≡ f(x+ h)− 2f(x) + f(x− h)h2
and
δ2xvnml =
vnm+1,l − 2vnml + vnm−1,l
h2and δ2
yvnml =
vnm,l+1 − 2vnml + vnm,l−1
h2.
Then
δ2f = f ′′ +h2
12f ′′′′ +O(h4) = f ′′ +O(h2),
so f ′′ = δ2f +O(h2). Then
δ2f = f ′′ +h2
12f ′′′′ +O(h4)
= f ′′ +h2
12(f ′′)′′ +O(h4)
= f ′′ +h2
12(δ2f ′′ +O(h2)) +O(h4)
= f ′′ +h2
12δ2f ′′ +O(h4)
= δ2f =(
1 +h2
12δ2
)d2
dx2f +O(h4). (3.1)
Now start with Peaceman–Rachford idea for ut = A1u+A2u:(I − k
2A1
)(I − k
2A2
)un+1 =
(I +
k
2A1
)(I +
k
2A2
)un +O(k3)
Multiply both sides by (1 +
h2
12δ2x
)(1 +
h2
12δ2y
)and replace (
1 +h2
12δ2x
)∂2
∂x2
by δ2x +O(h4) (see (3.1)). Similar changes are made for the derivatives with respect to y. The result is(1 +
h2
12δ2x −
k
2b1δ
2x
)(1 +
h2
12δ2y −
k
2b2δ
2y
)un+1
=(
1 +h2
12δ2x +
k
2b1δ
2x
)(1 +
h2
12δ2y +
k
2b2δ
2y
)un +O(k3) +O(kh4).
We obtain the Mitchell–Fairweather scheme:[1− 1
2
(b1µ1 −
16
)h2
1δ2x
]vn+1/2 =
[1 +
12
(b2µ2 +
16
)h2
2δ2y
]vn[
1− 12
(b2µ2 −
16
)h2
2δ2y
]vn+1 =
[1 +
12
(b1µ1 +
16
)h2
1δ2x
]vn+1/2.
3.6. BOUNDARY CONDITIONS FOR ADI METHODS 33
3.6 Boundary conditions for ADI methods
One can obtain the boundary conditions for the intermediate step vn+1/2 by solving for it using the boundaryconditions at steps n and n+ 1.
For example by subtracting the equations of the Peaceman–Rachford method(I − k
2A1
)un+1/2 =
(I +
k
2A2
)un(
I − k
2A2
)un+1 =
(I +
k
2A1
)un+1/2
(note that we use u’s and not v’s and keep everything is operator form for the moment) we get
un+1/2 =12
(I +
k
2A2
)un +
12
(I − k
2A2
)un+1.
Now if the indices i and j in vnij range from 1 to m, the desired boundary conditions at vn+1/21i and v
n+1/2mi
for i = 2, 3, . . . ,m− 1 are computed as
vn+1/21i =
12
(1 +
k
2A2
)vn1i +
12
(1− k
2A2
)vn+1
1i
=b2µv
n1,i−1 + 2(1− b2µ)vn1i + b2µv
n1,i+1
4+−b2µvn+1
1,i−1 + 2(1 + b2µ)vn+11i − b2µvn+1
1,i+1
4.
And similarly for vn+1/2mi for i = 2, 3, . . . ,m− 1.
The boundary conditionvn+1/2ij = u(tn+1/2, xi, yj)
is only first order accurate and if used with the Peaceman–Rachford method (or other similar second orderaccuracy) will result in the overall accuracy being only first order.
Boundary conditions for the Mitchell–Fairweather scheme are obtained as follows. First we eliminateδ2xvn+1/2 terms by multiplying the first equation by 1+ 1
2 (b1µ1+ 16 )h2
1δ2x and the second by 1− 1
2 (b1µ1− 16 )h2
1δ2x
to obtain:
vn+1/21i =
(b1µ1 + 1
6
) (1 + 1
2 (b2µ2 + 16 )h2
2δ2y
)vn1i +
(b1µ1 − 1
6
) (1− 1
2 (b2µ2 − 16 )h2
2δ2y
)vn+1
1i
2b1µ1
for i = 2, 3, . . . ,m− 1, and similarly for the other boundary.
34 CHAPTER 3. SYSTEMS OF PDES IN HIGHER DIMENSIONS
Chapter 4
Elliptic Equations
4.1 Steady-State Heat Equation
The steady-state heat equation isuxx + uyy = f(x, y). (4.1)
We solve it numerically by introducing a rectangular grid on a finite domain Ω (which need not be rectan-gular).
The numerical scheme is then
vm+1,l − 2vml + vm−1,l
h2+vm,l+1 − 2vml + vm,l−1
h2= fml, (4.2)
with boundary conditions vml specified on the boundary of Ω. We write Ωh for the set of grid points in Ω,∂Ω for the boundary of Ω and ∂Ωh for the set of grid points on the boundary of Ω.
Existence and uniqueness of the solution of (4.2)
Theorem 5. The equation (4.2) has a unique solution.
Proof. Assume there is another solution to (4.2); call it wml, and let eml = vml − wml. Subtract (4.2) for vand w to obtain
em+1,l − 2eml + em−1,l
h2+em,l+1 − 2eml + em,l−1
h2= 0,
with eml = 0 on the boundary. Assume now that eml attains its maximum at an inner point (m, l) on Ω.Then
eml =em+1,l + em−1,l + em,l+1 + em,l−1
4.
But eml ≥ em±1,l±1, so equality is only possible if eml = em±1,l±1. Continuing that way we observe that themaximum must also occur on the boundary, where eml = 0. Thus the maximum of eml is zero and occurs onthe boundary. Similarly for the minimum. Thus eml ≡ 0 for all m, l, which implies vml = wml for all m, l.
The equations (4.2) represent a linear system for vml, which we can write as Av = f . Since Ae = 0 hasthe unique solution e = 0, we conclude that detA 6= 0 and thus Av = f has an unique solution.
Convergence
Next we study convergence, namely does vml → u(xm, yl) as h→ 0? Let
∆hv ≡vm+1,l − 2vml + vm−1,l
h2+vm,l+1 − 2vml + vm,l−1
h2.
35
36 CHAPTER 4. ELLIPTIC EQUATIONS
To prove convergence we need two results. The first is analogous to the argument above:
∆hv ≥ 0⇒ maxΩh
vml = max∂Ωh
vml
and is called The Discrete Maximum Principle. Second, we establish the inequality
‖v‖∞,Ωh ≤18‖∆hv‖∞,Ωh ,
where Ω is the unit square, vml = 0 on ∂Ωh and ‖v‖∞,Ωh ≡ max(m,l)∈Ωh |vml|. The constant 18 is connected
with the shape of the domain. Start with the obvious
−‖f‖∞,Ωh ≤ fml ≤ ‖f‖∞,Ωh . (4.3)
Definewml =
14
[(xm − 1
2
)2 +(yl − 1
2
)2].
Check that (a) ∆hwml = 1, and (b) wml ≤ 18 on ∂Ωh (done in class).
Rewrite (4.3) as−‖f‖∞,Ωh∆hwml ≤ fml ≤ ‖f‖∞,Ωh∆hwml.
Therefore
∆h(v + ‖f‖∞,Ωhw) ≥ 0 (4.4)∆h(‖f‖∞,Ωhw − v) ≥ 0. (4.5)
Now from the discrete maximum principle we have
maxΩh
(vml + ‖f‖∞,Ωhwml) ≤ max∂Ωh
(vml + ‖f‖∞,Ωhwml).
Then
vml ≤ vml + ‖f‖∞,Ωhwml since w ≥ 0≤ max
Ωh(vml + ‖f‖∞,Ωhwml)
≤ max∂Ωh
(vml + ‖f‖∞,Ωhwml)
≤ 18‖f‖∞,Ωh since vml = 0 on ∂Ωh and wml ≤
18
on ∂Ωh.
From (4.5) (in class) we get − 18‖f‖∞,Ωh ≤ vml, therefore |vml| ≤ 1
8‖f‖∞,Ωh and we have
maxΩh|vml| ≤
18‖f‖∞,Ωh =
18‖∆hv‖∞,Ωh .
Talyor series implies
∆huml = (∆u)xm,yl +h2
12(∂4xu(xm + θmlh, yl) + ∂4
yu(xm, yl + θ′mlh) = fml +h2
12(∂4xu+ ∂4
yu)︸ ︷︷ ︸truncation error
.
As usual define eml =computed − exact = vml − uml, then
∆heml = ∆hvml −∆huml
= fml −[fml +
h2
12(∂4xu+ ∂4
yu)]
= −h2
12(∂4xu+ ∂4
yu).
4.1. STEADY-STATE HEAT EQUATION 37
Note that eml = 0 on ∂Ωh, thus
maxΩh|eml| ≤
18
maxΩh|∆heml|
≤ h2
8 · 12·max
Ωh|∂4xu+ ∂4
yu|
≤ h2
48·max
Ω(|∂4
xu(x, y)|, |∂4yu(x, y)|).
So the error goes to 0 as h→ 0. Remember that the constant 18 was married to the unit square.
The continuous case
We will now prove that the equation uxx + uyy = 0 with u(x, y) = 0 on ∂Ω has a unique solution u ≡ 0. Westart with the theorem of Gauss ∫
Ω
uxidx =∫∂Ω
uνids,
where νi is the normal in the ith direction. Since uuxx + uuyy = 0 we have (uux)x + (uuy)y − u2x − u2
y = 0,which we integrate over Ω using Gauss’ theorem to obtain∫
∂Ω
(uuxν1 + uuyν2)−∫
Ω
(u2x + u2
y) = 0.
The first integral is 0, since u = 0 on ∂Ω, therefore ux = uy = 0 on Ω, which along with u = 0 on ∂Ω impliesu ≡ 0 on Ω.
38 CHAPTER 4. ELLIPTIC EQUATIONS
4.2 Numerical methods for uxx + uyy = f
We have
vml −14
(vm−1,l + vm+1,l + vm,l−1 + vm,l+1) = −h2
4fml.
Where 0 ≤ m, l ≤ N . The above represents a linear system with (N − 1)2 unknowns vml, 1 ≤ m, l ≤ N − 1.Write it as Av = b. Solving Av = b directly using Gaussian elimination would cost O(((N −1)2)3) = O(N6),which is prohibitive. We’d better use the banded structure of the system. For example when N = 4 withzero boundary conditions we have
1 −1/4 −1/4−1/4 1 −1/4 −1/4
−1/4 1 −1/4−1/4 1 −1/4 −1/4
−1/4 −1/4 1 −1/4 −1/4−1/4 −1/4 1 −1/4
−1/4 1 −1/4−1/4 −1/4 1 −1/4
−1/4 −1/4 1
·
v11
v12
v13
v21
v22
v23
v31
v32
v33
= −h
2
4·
f11
f12
f13
f21
f22
f23
f31
f32
f33
(4.6)
or
Av = b.
The idea in Jacobi, Gauss–Seidel, and SOR is to split A = B + C and solve (B + C)v = b iteratively asvn+1 = B−1(b− Cvn), where n is the iteration, not the time. The error e = vn − v (= computed − exact)satisfies
en+1 = B−1Cen
or en+1 = Fen for short.
Lemma 1. Say en+1 = Fen for n = 1, 2 . . .. Then en → 0 if and only if ρ(F ) < 1, where
ρ(F ) = maxj|λj(F )|
is the spectral radius of F .
Proof. If ρ(F ) ≥ 1 and (say) |λ1| = ρ(F ), then by picking e1 as the eigenvector of F corresponding to λ1 wehave en+1 = λn1 e
1 which will never converge to 0.Now assume ρ(F ) < 1 and let F = T−1JT be the eigenvalue decomposition of F , where
J =
λ1 c1
λ2 c2. . . . . .
λk
and ci = 0 or 1 depending on whether there is a Jordan block. The choice of ones as superdiagonal elements ofJordan blocks is is a matter of convention, we can put any positive number there, since if S = diag(ε, . . . , εk),
4.3. JACOBI, GAUSS–SEIDEL, AND SOR(ω) 39
then
J = S−1JS
=
ε−1
ε−2
. . .ε−k
λ1 c1
λ2 c2. . . . . .
λk
ε1
ε2
. . .εk
=
λ1 εc1
λ2 εc2. . . . . .
λk
.Selecting ε = |1− ρ(F )|/2 guarantees that
maxj
(|λj |+ ε|cj |) < 1− ε < 1,
i.e., ‖J‖∞ < 1− ε. Now F = TSJ(TS)−1 and Fn = TSJn(TS)−1, which implies
‖en+1‖∞ = ‖TSJn(TS)−1e1‖∞≤ ‖TS‖∞ · ‖J‖n∞ · ‖(TS)−1‖∞ · ‖e1‖∞≤ ‖TS‖∞ · (1− ε)n · ‖(TS)−1‖∞ · ‖e1‖∞ → 0 as n→∞.
4.3 Jacobi, Gauss–Seidel, and SOR(ω)
All start with an initial guess and iterate
Jacobi vn+1ml =
14
(vnm+1,l + vnm−1,l + vnm,l+1 + vnm,l−1)− h2
4fml;
Gauss–Seidel vn+1ml =
14
(vnm+1,l + vn+1m−1,l + vnm,l+1 + vn+1
m,l−1)− h2
4fml;
SOR(ω) vn+1ml = (1− ω)vnml +
ω
4(vnm+1,l + vn+1
m−1,l + vnm,l+1 + vn+1m,l−1)− ωh2
4fml.
In matrix form we write the system (4.6) Av = b as
(I − L− U)v = b,
where L (U) is strictly lower (upper) triangular. Then
Jacobi vn+1 = (L+ U)vn + b
Gauss–Seidel vn+1 = (I − L)−1(Uvn + b)
SOR(ω) vn+1 =(
1ω I − L
)−1 (( 1ω I − I + U)vn + b
).
The convergence of each method will be determined by looking at
maxj|λj(L+ U)| < 1
maxj|λj((I − L)−1U)| < 1
maxj
∣∣λj (( 1ω I − L)−1( 1
ω − I + U))∣∣ < 1
40 CHAPTER 4. ELLIPTIC EQUATIONS
Convergence of Jacobi’s method
We must find the eigenvalues of L+ U , i.e., solve
vm+1,l + vm−1,l + vm,l+1 + vm,l−1 = 4λvml.
A direct verification (exercise in manipulating trigonometric functions) confirms that
vml = sinamπ
N· sin blπ
N
satisfy the above equation for any a, b = 1, 2, . . . , N − 1. Thus we have found the (N − 1)2 orthogonaleigenvectors. The eigenvalues are
λab =12
(cos
aπ
N+ cos
bπ
N
).
The spectral radius is now
ρ(L+ U) = cosπ
N≈ 1− 1
2
( πN
)2
.
Convergence of Gauss–Seidel method
We must find the eigenvalues of (I − L)−1U , i.e., solve
vm+1,l + λvm−1,l + vm,l+1 + λvm,l−1 = 4λvml.
The trick is to now set vml = λ(m+l)/2wml (we may lose a zero eigenvalue this way, but a zero eigenvaluewill not hinder convergence; it will turn out later there were no zero eigenvalues). Then
wm+1,l + wm−1,l + wm,l+1 + wm,l−1 = 4√λvml.
We already solved this problem for Jacobi. Therefore
ρ((I − L)−1U) = cos2 π
N≈ 1− π2
N2.
Gauss–Seidel converges about twice as fast as Jacobi. To compare convergence look at the number ofsteps it will take to decrease the error by a factor of e (think, e.g., e = 102 or e = 104, etc.). For Jacobi,(1− π2
2N2 )j = e−1 implies j log(1− π2
2N2 ) = log e−1. Since log(1−ρ) ≈ −ρ for small ρ, we have j ≈ 2N2
π2 · log e,whereas for Gauss–Seidel (1 − π2
N2 )g = e−1 implies g ≈ N2
π2 · log e, i.e., Gauss–Seidel converges twice as fastas Jacobi.
Convergence of SOR(ω)
Using the same substitution as in Gauss–Seidel we obtain
wm+1,l + wm−1,l + wm,l+1 + wm,l−1 = 4λ+ ω − 1ω√λ
wml,
i.e., µ = λ+ω−1√λω
is an eigenvalue of L + U . (Do not confuse ω with wml). Selecting ω to minimizeρ(( 1ω I − L)−1( 1
ω I − I + U))
we obtain
ωopt =2
1 +√
1− µ2=
21 + sin π
N
ρ(
( 1ωopt
I − L)−1( 1ωopt
I − I + U))
=cos2 π
N
(1 + sin πN )2 ≈ 1− 2π
N,
i.e., SOR(ω) is N times faster than Gauss–Seidel.
Index
L2 norm, 6
ADI methods, 31boundary conditions, 33
amplification factor, 7
consistency, 5convergence, 6
dispersion, 13dissipation, 11Douglas–Rachford, 31
Fourier Analysis, 6
Gauss–Seidel method, 39convergence, 40
group velocity, 15
Jacobi’s method, 39convergence, 40
Mitchell–Fairweather, 32
Parseval identity, 6phase speed, 14
schemeheat equation
Crank–Nicolson, 25Du Fort–Frankel, 23, 25Lax–Friedrichs, 25Lax–Wendroff, 25Leap-frog, 25
wave equationforward time centered space, 8Lax-Wendroff, 8Leap-frog, 9
SOR(ω), 39convergence, 40
stability, 6
Von Neumann analysis, 7
wave packet, 15well-posedness, 6
41