Numerical Methods for Differential Equations
Chapter 4: Two-point boundary value problems
Gustaf Soderlind and Carmen Arevalo
Numerical Analysis, Lund University
Textbooks: A First Course in the Numerical Analysis of Differential Equations, by Arieh Iserles
and Introduction to Mathematical Modelling with Differential Equations, by Lennart Edsberg
c© Gustaf Soderlind, Numerical Analysis, Mathematical Sciences, Lund University, 2008-09
Numerical Methods for Differential Equations – p. 1/86
Chapter 4: contents
◮ Finite difference approximation of derivatives
◮ Finite difference methods for the 2p-BVP
◮ Newton’s method
◮ Sturm–Liouville problems
◮ Toeplitz matrices
◮ Convergence: Lax’ equivalence theorem
◮ Differential operators
◮ From finite differences to finite elements
Numerical Methods for Differential Equations – p. 2/86
1. Approximation of derivatives ( y′ = dy/dx)
First order approximations
Forward difference
y′(x) =y(x + ∆x) − y(x)
∆x+ O(∆x)
Backward difference
y′(x) =y(x) − y(x − ∆x)
∆x+ O(∆x)
Numerical Methods for Differential Equations – p. 3/86
Approximation of derivatives . . .
Second order approximations
Symmetric difference quotients
y′(x) =y(x + ∆x) − y(x − ∆x)
2∆x+ O(∆x2)
y′′(x) =y(x + ∆x) − 2y(x) + y(x − ∆x)
∆x2+ O(∆x2)
Numerical Methods for Differential Equations – p. 4/86
Derivatives → finite differences → matrices
Matrix representation of forward difference
y′(x) =y(x + ∆x) − y(x)
∆x+ O(∆x)
Introduce vectors y = {y(xi)} and y′ = {y′(xi)} :
y′0
y′1
...
y′N
≈ 1
∆x
−1 1
−1 1. . . . . .
−1 1
y0
y1
...
yN+1
Numerical Methods for Differential Equations – p. 5/86
Derivatives . . . matrices
Note Forward difference ∼ (N + 1) × (N + 2) matrix
y′0
y′1
...
y′N
≈ 1
∆x
−1 1
−1 1. . . . . .
−1 1
y0
y1
...
yN+1
Nullspace spanned by y = (1 1 1 . . . 1)T
Compare nullspace of d/dx : y = 1 ⇒ y′ ≡ 0
Analogous result for backward difference
Numerical Methods for Differential Equations – p. 6/86
Derivatives . . . matrices
Central difference
y′(x) ≈ y(x + ∆x) − y(x − ∆x)
2∆x
Matrix representation
y′1
y′2
...
y′N
≈ 1
2∆x
−1 0 1. . . . . . . . .
−1 0 1
y0
y1
...
yN+1
Numerical Methods for Differential Equations – p. 7/86
Derivatives . . . matrices
Note N × (N + 2) matrix
y′1
y′2
...
y′N
≈ 1
2∆x
−1 0 1. . . . . . . . .
−1 0 1
y0
y1
...
yN+1
Nullspace is two-dimensional:y = (1 1 1 . . . 1)T and y = (1 −1 1 −1 . . . 1)T
Numerical Methods for Differential Equations – p. 8/86
Derivatives . . . matrices
“False” nullspace y = (1 −1 1 −1 . . . 1)T does notconverge to a C1 function!
Compare difference equation yn+1 − yn−1 = 0, withcharacteristic equation
z2 − 1 = 0 ⇒ z = ±1
and solutions yn = 1 and yn = (−1)n
Numerical Methods for Differential Equations – p. 9/86
2nd order derivatives → matrices
Central difference
y′′(x) ≈ y(x + ∆x) − 2y(x) + y(x − ∆x)
∆x2
y′′1
y′′2
...
y′′N
≈ 1
∆x2
1 −2 1. . . . . . . . .
1 −2 1
y0
y1
...
yN+1
Note N × (N + 2) matrix with nullspace y = (1 1 . . . 1)T
and y = (0 1 2 3 . . . N + 1)T
Numerical Methods for Differential Equations – p. 10/86
2nd order derivatives . . .
Nullspace of d2/dx2 :y = 1 and y = x both have y′′ ≡ 0
Compare difference equation yn+1 − 2yn + yn−1 = 0, withcharacteristic equation
z2 − 2z + 1 = 0 ⇒ z = 1, 1
and solutions yn = 1 and yn = n , respectively
Numerical Methods for Differential Equations – p. 11/86
Numerical differentiation
First and second derivatives of y = sin πx
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.5
1Data function sin(pi*x) and sampled points for h=1/6
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−4
−2
0
2
4First derivative, fwd diff (+), bwd diff (x), symmetric diff (o)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−10
−5
0Second derivative, symmetric 2nd diff (o)
Numerical Methods for Differential Equations – p. 12/86
2. Finite difference methods for 2p-BVP
Consider simplest problem
y′′ = f(x, y)
y(0) = α; y(1) = β
Introduce equidistant grid with ∆x = 1/(N + 1)
Discretization
yi+1 − 2yi + yi−1
∆x2= f(xi, yi)
y0 = α; yN+1 = β
Numerical Methods for Differential Equations – p. 13/86
Discrete 2pBVP
yi+1 − 2yi + yi−1
∆x2= f(xi, yi) i = 1 : N
y0 = α; yN+1 = β
This is a (nonlinear) system of equations F (y) = 0 for theN unknowns y1, y2, . . . , yN
Solve F (y) = 0 using Newton’s method
Numerical Methods for Differential Equations – p. 14/86
Equation system F (y) = 0
F1(y) =α − 2y1 + y2
∆x2− f(x1, y1)
Fi(y) =yi−1 − 2yi + yi+1
∆x2− f(xi, yi)
FN (y) =yN−1 − 2yN + β
∆x2− f(xN , yN)
Note how boundary values enter
Numerical Methods for Differential Equations – p. 15/86
Jacobian matrix
F ′(y) = tridiag (1/∆x2 , −2/∆x2 +∂f
∂yi
, 1/∆x2)
is tridiagonal , and
◮ Super- and subdiagonal elements 1/∆x2
◮ Diagonal elements −2/∆x2 − ∂f/∂yi
◮ Sparse LU decomposition runs in O(N) time
◮ Solution effort moderate even when N is large
Numerical Methods for Differential Equations – p. 16/86
3. Newton’s method (recap)
Let y(k) be an approximation to the root y
Taylor series expansion
0 = F (y) ≈ F (y(k)) + F ′(y(k)) · (y − y(k))
Define y(k+1) by
0 =: F (y(k)) + F ′(y(k)) · (y(k+1) − y(k))
Numerical Methods for Differential Equations – p. 17/86
Newton’s method for F (y) = 0
Newton iteration
1. Compute Jacobian F ′(y(k)) = {∂Fi/∂yj}2. Factorize Jacobian matrix F ′(y(k)) → LU
3. Solve linear system LUδy(k) = −F (y(k))
4. Update y(k+1) := y(k) + δy(k)
Newton’s method is quadratically convergent
Numerical Methods for Differential Equations – p. 18/86
Quadratic convergence
Newton’s method converges if
1. ‖F ′(y(k))−1‖ ≤ C ′
2. ‖F ′′(y(k))‖ ≤ C ′′
3. ‖y(0) − y‖ < ε (close enough starting value)
Then convergence is quadratic
‖y(k+1) − y‖ ≤ C · ‖y(k) − y‖2
Numerical Methods for Differential Equations – p. 19/86
Boundary Conditions come in many shapes
◮ “Dirichlet” boundary conditionsy(0) = α straightforward to implement
◮ “Neumann” boundary conditionsy′(0) = γ requires special attention
◮ “Robin” conditionsy(0) + c · y′(0) = κ requires same attention
for the method’s convergence order to be preserved
Numerical Methods for Differential Equations – p. 20/86
Neumann problem
Exampley′′ = f(x, y)
y(0) = α; y′(1) = β
Equidistant grid, with x = 1 between grid points!
xN + ∆x/2 = 1 = xN+1 − ∆x/2
y′(1) = β → yN+1 − yN
∆x= β
⇒ yN+1 := β ∆x + yN is of second order at x = 1
Numerical Methods for Differential Equations – p. 21/86
Robin problem
Exampley′′ = f(x, y)
y(0) = α; y(1) + c · y′(1) = κ
Equidistant grid, with x = 1 between grid points!
y(1) + cy′(1) = κ → yN+1 + yN
2+ c
yN+1 − yN
∆x= κ
⇒ yN+1 :=(2c − ∆x)yN + 2κ∆x
2c + ∆x
Numerical Methods for Differential Equations – p. 22/86
4. Sturm–Liouville eigenvalue problems
Diffusion problem
∂u
∂t=
∂
∂x
(
p(x)∂u
∂x
)
= 0 ; y(a) = 0 , y(b) = 0
Separation of variables (one space dimension)
u(t, x) := y(x) · v(t) ⇒ v
v=
(p(x) y′(x))′
y=: λ
Sturm–Liouville eigenvalue problem
d
dx
(
p(x)dy
dx
)
− λy = 0 ; y(a) = 0 , y(b) = 0
Numerical Methods for Differential Equations – p. 23/86
Sturm–Liouville eigenvalue problem. . .
Wave equation
∂2u
∂t2= c2 ∂2u
∂x2; y(a) = . . . , y(b) = . . .
Express solution as u(t, x) = y(x) eiωt ⇒
−ω2y = c2y′′
Sturm–Liouville eigenvalue problem
y′′ = λy with λ = −ω2/c2
Numerical Methods for Differential Equations – p. 24/86
Sturm–Liouville eigenvalue problem. . .
Find eigenvalues λ and eigenfunctions y(x) with
d
dx
(
p(x)dy
dx
)
= λy ; y(a) = 0 , y(b) = 0
Discretization Matrix eigenvalue problem
T∆x y = λ∆x y
Note Analytic eigenvalue problem converts to algebraic!
Numerical Methods for Differential Equations – p. 25/86
Sturm–Liouville problem. Discretization
pi−1/2yi−1 − (pi−1/2 + pi+1/2)yi + pi+1/2yi+1
∆x2= λ∆xyi
y0 = yN+1 = 0
Symmetric tridiagonal N × N eigenvalue problem
T∆x y = λ∆x y
There are N eigenvalues λ∆x,n = λn + O(∆x2)
Numerical Methods for Differential Equations – p. 26/86
Sturm–Liouville problem. Simple example
Exampley′′ = λy
y(0) = y(1) = 0
Analytic solution
y(x) = A sin√−λx + B cos
√−λx
Boundary values ⇒ B = 0 and
A sin√−λ = 0
A 6= 0 ⇒√
−λn = nπ n = 1, 2, . . .
Numerical Methods for Differential Equations – p. 27/86
Simple Sturm–Liouville example . . .
y′′ = λy
y(0) = y(1) = 0
Eigenvalues λn = −n2π2 , n = 1, 2, . . .
Eigenfunctions yn(x) = sin nπx
Numerical Methods for Differential Equations – p. 28/86
Discrete Sturm–Liouville. Same example
Discretization of y′′ = λy with BVs ⇒
yi−1 − 2yi + yi+1
∆x2= λ∆xyi
y0 = yN+1 = 0 ; ∆x = 1/(N + 1)
Tridiagonal N × N matrix formulation
1
∆x2
−2 1
1 −2 1. . .
1 −2
y1
y2
...
yN
= λ∆x
y1
y2
...
yN
Numerical Methods for Differential Equations – p. 29/86
Discrete Sturm–Liouville . . .
Algebraic eigenvalue problem
T∆x y = λ∆x y
Smallest eigenvalue λ∆x = −π2 + O(∆x2)
The first few eigenvalues are well approximated, but theapproximation gradually gets worse
Numerical Methods for Differential Equations – p. 30/86
Discrete Sturm–Liouville. ComputationFirst three (N =19) eigenvectors of T∆x
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.4
−0.2
0
0.2
0.4
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.4
−0.2
0
0.2
0.4
Numerical Methods for Differential Equations – p. 31/86
Discrete Sturm–Liouville. Eigenvalues N = 19
Discrete eigenvalues λ∆x −9.8493 −39.1548 −87.1948
Exact eigenvalues λ −9.8696 −39.4784 −88.8264
Relative errors 0.21% 0.82% 1.63%
◮ Lowest eigenvalues are more accurate
◮ Good approximations for√
N first eigenvalues
(Here approximately first 4 – 5 modes)
Numerical Methods for Differential Equations – p. 32/86
Discrete Sturm–Liouville. High modesEigenvectors 7, 13, 19 (N =19) of T∆x
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.4
−0.2
0
0.2
0.4
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.4
−0.2
0
0.2
0.4
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.4
−0.2
0
0.2
0.4
Numerical Methods for Differential Equations – p. 33/86
Discrete Sturm–Liouville. High modesEigenvectors 7, 13, 19 (N =19) of T∆x
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.4
−0.2
0
0.2
0.4
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.4
−0.2
0
0.2
0.4
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−0.4
−0.2
0
0.2
0.4
Numerical Methods for Differential Equations – p. 34/86
5. Toeplitz matrices
A Toeplitz matrix is constant along diagonals
Example (symmetric)
T∆x =1
∆x2
−2 1 0 . . .
1 −2 1
1 −2 1. . .
. . . 0 1 −2
Numerical Methods for Differential Equations – p. 35/86
Toeplitz matrices . . .
Much is known about Toeplitz matrices
◮ Eigenvalues
◮ Norms
◮ Inverses
◮ etc.
They can be generated in MATLAB using the built-infunction toeplitz
Numerical Methods for Differential Equations – p. 36/86
Eigenvalues of Toeplitz matrices
Example Solve the eigenvalue problem Ty = λy for
T =
−2 1 0 . . .
1 −2 1
1 −2 1. . .
. . . 0 1 −2
Note λ[T ] = −2 + λ[S]
Numerical Methods for Differential Equations – p. 37/86
Eigenvalues . . .
. . . the problem gets simplified
Sy =
0 1 0 . . .
1 0 1
1 0 1. . . 1
. . . 0 1 0
y1
y2
...
yN
= λy
Find the eigenvalues of S!
Numerical Methods for Differential Equations – p. 38/86
Eigenvalues . . . and difference equations!
Consider the nth equation of Sy = λy :
yn+1 + yn−1 = λyn
Linear difference equation with boundary values
y0 = 0; yN+1 = 0
Characteristic equation
z2 − λz + 1 = 0
Numerical Methods for Differential Equations – p. 39/86
Eigenvalues . . . characteristic equation
Roots of z2 − λz + 1 = 0 are z and 1/z (product 1)
General solution yn = αzn + βz−n
Numerical Methods for Differential Equations – p. 40/86
Eigenvalues . . . characteristic equation
Roots of z2 − λz + 1 = 0 are z and 1/z (product 1)
General solution yn = αzn + βz−n
Boundary condition y0 = 0 = α + β ⇒
Solution yn = α(zn − z−n)
Numerical Methods for Differential Equations – p. 40/86
Eigenvalues . . . characteristic equation
Roots of z2 − λz + 1 = 0 are z and 1/z (product 1)
General solution yn = αzn + βz−n
Boundary condition y0 = 0 = α + β ⇒
Solution yn = α(zn − z−n)
Boundary conditionyN+1 = 0 = α(zN+1 − z−(N+1)) ⇒ z2(N+1) = 1
Numerical Methods for Differential Equations – p. 40/86
Eigenvalues . . . characteristic equation
Roots of z2 − λz + 1 = 0 are z and 1/z (product 1)
General solution yn = αzn + βz−n
Boundary condition y0 = 0 = α + β ⇒
Solution yn = α(zn − z−n)
Boundary conditionyN+1 = 0 = α(zN+1 − z−(N+1)) ⇒ z2(N+1) = 1
Roots zk = exp( kπi
N + 1
)
k = 1 : N
Numerical Methods for Differential Equations – p. 40/86
Eigenvalues . . .
Sum of the roots of z2 − λz + 1 = 0 are
λk = zk + 1/zk ⇒
λk[S] = exp( kπi
N + 1
)
+ exp(
− kπi
N + 1
)
= 2 coskπ
N + 1
Numerical Methods for Differential Equations – p. 41/86
Eigenvalues . . .
Sum of the roots of z2 − λz + 1 = 0 are
λk = zk + 1/zk ⇒
λk[S] = exp( kπi
N + 1
)
+ exp(
− kπi
N + 1
)
= 2 coskπ
N + 1
Hence
λk[T ] = − 2 + 2 coskπ
N + 1= −4 sin2 kπ
2(N + 1)
Numerical Methods for Differential Equations – p. 41/86
Eigenvalues of Toeplitz matrices
Theorem The N × N Toeplitz matrix
T =
−2 1 0 . . .
1 −2 1
1 −2 1. . .
. . . 0 1 −2
has N real eigenvalues ( k = 1 : N )
λk[T ] = −4 sin2 kπ
2(N + 1)∈ (−4, 0)
Numerical Methods for Differential Equations – p. 42/86
Eigenvalues of Toeplitz matrices . . .
Consider the operator approximation
d2
dx2↔ 1
∆x2T
on x ∈ [0, 1], with ∆x = 1/(N + 1)
Corollary The eigenvalues of T∆x := T/∆x2 are
λk[T∆x] = −4(N + 1)2 sin2 kπ
2(N + 1)≈ −k2π2
for k ≪ N
Numerical Methods for Differential Equations – p. 43/86
What are the eigenvalues of d2/dx2 on [0, 1]?
Consider the Sturm–Liouville problem
u′′ = λu ; u(0) = u(1) = 0
Solutions u(x) = A sin√−λ x + B cos
√−λ x
Boundary conditions B = 0 and A sin√−λ = 0
Numerical Methods for Differential Equations – p. 44/86
What are the eigenvalues of d2/dx2 on [0, 1]?
Consider the Sturm–Liouville problem
u′′ = λu ; u(0) = u(1) = 0
Solutions u(x) = A sin√−λ x + B cos
√−λ x
Boundary conditions B = 0 and A sin√−λ = 0
Theorem The eigenvalues are λk[d2/dx2] = −k2π2
Note k ∈ Z+
Numerical Methods for Differential Equations – p. 44/86
What are the norms of T?
Lemma For a symmetric matrix A, it holds
‖A‖2 = maxk
|λk|
Numerical Methods for Differential Equations – p. 45/86
What are the norms of T?
Lemma For a symmetric matrix A, it holds
‖A‖2 = maxk
|λk|
Lemma For a symmetric matrix A, it holds
µ2[A] = maxk
λk
(Both results actually hold for normal matrices)
Numerical Methods for Differential Equations – p. 45/86
Proofs: Norm
Definition
‖A‖22 = max
xTx6=0
xTATAx
xTx
Find stationary points of the Rayleigh quotient of ATA,given by ρ(x) = xTATAx/xTx
gradxρ(x) = (2ATAxxTx − 2xxTATAx)/(xTx)2 := 0
ATAx = ρ(x)x ⇒ A2x = ρ(x)x
Numerical Methods for Differential Equations – p. 46/86
Proofs: Norm . . .
So ρ(x) = λ2, where λ is an eigenvalue of A
Therefore ‖A‖22 = max |λ[A]|2 or
‖A‖2 = max |λ[A]|
when A is symmetric
Numerical Methods for Differential Equations – p. 47/86
Proofs: Logarithmic norm
Definition
µ2[A] = maxxTx6=0
xTAx
xTx
Find stationary points of the Rayleigh quotient of A, givenby ρ(x) = xTAx/xTx
gradxρ(x) = [(A + AT)xxTx − 2xxTAx]/(xTx)2 := 0
1
2(A + AT)x = ρ(x)x ⇒ Ax = ρ(x)x
Numerical Methods for Differential Equations – p. 48/86
Proofs: Logarithmic norm . . .
So ρ(x) = λ, where λ is an eigenvalue of A
Therefore µ2[A] = maxλ[A] when A is symmetric.
For symmetric matrices we have proved
‖A‖2 = maxk
|λk| µ2[A] = maxk
λk
Numerical Methods for Differential Equations – p. 49/86
What are the norms of T∆x?
Eigenvalues of T∆x = T/∆x2 are
λk[T∆x] = −4(N + 1)2 sin2 kπ
2(N + 1)
So ‖T∆x‖2 = |λN | and µ2[T∆x] = λ1
Theorem The Euclidean norms of T∆x are
‖T∆x‖2 ≈4
∆x2µ2[T∆x] ≈ −π2
Numerical Methods for Differential Equations – p. 50/86
The norm of T−1∆x
Recall that µ[A] < 0 ⇒ ‖A−1‖ ≤ −1/µ[A]
Approximate y′′ = f(x) with y(0) = y(1) = 0 by
T∆xu = q
Note that µ2[T∆x] ≈ −π2 then implies that this problem hasa unique solution, as
‖T−1∆x‖2 /
1
π2
Numerical Methods for Differential Equations – p. 51/86
What norms to use. RMS and Euclidean norms
The norm of a function is measured in the L2 norm
‖l‖22 =
∫ 1
0
l(x)2dx
A corresponding discrete function (vector) is thenmeasured in the root mean square (RMS) norm
‖l(x)‖2∆x =
N∑
1
l(xi)2∆x =
1
N + 1
N∑
1
l(xi)2 =
1
N + 1‖l(x)‖2
2
Important! Note that ‖T−1∆x‖∆x ≡ ‖T−1
∆x‖2
Numerical Methods for Differential Equations – p. 52/86
Solution and error bounds
Approximate y′′ = f(x) with y(0) = y(1) = 0 by
T∆xu = q
Convergence analysis will show
Theorem µ2[T∆x] ≈ −π2 implies a unique solution with
‖u‖∆x /‖q‖∆x
π2‖e(x)‖∆x /
‖l(x)‖∆x
π2
Solution–data Global–local error
Numerical Methods for Differential Equations – p. 53/86
6. Convergence of finite difference methods
Basic problem
y′′ = f(x, y)
y(0) = α; y(1) = β
Equidistant discretization
yi+1 − 2yi + yi−1
∆x2= f(xi, yi)
y0 = α; yN+1 = β
Numerical Methods for Differential Equations – p. 54/86
Error definitions
Local error
Insert exact solution y(x) into discretization:
y(xi−1) − 2y(xi) + y(xi+1)
∆x2= f(xi, y(xi))−l(xi)
Taylor expansion, using f(xi, y(xi)) = y′′(xi):
−l(xi) = 2
(
∆x2
4!y(4)(xi) +
∆x4
6!y(6)(xi) + . . .
)
Only even powers of ∆x due to symmetry
Numerical Methods for Differential Equations – p. 55/86
Global error
Definition The global error is defined e(xi) = yi − y(xi)
Convergence Will show that e(x) → 0 as ∆x → 0, ormore specificallly
e(xi) = c1∆x2 + c2∆x4 + . . .
Again only even powers due to symmetry
Numerical Methods for Differential Equations – p. 56/86
Convergence
Assume f is a linear function, ⇒ F is linear ⇒
F (y) = 0 ⇔ T∆xy = q
with T∆x tridiagonal. Then
Numerical solution T∆xy = q
Exact solution T∆xy(x) = q − l(x)
Numerical Methods for Differential Equations – p. 57/86
Convergence . . .
Solve T∆xy = q formally to get
Numerically y = T−1∆x · q
Exact y(x) = T−1∆x · (q − l(x))
Global error e(x) = T−1∆x · l(x)
Error bound ‖e(x)‖∆x ≤ ‖T−1∆x‖2 · ‖l(x)‖∆x
Numerical Methods for Differential Equations – p. 58/86
Matching norms: RMS and Euclidean norms
Note how root mean square (RMS) norm of vector
‖l(x)‖2∆x =
N∑
1
l(xi)2∆x =
1
N + 1
N∑
1
l(xi)2 =
1
N + 1‖l(x)‖2
2
is matched to the Euclidean matrix norm, by way of
‖T−1∆x‖∆x ≡ ‖T−1
∆x‖2
(Prove this relation!)
Numerical Methods for Differential Equations – p. 59/86
Convergence . . .
Recall µ2[T∆x] ≈ −π2 ⇒ ‖T−1∆x‖2 / 1/π2 (∆x→0)
Since ‖e(x)‖∆x ≤ ‖T−1∆x‖2 · ‖l(x)‖∆x and
‖l‖∆x = γ1∆x2 + γ2∆x4 . . . we have
‖e‖∆x ≤ C · ‖l‖∆x = c1∆x2 + c2∆x4 + . . .
. . . and we have convergence as ∆x→0! :)
Numerical Methods for Differential Equations – p. 60/86
Convergence: Lax’ principle
Conclusion
Consistency: local error l → 0 as ∆x → 0
Stability: ‖T−1∆x‖2 ≤ C as ∆x → 0
Convergence: global error e → 0 as ∆x → 0
Theorem (Lax’ Principle)
Consistency + Stability ⇒ Convergence
(“Fundamental theorem of numerical analysis”)
Numerical Methods for Differential Equations – p. 61/86
7. Differential operators. Integration by parts
Logarithmic norm of matrix:
µ2[A] = maxx6=0
xTAx
xTx⇒ xTAx ≤ µ2[A] · xTx
For d2/dx2 , introduce the inner product
〈u, v〉 =
∫ 1
0
u(x)v(x) dx ⇒ ‖u‖22 = 〈u, u〉
Numerical Methods for Differential Equations – p. 62/86
The logarithmic norm of d2/dx2
Can we find a constant µ2[d2/dx2] such that
〈u, u′′〉 ≤ µ2[d2/dx2] · ‖u‖2
2
for all functions u ∈ C20 [0, 1]?
Yes and µ2[d2/dx2] = −π2
Numerical Methods for Differential Equations – p. 63/86
Integration by parts
∫ 1
0
uv′ dx = [uv]10 −∫ 1
0
u′v dx
Because u(0) = u(1) = 0, this can be written
〈u, v′〉 = −〈u′, v〉
Apply to d2/dx2 ⇒
〈u, u′′〉 = −〈u′, u′〉 = −‖u′‖22
Numerical Methods for Differential Equations – p. 64/86
Sobolev’s lemma
Lemma For all functions u with u(0) = u(1) = 0 it holdsthat
‖u′‖2 ≥ π‖u‖2
Proof Fourier analysis (Parseval’s theorem)
u =√
2∞
∑
k=1
ck sin kπx ⇒ u′ = π√
2∞
∑
k=1
kck cos kπx
implies ‖u′‖2 ≥ π‖u‖2
Numerical Methods for Differential Equations – p. 65/86
Logarithmic norm of d2/dx2 on [0, 1]
〈u, u′′〉 = −〈u′, u′〉 = −‖u′‖22 ≤ −π2‖u‖2
2
Theorem The logarithmic norm of d2/dx2 on C20 [0, 1] is
µ2[d2/dx2] = −π2
Corollary The 2pBVP u′′ = f(x); u(0) = u(1) = 0; has aunique solution with ‖u‖2 ≤ ‖f‖2/π
2
Numerical Methods for Differential Equations – p. 66/86
Self-adjoint operators
Definition
〈v,Au〉 = 〈A∗v, u〉
defines the adjoint operator A∗
Example For vectors and matrices
〈v,Au〉 = vTAu = (ATv)Tu = 〈A∗v, u〉
AT is the adjoint of A
A matrix is self-adjoint (“symmetric”) if A = AT
Numerical Methods for Differential Equations – p. 67/86
Self-adjoint differential operators . . .
Example d2/dx2 on C20 [0, 1]
Integrate by parts
〈v, u′′〉 = −〈v′, u′〉 = 〈v′′, u〉
So d2/dx2 is a self-adjoint operator
More generally,
d
dx
(
p(x)d
dx
)
+ q(x)
is self-adjoint on C20 [0, 1]
Numerical Methods for Differential Equations – p. 68/86
Eigenvalues of self-adjoint operators . . .
. . . are real: let Au = λu
λ‖u‖22 = 〈u, λu〉 = 〈u,Au〉 = 〈A∗u, u〉
= 〈Au, u〉 = 〈λu, u〉 = λ‖u‖22
So λ = λ implies real eigenvalues
Numerical Methods for Differential Equations – p. 69/86
Eigenvectors of self-adjoint operators . . .
. . . are orthogonal; let Au = λu and Av = µv
λ〈v, u〉 = 〈v,Au〉 = 〈A∗v, u〉= 〈Av, u〉 = µ〈v, u〉
So (λ − µ)〈v, u〉 = 0 ⇒ 〈v, u〉 = 0 implying that
eigenvectors are orthogonal
Numerical Methods for Differential Equations – p. 70/86
Elliptic operators
Definition An operator is elliptic if for all u 6= 0
〈u,Au〉 > 0
Example −d2/dx2 on C20 [0, 1]
Integrate by parts
−〈u, u′′〉 = 〈u′, u′〉 ≥ π2〈u, u〉
by Sobolev’s lemma
Numerical Methods for Differential Equations – p. 71/86
Elliptic operators . . .
More generally,
− d
dx
(
p(x)d
dx
)
+ q(x)
is elliptic if p(x) > 0 and q(x) ≥ 0
Example Laplace’s equation
∆u =∂2u
∂x2+
∂2u
∂y2= 0
−∆ is an elliptic operator
Numerical Methods for Differential Equations – p. 72/86
Positive definite operators
Definition An operator is positive definite if it isself-adjoint and elliptic
µ2[−A] < 0
Example −d2/dx2 as µ2[d2/dx2] = −π2 on C2
0 [0, 1]
Negative Laplacian −∆ (leads to FEM theory)
Numerical Methods for Differential Equations – p. 73/86
8. From Finite Difference to Finite Element
Start with linear differential equation
Au = f + boundary conditions
Finite Difference Method (FDM). The main idea
Replace functions u and f by vectors and differentialoperator A by matrix to get a linear system of equations
Exampled2
dx2u = f(x) ⇒ T∆xu = f
Numerical Methods for Differential Equations – p. 74/86
Finite Elements and the Galerkin method
Galerkin Method. The main idea
Approximate function u by polynomial and keep differentialoperator A as is!
Take some polynomial v satisfying boundary conditions.Insert into original equation to get Av ≈ f
Question Which polynomial gives the best approximation?
Answer Choose v to minimize the residual ‖Av − f‖2
Numerical Methods for Differential Equations – p. 75/86
Best approximation = least squares
Let {ϕj} be a polynomial basis, and make the ansatz
v(x) =
N∑
1
cjϕj(x)
Minimizing ‖Av − f‖2 is equivalent to requiring thatthe residual is orthogonal to each and every ϕi
〈ϕi,Av − f〉 = 0 ∀i
This is the least-squares approximation!
Numerical Methods for Differential Equations – p. 76/86
Best approximation. . .
As A is linear,
Av = AN
∑
j=1
cjϕj =N
∑
j=1
cjAϕj
Then
〈ϕi,Av − f〉 = 0 ⇔N
∑
j=1
〈ϕi,Aϕj〉cj = 〈ϕi, f〉
This is a linear system of equations∑
aijcj = bi or Ac = b,with matrix and vector elements
aij = 〈ϕi,Aϕj〉 bi = 〈ϕi, f〉
Numerical Methods for Differential Equations – p. 77/86
Weak formulation. The problem −u′′ = f
If −u′′ = f then for all v
〈v,−u′′〉 = 〈v, f〉
Integrate by parts and use Dirichlet boundary data to get
Weak formulation 〈v′, u′〉 = 〈v, f〉 ∀v
The weak formulation requires u to be once continuouslydifferentiable, not twice
Numerical Methods for Differential Equations – p. 78/86
Weak formulation and energy norm
Definition The energy norm is defined by
a(v, u) = 〈v′, u′〉
and the weak formulation can be written: Find a function u
such that for all test functions v it holds
a(v, u) = 〈v, f〉
What functions? Choose a polynomial space V withbasis {ϕj}, satisfying the boundary conditions, and requirev ∈ V and u ∈ V , all defined on suitable grid {xi}
Numerical Methods for Differential Equations – p. 79/86
Example. The Finite Element Method (FEM)
Given grid {xi}, choose piecewise linear basis polynomials
ϕj(xi) = 1 if i = j, otherwise 0
0 0.5 1 1.5 2 2.5 3 3.5 4−0.2
0
0.2
0.4
0.6
0.8
1
1.2
Piecewise linear interpolant v ≈ u can be written
v(x) =N
∑
j=1
cjϕj(x) Note that v(xi) = ci ≈ u(xi)
Numerical Methods for Differential Equations – p. 80/86
Galerkin Finite Element Method for −u′′ = f
Best approximation a(v, u) = 〈v, f〉, with u, v ∈ V , leads to
a(ϕi,∑
j=1
cjϕj) = 〈ϕi, f〉
which is equivalent to the finite element equation Kc = b
N∑
j=1
〈ϕ′i, ϕ
′j〉cj = 〈ϕi, f〉 ∀ϕi ∈ V
The stiffness matrix K with elements {〈ϕ′i, ϕ
′j〉}N
i,j=1 can becomputed as soon as the basis {ϕj} has been constructed
The right-hand side vector b depends on the data f
Numerical Methods for Differential Equations – p. 81/86
Equation system
Assume an equidistant grid with spacing ∆x. Note that
ϕ′i(x) = 1/∆x on [xi−1, xi]
ϕ′i(x) = −1/∆x on [xi, xi+1]
ϕ′i(x) = 0 elsewhere
Then
〈ϕ′i, ϕ
′i〉 =
∫ xi+1
xi−1
1
∆x2dx =
2
∆x
〈ϕ′i, ϕ
′i+1〉 =
∫ xi+1
xi
−1
∆x2dx =
−1
∆x
Numerical Methods for Differential Equations – p. 82/86
Stiffness matrix
For equidistant grid with spacing ∆x the stiffness matrix is
K∆x =1
∆xtridiag(−1 2 − 1)
Note
1) The stiffness matrix is K∆x = −∆x · T∆x
2) It is positive definite, therefore nonsingular
3) Smallest eigenvalue λ1[K∆x] ≈ π2∆x
Numerical Methods for Differential Equations – p. 83/86
Mass matrix
Compute RHS integrals using numerical integration
〈ϕi, f〉 ≈ 〈ϕi,N
∑
0
fjϕj〉 =1
∑
k=−1
fi+k〈ϕi, ϕi+k〉
Need to compute 〈ϕi, ϕi+k〉 =∫ xi+1
xi−1ϕi(x)ϕi+k(x) dx
The integrals are 〈ϕi, ϕi〉 = 2∆x/3 and 〈ϕi, ϕi+1〉 = ∆x/6
For equidistant grid with spacing ∆x the mass matrix is
B∆x =∆x
6tridiag(1 4 1)
Numerical Methods for Differential Equations – p. 84/86
Assembling the system of equations
Final finite element equations is a tridiagonal linear system
K∆x c = B∆xf
with stiffness matrix
K∆x =1
∆xtridiag(1 − 2 1)
and mass matrix
B∆x =∆x
6tridiag(1 4 1)
Numerical Methods for Differential Equations – p. 85/86
Advantages of the Finite Element Method
◮ Produces “continuous solution” not only on grid points
◮ Can also use basis of higher degree splines
◮ Can easily use nonuniform grids
◮ Boundary conditions built into test functions
◮ In PDEs, easy to work with complex geometries
◮ Rich theoretical foundation
Numerical Methods for Differential Equations – p. 86/86