numerical methods for two–point boundary value problems · numerical methods for two–point...

83
Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods 1.1 Introduction Consider the second order linear two–point boundary value problem Lu(x) ≡-u + p(x)u + q(x)u = f (x), x I, (1.1) u(0) = g 0 , u(1) = g 1 , (1.2) where I = [0, 1]. We assume that the functions p, q and f are smooth on I , q is positive, and p * and q * are positive constants such that |p(x)|≤ p * , 0 <q * q(x), x I. (1.3) Let π = {x j } N +1 j =0 denote a uniform partition of the interval I such that x j = jh,j =0, 1,...,N +1, and (N +1)h = 1. On this partition, the solution u of (1.1)–(1.2) is approximated by the mesh function {u j } N +1 j =0 defined by the finite difference equations L h u j ≡- u j +1 - 2u j + u j -1 h 2 + p j u j +1 - u j -1 2h + q j u j = f j , j =1,...,N, (1.4) u 0 = g 0 , u N +1 = g 1 , (1.5) where p j = p(x j ), q j = q(x j ), f j = f (x j ). Equations (1.4) are obtained by replacing the derivatives in (1.1) by basic centered difference quotients. We now show that under certain conditions the difference problem (1.4)– (1.5) has a unique solution {u j } N +1 j =0 , which is second order accurate; that is, |u(x j ) - u j | = O(h 2 ), j =1,...,N. 1

Upload: others

Post on 19-Jun-2020

13 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

Numerical Methods for Two–Point Boundary Value

Problems

Graeme Fairweather and Ian Gladwell

1 Finite Difference Methods

1.1 Introduction

Consider the second order linear two–point boundary value problem

Lu(x) ≡ −u′′ + p(x)u′ + q(x)u = f(x), x ∈ I,(1.1)

u(0) = g0, u(1) = g1,(1.2)

where I = [0, 1]. We assume that the functions p, q and f are smooth on I,q is positive, and p∗ and q∗ are positive constants such that

|p(x)| ≤ p∗, 0 < q∗ ≤ q(x), x ∈ I.(1.3)

Let π = xjN+1j=0 denote a uniform partition of the interval I such that

xj = jh, j = 0, 1, . . . , N+1, and (N+1)h = 1. On this partition, the solutionu of (1.1)–(1.2) is approximated by the mesh function ujN+1

j=0 defined bythe finite difference equations

Lhuj ≡ −uj+1 − 2uj + uj−1

h2+pj

uj+1 − uj−1

2h+qjuj = fj , j = 1, . . . , N,(1.4)

u0 = g0, uN+1 = g1,(1.5)

wherepj = p(xj), qj = q(xj), fj = f(xj).

Equations (1.4) are obtained by replacing the derivatives in (1.1) by basiccentered difference quotients.

We now show that under certain conditions the difference problem (1.4)–(1.5) has a unique solution ujN+1

j=0 , which is second order accurate; thatis,

|u(xj)− uj | = O(h2), j = 1, . . . , N.

1

Page 2: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

1.2 The Uniqueness of the Difference Approximation

From (1.4), we obtain

h2Lhuj = −(

1 +h

2pj

)uj−1+

(2 + h2qj

)uj−

(1− h

2pj

)uj+1 = h2fj , j = 1, . . . , N.

(1.6)The totality of difference equations (1.6), subject to (1.5), may be writtenin the form

Au = b,(1.7)

where

u =

u1

u2...uN−1

uN

, A =

d1 e1

c2. . . . . .. . . . . . . . .

. . . . . . eN−1

cN dN

,b = h2

f1

f2...fN−1

fN

−c1g00...0

eNg1

,

and, for j = 1, . . . , N,

cj = −(

1 +12hpj

), dj = 2 + h2qj , ej = −

(1− 1

2hpj

).(1.8)

We prove that there is a unique ujNj=1 by showing that the tridiagonal

matrix A is strictly diagonally dominant and hence nonsingular.

Theorem 1.1. If h < 2/p∗, then the matrix A is strictly diagonally domi-nant.

Proof — If h < 2/p∗ then

|cj | = 1 +h

2pj , |ej | = 1− h

2pj ,

and|cj |+ |ej | = 2 < dj , j = 2, . . . , N − 1.

Also,|e1| < d1, |cN | < dN ,

which completes the proof.

Corollary If p = 0, the matrix A is positive definite, with no restrictionon the mesh spacing h.Proof — If p = 0, then A is strictly diagonally dominant with no restrictionon the mesh spacing h. In addition, A is symmetric and has positive diagonalentries and is therefore positive definite.

2

Page 3: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

1.3 Consistency, Stability and Convergence

To study the accuracy and the computability of the difference approximationujN+1

j=0 , we introduce the concepts of consistency, stability and convergenceof finite difference methods. The basic result proved in this section is that,for a consistent method, stability implies convergence.

Definition 1.1 (Consistency). Let

τj,π[w] ≡ Lhw(xj)− Lw(xj), j = 1, . . . , N,

where w is a smooth function on I. Then the difference problem (1.4)–(1.5)is consistent with the differential problem (1.1)–(1.2) if

|τj,π[w]| → 0 as h→ 0.

The quantities τj,π[w], j = 1, . . . , N , are called the local truncation (or localdiscretization) errors.

Definition 1.2. The difference problem (1.4)–(1.5) is locally pth– order ac-curate if, for sufficiently smooth data, there exists a positive constant C,independent of h, such that

max1≤j≤N

|τj,π[w]| ≤ Chp.

The following lemma demonstrates that the difference problem (1.4)–(1.5) is consistent with (1.1)–(1.2) and is locally second–order accurate.

Lemma 1.1. If w ∈ C4(I), then

τj,π[w] = −h2

12[w(4)(νj)− 2p(xj)w(3)(θj)],

where νj and θj lie in (xj−1, xj+1).

Proof — By definition

τj,π[w] = −[w(xj+1)− 2w(xj) + w(xj−1)

h2− w′′(xj)

]+pj

[w(xj+1)− w(xj−1)

2h− w′(xj)

], j = 1, . . . , N.

(1.9)

3

Page 4: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

It is easy to show using Taylor’s theorem that

w(xj+1)− w(xj−1)2h

− w′(xj) =h2

6w(3)(θj), θj ∈ (xj−1, xj+1).(1.10)

Also,

w(xj+1)− 2w(xj) + w(xj−1)h2

− w′′(xj) =h2

12w(4)(νj), νj ∈ (xj−1, xj+1).(1.11)

The desired result now follows on substituting (1.10) and (1.11) in (1.9).

Definition 1.3 (Stability). The linear difference operator Lh is stable if,for sufficiently small h, there exists a constant K, independent of h, suchthat

|vj | ≤ Kmax(|v0|, |vN+1|) + max1≤i≤N

|Lhvi| j = 0, . . . , N + 1,

for any mesh function vjN+1j=0 .

We now prove that, for h sufficiently small, the difference operator Lh

of (1.4) is stable.

Theorem 1.2. If the functions p and q satisfy (1.3), then the differenceoperator Lh of (1.4) is stable for h < 2/p∗, with K = max1, 1/q∗.

Proof — If|vj∗| = max

0≤j≤N+1|vj |, 1 ≤ j∗ ≤ N,

then, from (1.6), we obtain

dj∗vj∗ = −ej∗vj∗+1 − cj∗vj∗−1 + h2Lhvj∗.

Thus,dj∗|vj∗| ≤ (|ej∗|+ |cj∗|) |vj∗|+ h2 max

1≤j≤N|Lhvj |.

If h < 2/p∗, thendj∗ = |ej∗|+ |cj∗|+ h2qj∗,

and it follows thath2qj∗|vj∗| ≤ h2 max

1≤j≤N|Lhvj |,

4

Page 5: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

or|vj∗| ≤

1q∗

max1≤j≤N

|Lhvj |.

Thus, if max0≤j≤N+1 |vj | occurs for 1 ≤ j ≤ N then

max0≤j≤N+1

|vj | ≤1q∗

max1≤j≤N

|Lhvj |,

and clearly

max0≤j≤N+1

|vj | ≤ Kmax(|v0|, |vN+1|) + max1≤j≤N

|Lhvj |(1.12)

with K = max1, 1/q∗. If max0≤j≤N+1 |vj | = max|v0|, |vN+1|, then(1.12) follows immediately.

An immediate consequence of stability is the uniqueness (and hence ex-istence since the problem is linear) of the difference approximation ujN+1

j=0

(which was proved by other means in earlier), for if there were two solutions,their difference vjN+1

j=0 , say, would satisfy

Lhvj = 0, j = 1, . . . , N,v0 = vN+1 = 0.

Stability then implies that vj = 0, j = 0, 1, . . . , N + 1.

Definition 1.4 (Convergence). Let u be the solution of the boundary valueproblem (1.1)–(1.2) and ujN+1

j=0 the difference approximation defined by(1.4)–(1.5). The difference approximation converges to u if

max1≤j≤N

|uj − u(xj)| → 0,

as h → 0. The difference uj − u(xj) is the global truncation (or discretiza-tion) error at the point xj , j = 1, . . . , N .

Definition 1.5. The difference approximation ujN+1j=0 is a pth–order ap-

proximation to the solution u of (1.1)–(1.2) if, for h sufficiently small, thereexists a constant C independent of h, such that

max0≤j≤N+1

|uj − u(xj)| ≤ Chp.

The basic result connecting consistency, stability and convergence isgiven in the following theorem.

5

Page 6: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

Theorem 1.3. Suppose u ∈ C4(I) and h < 2/p∗. Then the differencesolution ujN+1

j=0 of (1.4)–(1.5) is convergent to the solution u of (1.1)–(1.2). Moreover,

max0≤j≤N+1

|uj − u(xj)| ≤ Ch2.

Proof — Under the given conditions, the difference problem (1.4)–(1.5) isconsistent with the boundary value problem (1.1)–(1.2) and the operator Lh

is stable.Since

Lh[uj − u(xj)] = f(xj)− Lhu(xj) = Lu(xj)− Lhu(xj) = −τj,π[u],

and u0 − u(x0) = uN+1 − u(xN+1) = 0, the stability of Lh implies that

|uj − u(xj)| ≤1q∗

max1≤j≤N

|τj,π[u]|.

The desired result follows from Lemma 1.1.It follows from this theorem that ujN+1

j=0 is a second–order approxima-tion to the solution u of (1.1).

1.4 Experimental Determination of the Asymptotic Rate ofConvergence

If a norm of the global error is O(hp) then an estimate of p can be determinedin the following way. For ease of exposition, we use a different notation inthis section and denote by uh

j the difference approximation computed witha mesh length of h. Also let

eh ≡ ‖u− uh‖ = maxj|u(xj)− uh

j |.

If eh = O(hp), then, for h sufficiently small, h < h0 say, there exists aconstant C, independent of h, such that

eh ≈ Chp.

If we solve the difference problem with two different mesh lengths h1 and h2

such that h2 < h1 < h0, then

eh1 ≈ Chp1

6

Page 7: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

andeh2 ≈ Chp

2

from which it follows that

ln eh1 ≈ p lnh1 + lnC

andln eh2 ≈ p lnh2 + lnC.

Therefore an estimate of p can be calculated from

p ≈ ln(eh1/eh2)/ ln (h1/h2)(1.13)

In practice, one usually solves the difference problem for a sequence of valuesof h, h0 > h1 > h2 > h3 > . . ., and calculates the ratio on the right handside of (1.13) for successive pairs of values of h. These ratios converge tothe value of p as h→ 0.

1.5 Higher–Order Finite Difference Approximations

1.5.1 Richardson extrapolation and deferred corrections

Commonly used methods for improving the accuracy of approximate so-lutions to differential equations are Richardson extrapolation and deferredcorrections. These techniques are applicable if there exists an expansion ofthe local truncation error of the form

τj,π[u] = h2τ [u(xj)] +O(h4).(1.14)

As is shown in the next theorem, if (1.14) holds, then the stability of thedifference operator Lh ensures that there exists a function e(x) such that

u(xj) = uj + h2e(xj) +O(h4), j = 0, 1, . . . , N + 1.(1.15)

In Richardson extrapolation, the difference problem (1.4)–(1.5) is solvedtwice with mesh spacings h and h/2 to yield difference solutions uh

j N+1j=0 ,

and uh/2j 2(N+1)

j=0 . Then at a point x = jh = (2j)(h/2) common to bothmeshes we have, from (1.15),

u(x) = uhj + h2e(x) +O(h4)

and

u(x) = uh/22j +

h2

4e(x) +O(h4),

7

Page 8: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

from which it follows that

u(x) = uh/22j +

13(uh/2

2j − uhj ) +O(h4).

Thusu

(1)j ≡ u

h/22j +

13(uh/2

2j − uhj )

is a fourth–order approximation to u(xj), j = 0, . . . , N + 1.In deferred corrections, the difference equations (1.4)–(1.5) are solved in

the usual way to obtain uj. Then a fourth–order difference approximationujN+1

j=0 to u(x) is computed by solving difference equations which are aperturbation of (1.4)–(1.5) expressed in terms of uj. Suitable definitionsof ujN+1

j=0 are discussed once we derive (1.15).

Theorem 1.4. Suppose u ∈ C6(I), and h < 2/p∗. Then (1.14) and (1.15)hold with e(x) defined as the solution of the boundary value problem

Le(x) = τ [u(x)], x ∈ I,(1.16)

e(0) = e(1) = 0,

whereτ [u(x)] = − 1

12[u(4)(x)− 2p(x)u(3)(x)].(1.17)

Proof — Since u ∈ C6(I), it is easy to show by extending the argumentused in Lemma 1.1 that (1.14) holds with τ [u(x)] defined by (1.17).

As in the proof of Theorem 1.3, we have

Lh[u(xj)− uj ] = τj,π[u], j = 1, . . . , N.(1.18)

With (1.14) in (1.18) and using (1.16), we obtain

Lh[u(xj)− uj ] = h2τ [u(xj)] +O(h4)(1.19)= h2Le(xj) +O(h4)= h2 [Le(xj)− Lhe(xj)] + h2Lhe(xj) +O(h4)= h2τj,π[e] + h2Lhe(xj) +O(h4).

From the smoothness properties of the functions p, q, and τ , it follows thatthe solution e(x) of (1.16) is unique. Moreover, since τ ∈ C2(I), e(x) ∈C4(I) and τj,π[e] = O(h2). Using this result in (1.19) and rearranging, wehave

Lh[u(xj)− uj + h2e(xj)] = O(h4).

8

Page 9: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

The desired result, (1.15), follows from the stability of Lh.Deferred corrections is defined in the following way. Suppose τj,π[ · ] is a

difference operator such that

|τ [u(xj)]− τj,π[uh]| = O(h2),(1.20)

where uh denotes the solution ujN+1j=0 of (1.4)–(1.5) We define the mesh

function ujN+1j=0 by

Lhuj = fj + h2τj,π[uh], j = 1, . . . , N,u0 = g0, uN+1 = g1.

Then it is easy to show that ujN+1j=0 is a fourth-order approximation to

u(x), since

Lh[uj − u(xj)] = fj + h2τj,π[uh]− Lhu(xj)= [Lu(xj)− Lhu(xj)] + h2τj,π[uh]= −τj,π[u] + h2τj,π[uh]= −h2τ [u(xj)]− τj,π[uh]+O(h4).

Using (1.20) and the stability of Lh, it follows that

|uj − u(xj)| = O(h4).

The main problem in deferred corrections is the construction of second-order difference approximations τj,π[uh] to the truncation error term τ [u(xj)].One way is to replace the derivatives appearing in τ [u(x)] by standard differ-ence approximations using the finite difference solution ujN+1

j=0 in place ofthe exact solution u(x). One problem with this approach is that it requiressome modification near the end–points of the interval, or it is necessary tocompute the numerical solution outside the interval I. A second approachis to use the differential equation to express τ [u(x)] in the form

τ [u(x)] = C2(x)u′(x) + C1(x)u(x) + C0(x),(1.21)

where the functions C0, C1, C2 are expressible in terms of the functions p, q, fand their derivatives. Then choose

τj,π[uh] = C2(xj)uj+1 − uj−1

2h+ C1(xj)uj + C0(xj).

Sinceu′′ = pu′ + qu− f,

9

Page 10: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

we have

u(3) = pu′′ + p′u′ + qu′ + q′u− f ′(1.22)= p(pu′ + qu− f) + (p′ + q)u′ + q′u− f ′

= (p2 + p′ + q)u′ + (pq + q′)u− (pf + f ′)≡ Pu′ +Qu− F.

Similarly,

u(4) = (Pp+ P ′ +Q)u′ + (Pq +Q′)u− (Pf + F ′).(1.23)

When (1.22) and (1.23) are substituted into (1.17), we obtain the desiredform (1.21).

In deferred corrections, the linear algebraic systems defining the basicdifference approximation and the fourth-order approximation have the samecoefficient matrix, which simplifies the algebraic problem.

If the solution u of the boundary value problem (1.1)–(1.2) is sufficientlysmooth, then it can be shown that the local truncation error τj,π[u] has anasymptotic expansion of the form

τj,π[u] =m∑

ν=1

h2ντν [u(xj)] +O(h2m+2),

and, if m > 1, deferred corrections can be extended to compute approxima-tions of higher order than 4.

1.5.2 Numerov’s method

A second higher-order finite difference method is easily constructed in thecase p = 0, when (1.1)–(1.2) becomes

Lu(x) = −u′′(x) + q(x)u(x) = f(x), x ∈ I,u(0) = g0, u(1) = g1.

Now

∆hu(xj) ≡ [u(xj+1)− 2u(xj) + u(xj−1)]/h2(1.24)

= u′′(xj) +112h2u(4)(xj) +O(h4).

From the differential equation, we have

u(4)(x) = [q(x)u(x)− f(x)]′′

10

Page 11: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

and therefore

u(4)(xj) = ∆h[q(xj)u(xj)− f(xj)] +O(h2).(1.25)

Thus, from (1.24) and (1.25),

u′′(xj) = ∆hu(xj)−112h2∆h[q(xj)u(xj)− f(xj)] +O(h4).

We define ujN+1j=0 by

Lhuj ≡ −∆huj +(

1 +112h2∆h

)qj uj =

(1 +

112h2∆h

)f(xj), j = 1, . . . , N,

(1.26)u0 = g0, uN+1 = gN+1,

which is commonly known as Numerov’s method. Equations (1.26) are sym-metric tridiagonal and may be written in the form

cjuj−1 + djuj + ejuj+1 =112h2[fj+1 + 10fj + fj−1], j = 1, . . . , N,

where

cj = −(

1− 112h2qj−1

), dj = 2 +

56h2qj , ej = −

(1− 1

12h2qj+1

).

It is easy to show that, for h sufficiently small, the coefficient matrix of thissystem is strictly diagonally dominant and hence ujN+1

j=0 is unique. Froma similar analysis, it follows that the difference operator Lh is stable. Also,since

Lh[uj − u(xj)] =(

1 +112h2∆h

)f(xj)− Lhu(xj) = O(h4),

the stability of Lh implies that

|uj − u(xj)| = O(h4).

1.6 Second–Order Nonlinear Equations

In this section, we discuss finite difference methods for the solution of thesecond–order nonlinear two-point boundary value problem

Lu(x) ≡ −u′′ + f(x, u) = 0, x ∈ I,(1.27)

u(0) = g0, u(1) = g1.

11

Page 12: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

The basic second–order finite difference approximation to (1.27) takes theform

Lhuj ≡ −∆huj + f(xj , uj) = 0, j = 1, . . . , N.(1.28)

u0 = g0, uN+1 = g1.

Ifτj,π[u] ≡ Lhu(xj)− Lu(xj)

then clearly

τj,π[u] = −h2

12u(4)(ξj), ξj ∈ (xj−1, xj+1),

if u ∈ C4(I). Stability of the nonlinear difference problem is defined in thefollowing way.

Definition 1.6. A difference problem defined by the nonlinear differenceoperator Lh is stable if, for sufficiently small h, there exists a positiveconstant K, independent of h, such that, for all mesh functions vjN+1

j=0

and wjN+1j=0 ,

|vj − wj | ≤ Kmax(|v0 − w0|, |vN+1 − wN+1|) + max

1≤i≤N|Lhvj − Lhwj |

.

Notice that when Lh is linear, this definition reduces to the definition ofSection 1.3 applied to the mesh function vj − wjN+1

j=0 .

Theorem 1.5. If fu ≡∂f

∂uis continuous on I × (−∞,∞) such that

0 < q∗ ≤ fu,

then the difference problem (1.28) is stable, with K = max(1, 1/q∗)

Proof — For mesh functions vj and wj, we have

Lhvj − Lhwj = −∆h(vj − wj) + f(xj , vj)− f(xj , wj)= −∆h(vj − wj) + fu(xj , vj)(vj − wj),

on using the mean value theorem, where vj lies between vj and wj . Then

h2[Lhvj − Lhwj ] = cj [vj−1 − wj−1] + dj [vj − wj ]+ ej [vj+1 − wj+1]

12

Page 13: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

where cj = ej = −1 and

dj = 2 + h2fu(xj , vj).

Clearly|cj |+ |ej | = 2 < |dj |.

The remainder of the proof is similar to that of Theorem 1.2.An immediate consequence of the stability of Lh is the following:

Corollary 1.1. If the difference approximation uj defined by (1.28) existsthen it is unique.

For a proof of the existence of the solution, see Keller (1968).

Corollary 1.2. If u is the solution of (1.27) and ujN+1j=0 the solution of

(1.28) then, under the assumptions of Theorem 1.5,

|uj − u(xj)| ≤h2

12Kmax

x∈I|u(4)(x)|, j = 0, . . . , N + 1,

provided u ∈ C4(I).

In order to determine the difference approximation ujN+1j=0 , we must

solve the system of nonlinear equations (1.28), which may be written as

Ju + h2f(u) = g,(1.29)

where

J =

2 −1−1 2 −1

. . . . . . . . .. . . . . . −1

−1 2

, u =

u1

u2...

uN−1

uN

, f(u) =

f(x1, u1)f(x2, u2)

...f(xN−1, uN−1)f(xN , uN )

, g =

g00...0g1

.(1.30)This system is usually solved using Newton’s method (or a variant of it),which is described in Section 1.8.

13

Page 14: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

1.7 Numerov’s Method for Second Order Nonlinear Equa-tions

Numerov’s method for the solution of (1.27) may be derived in the followingway. If u is the solution of (1.27) and f is sufficiently smooth then

u(4)(xj) =d2

dx2f(x, u)|x=xj

= ∆hf(xj , u(xj)) +O(h2).

Since∆hu(xj) = u′′(xj) +

112h2u(4)(xj) +O(h4),

we have

∆hu(xj) = f(xj , u(xj)) +112h2∆hf(xj , u(xj)) +O(h4).

Based on this equation, Numerov’s method is defined by the nonlinear equa-tions

−∆huj + [1 +112h2∆h]f(xj , uj) = 0, j = 1, . . . , N

(1.31)u0 = g0, uN+1 = g1,

which may be written in the form

Ju + h2Bf(u) = g,(1.32)

where J and u are as in (1.30), and

B =112

10 11 10 1

. . . . . . . . .. . . . . . 1

1 10

, g =

g0 − 1

12h2f(0, g0)0...0

g1 − 112h

2f(1, g1)

.(1.33)

Again Newton’s method is used to solve the nonlinear system (1.32).Stepleman (1976) formulated and analyzed a fourth-order difference method

for the solution of the equation

−u′′ + f(x, u, u′) = 0, x ∈ I,

14

Page 15: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

subject to general linear boundary conditions. When applied to the bound-ary value problem (1.27), this method reduces to Numerov’s method. Higher-order finite difference approximations to nonlinear equations have also beenderived by Doedel (1979) using techniques different from those employed byStepleman.

1.8 Newton’s Method for Systems of Nonlinear Equations

In the case of a single equation, Newton’s method consists in linearizing thegiven equation

φ(u) = 0

by approximating ϕ(u) by

φ(u(0)) + φ′(u(0))(u− u(0)),

where u(0) is an approximation to the actual solution, and solving the lin-earized equation

φ(u(0)) + φ′(u(0))∆u = 0.

The value u(1) = u(0) + ∆u is then accepted as a better approximation andthe process is continued if necessary.

Consider now the N equations

φi(u1, u2, . . . , uN ) = 0, i = 1, . . . , N,(1.34)

for the unknowns u1, u2, . . . , uN , which we may write in vector form as

Φ(u) = 0.

If we linearize the ith equation, we obtain

φi(u(0)1 , u

(0)2 , . . . , u

(0)N ) +

N∑j=1

∂φi

∂uj(u(0)

1 , . . . , u(0)N )∆uj = 0, i = 1, . . . , N.(1.35)

If J (u) denotes the matrix with (i, j) element∂φi

∂uj(u), then (1.35) can be

written in the formJ(u(0)

)∆u = −Φ

(u(0)

).

If J (u(0)) is nonsingular, then

∆u = −[J (u(0))

]−1Φ(u(0)),

15

Page 16: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

andu(1) = u(0) + ∆u

is taken as the new approximation. If the matrices J (u(ν)), ν = 1, 2, . . . ,are nonsingular, one hopes to determine a sequence of successively betterapproximations u(ν), ν = 1, 2, . . . from the algorithm

u(ν+1) = u(ν) + ∆u(ν), ν = 0, 1, 2, . . . ,

where ∆u(ν) is obtained by solving the system of linear equations

J (u(ν))∆u(ν) = −Φ(u(ν)).

This procedure is known as Newton’s method for the solution of the systemof nonlinear equations (1.34). It can be shown that as in the scalar casethis procedure converges quadratically if u(0) is chosen sufficiently close tou, the solution of (1.34).

Now consider the system of equations

φi(u1, . . . , uN ) = −ui−1 + 2ui − ui+1

+h2

12[f(xi−1, ui−1) + 10f(xi, ui) + f(xi+1, ui+1)],

i = 1, . . . , N,

arising in Numerov’s method (1.31). In this case, the (i, j) element of theJacobian J is given by

∂φi

∂uj=

2 + 5

6h2fu(xi, ui), if j = i,

−1 + 112h

2fu(xj , uj), if |i− j| = 1,

0, if |i− j| > 1.

ThusJ (u) = J + h2BF (u)

whereF (u) = diag(fu(xi, ui)).

In this case, Newton’s method becomes[J + h2BF (u(ν))

]∆u(ν) = −

[Ju(ν) + h2Bf(u(ν))− g

],

(1.36)u(ν+1) = u(ν) + ∆u(ν), ν = 0, 1, 2, . . . .

16

Page 17: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

For the system (1.28),

φi(u1, . . . , uN ) ≡ −ui−1 + 2ui − ui+1 + h2f(xi, ui) = 0, i = 1, . . . , N,

and Newton’s method is given by (1.36) withB = I and g = (g0, 0, . . . , 0, g1)T .Note that in each case the Jacobian has the same structure and propertiesas the matrix arising in the linear case.

It is customary to stop the iteration (1.36) when

max1≤i≤N

|∆u(ν)i | ≤ XTOL(1 + max

1≤i≤N|u(ν)

i |)

andmax

1≤i≤N|φi(u(ν+1))| ≤ FTOL,

where XTOL and FTOL are prescribed tolerances.

17

Page 18: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

2 Algorithms for Solving Tridiagonal Linear Sys-tems

2.1 General Tridiagonal Systems

The algorithm used to solve a general tridiagonal system of the form

Au = b,(2.1)

where

A =

d1 e1c2 d2 e2

. . . . . . . . .. . . . . . . . .

cN−1 dN−1 eN−1

cN dN

,(2.2)

is a version of Gaussian elimination with partial pivoting. In the decompo-sition step of the algorithm, the matrix A is factored in the form

A = PLR,(2.3)

where P is a permutation matrix recording the pivotal strategy, and L is unitlower bidiagonal and contains on its subdiagonal the multipliers used in theelimination process. If no row interchanges are performed, then R is upperbidiagonal, but, in general, this matrix may also have nonzero elements onits second superdiagonal. Thus fill–in may occur in this process; that is, thefactored form of A may require more storage than A, in addition to a vectorto record the pivotal strategy.

Using (2.3), the system (2.1) is equivalent to the systems

PLv = b.

Ru = v.

which can be easily solved for v and the desired solution u. Software, sgtsv,based on this algorithm is available in LAPACK [4].

2.2 Diagonally Dominant Systems

Finite difference methods discussed in section 1 give rise to tridiagonal linearsystems with certain properties which can be exploited to develop stable

18

Page 19: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

algorithms in which no pivoting is required. In this section, we consider thecase in which the elements of the coefficient matrix A satisfy

(i) |d1| > |e1|,(ii) |di| ≥ |ci|+ |ei|, i = 2, . . . , N − 1,(2.4)(iii) |dN | > |cN |,

andciei−1 6= 0, i = 2, . . . , N ;(2.5)

that is, A is irreducibly diagonally dominant and hence nonsingular. Weshow that, without pivoting, such a matrix can be factored in a stable fashionin the form

A = LR,(2.6)

where L is lower bidiagonal and R is unit upper bidiagonal. The matricesL and R are given by

L =

α1

c2 α2

· ·· ·cN αN

, R =

1 γ1

1 γ2

· ·1 γN−1

1

,(2.7)

where

(i) α1 = d1,

(ii) γi = ei/αi, αi+1 = di+1 − ci+1γi, i = 1, . . . , N − 1.

Since

detA = det L · det R =N∏

i=1

αi · 1

and A is nonsingular, it follows that none of the αi vanishes; hence therecursion in (2.8) is well–defined.

We now show that conditions (2.4) ensure that the quantities αi and γi

are bounded and that the bounds are independent of the order of the matrixA.

Theorem 2.1. If the elements of A satisfy conditions (2.4), then

|γi| < 1,(2.8)

and0 < |di| − |ci| < |αi| < |di|+ |ci|, i = 2, . . . , N.(2.9)

19

Page 20: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

Proof — Since γi = ei/αi = ei/di,

|γ1| = |e1|/|d1| < 1,

from (2.4(i)). Now suppose that

|γj | < 1, j = 1, . . . , i− 1.(2.10)

Then, with (2.8(iii)) in (2.8(ii)), we have

γi = ei/(di − ciγi−1)

and|γi| = |ei|/(|di| − |ci||γi−1|) < |ei|/(|di| − |ci|)

from (2.10). Thus, using (2.4(ii)), it follows that |γi| < 1, and (2.8) followsby induction.

From (2.8(ii)), (2.8) and (2.4(ii)), it is easy to show that (2.9) holds.

The system (2.1) with A given by (2.6) is equivalent to

Lv = b, Ru = v,(2.11)

from which the desired solution u is obtained in the following fashion:

v1 = b1/α1

For i = 2 to N do:

vi = (bi − civi−1)/αi

end

uN = vN

For i = 1 to N − 1 do:

uN−i = vN−i − γN−iuN−i+1

end

(2.12)

The nonzero elements of L and R overwrite the corresponding elements ofA, and no additional storage is required.

20

Page 21: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

2.3 Positive Definite Systems

In section 1, we saw that if in equation (1.1) p = 0, then in addition topossessing properties (2.4)–(2.5), the matrix A is also symmetric (so thatci = ei−1, i = 1, . . . , N − 1) and its diagonal elements, di, i = 1, . . . , N , arepositive. In this case, the matrix A is positive definite.

It is well known that when A is positive definite a stable decomposi-tion is obtained using Gaussian elimination without pivoting. The LDLT

decomposition of A can easily be obtained from the recursion (2.8). If1l1 1

· ·· 1lN−1 1

,

and D = diag(δ1, . . . , δN ), then by identifying li with γi, and δi with αi, andsetting ci = ei−1, we obtain from (2.8)

δ1 = d1

For i = 1 to N − 1 do:

li = ei/δi

δi+1 = di+1 − liei

end

(2.13)

Since A is positive definite, D is also positive definite1, and hence the diag-onal elements of D are positive. Thus the recursions (2.13) are well defined.

Using the LDLT decomposition of A, the solution of the system (2.1) isdetermined by first solving

Lv = b,

which gives

v1 = b1

vi+1 = bi+1 − livi, i = 1, . . . , N − 1;

thenDLTu = v

1For u = LT v 6= 0, uT Du = vT LDLT v = vT Av > 0, v 6= 0 since L nonsingular.

21

Page 22: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

yieldsuN = vN/δN

and, for i = 1, . . . , N − 1,

uN−i =vN−i

δN−i− lN−iuN−i+1

= (vN−i − lN−iδN−iuN−i+1)/δN−i

= (vN−i − eN−iuN−i+1)/δN−i,

where in the last step we have used from (2.13) the fact that lN−iδN−i =eN−i. Thus,

v1 = b1

For i = 1 to N − 1 do:

vi+1 = bi+1 − livi

end

uN = vN/δN

For i = 1 to N − 1 do:

uN−i = (vN−i − eN−iuN−i+1)/δN−i

end

(2.14)

The code sptsv in LAPACK is based on (2.13)–(2.14)

22

Page 23: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

3 The Finite Element Galerkin Method

3.1 Introduction

Consider the two–point boundary value problem

Lu ≡ −u′′ + p(x)u′ + q(x)u = f(x), x ∈ I.(3.1)

u(0) = u(1) = 0,

where the functions p, q and f are smooth on I, and (1.3) holds. Let H10 (I)

denote the space of all piecewise continuously differentiable functions on Iwhich vanish at 0 and 1. If v ∈ H1

0 (I), then

−u′′v + pu′v + quv = fv,

and ∫ 1

0[−u′′v + pu′v + quv]dx =

∫ 1

0fvdx.

On integrating by parts, we obtain∫ 1

0u′v′dx− [u′v]|10 +

∫ 1

0pu′vdx+

∫ 1

0quvdx =

∫ 1

0fvdx.

Since v(0) = v(1) = 0, we have

(u′, v′) + (pu′, v) + (qu, v) = (f, v), v ∈ H10 (I),(3.2)

where

(ϕ,ψ) =∫ 1

0ϕ(x)ψ(x)dx.

Equation (3.2) is called the weak form of the boundary value problem (3.1),and is written in the form

a(u, v) = (f, v), v ∈ H10 (I),(3.3)

wherea(φ, ψ) = (φ′, ψ′) + (pφ′, ψ) + (qφ, ψ), φ, ψ ∈ H1

0 (I).

Suppose that Sh is a finite dimensional subspace of H10 (I). The finite ele-

ment Galerkin method consists in finding the element uh ∈ Sh, the Galerkinapproximation to the solution u of (3.1), satisfying

a(uh, v) = (f, v), v ∈ Sh.(3.4)

23

Page 24: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

Suppose w1, . . . , ws is a basis for Sh, and let

uh(x) =s∑

j=1

αjwj(x).(3.5)

Then, on substituting (3.5) in (3.4) with v = wi, we have

(s∑

j=1

αjw′j , w

′i) + (p

s∑j=1

αjw′j , wi) + (q

s∑j=1

αjwj , wi) = (f, wi), i = 1, . . . , s,

from which it follows thats∑

j=1

[(w′j , w′i) + (pw′j , wi) + (qwj , wi)]αj = (f, wi), i = 1, . . . , s;

that is,Aα = f ,(3.6)

where the (i, j) element of the matrix A is a(wj , wi), and the vectors α andf are given by

α = [α1, . . . , αs]T , f = [(f, w1), . . . , (f, ws)]T .

The system of equations (3.6) is often referred to as the Galerkin equations.

3.2 Spaces of Piecewise Polynomial Functions

The choice of the subspace Sh is a key element in the success of the Galerkinmethod. It is essential that Sh be chosen so that the Galerkin approximationcan be computed efficiently. Secondly, Sh should possess good approxima-tion properties as the accuracy of the Galerkin approximation uh dependson how well u can be approximated by elements of Sh. The subspace Sh isusually chosen to be a space of piecewise polynomial functions. To definesuch spaces, let Pr denote the set of polynomials of degree ≤ r, let

π : 0 = x0 < x1 < . . . < xN < xN+1 = 1(3.7)

denote a partition of I, and set

Ij = [xj−1, xj ], j = 1, . . . , N + 1,

hj = xj − xj−1 and h = maxj hj . We define

Mrk(π) = v ∈ Ck(I) : v|Ij ∈ Pr, j = 1, . . . , N + 1,

24

Page 25: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

where Ck(I) denotes the space of functions which are k times continuouslydifferentiable on I, 0 ≤ k < r, and v|Ij denotes the restriction of the functionv to the interval Ij . We denote by Mr,0

k (π) the space

Mrk(π) ∩ v|v(0) = v(1) = 0.

It is easy to see that Mrk(π) and Mr,0

k (π) are linear spaces of dimensionsN(r − k) + r + 1 and N(r − k) + r − 1, respectively. These spaces have thefollowing approximation properties.

Theorem 3.1. For any u ∈W jp (I), there exists a u ∈Mr

k(π) and a constantC independent of h and u such that

‖(u− u)(`)‖Lp(I) ≤ Chj−`‖u(j)‖Lp(I),(3.8)

for all integers ` and j such that 0 ≤ ` ≤ k + 1, ` ≤ j ≤ r + 1. Ifu ∈ W j

p (I) ∩ v|v(0) = v(1) = 0 then there exists a u ∈ Mr,0k (π) such that

(3.8) holds.

Commonly used spaces are the spaces of piecewise Hermite polynomials,M2m−1

m−1 (π), m ≥ 1. A convenient basis for M2m−1m−1 (π) is

Si,k(x;m;π)N+1, m−1i=0, k=0 ,(3.9)

where

DlSi,k(xj ;m;π) = δijδlk, 0 ≤ l ≤ m− 1, 0 ≤ j ≤ N + 1,

and δmn denotes the Kronecker delta. It is easy to see that Si,k(x;m;π) iszero outside [xi−1, xi+1]. A basis for M2m−1,0

m−1 (π) is obtained by omittingfrom (3.9) the functions Si,0(x;m;π), i = 0, N + 1, which are nonzero atx = 0 and x = 1.

Example 1. M10(π): the space of piecewise linear functions. The

dimension of this space is N+2, and, if `i(x) = (x−xi)/hi+1, then the basisfunctions Si,0(x; 1;π), i = 0, . . . , N + 1, are the piecewise linear functionsdefined by

S0,0(x; 1;π) =

1− `0(x), x ∈ I1,

0, otherwise,

Si,0(x; 1;π) =

`i−1(x), x ∈ Ii,

1− `i(x), x ∈ Ii+1,

0, otherwise, 1 ≤ i ≤ N,

25

Page 26: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

SN+1,0(x; 1;π) =

`N (x), x ∈ IN+1,

0, otherwise.

Thus there is one basis function associated with each node of the partitionπ. For convenience, we set

wi(x) = Si,0(x; 1;π), i = 0, . . . , N + 1.(3.10)

Note that if uh(x) =∑N+1

j=0 αjwj(x) then αj = uh(xj), j = 0, . . . , N + 1,since wj(xi) = δij . A basis for M1,0

0 (π) is obtained by omitting from (3.10)the functions w0(x) and wN+1(x).

Example 2. M31(π): the space of piecewise Hermite cubic func-

tions. This space has dimension 2N + 4. If

g1(x) = −2x3 + 3x2,

andg2(x) = x3 − x2,

the basis functions Si,0(x; 2;π), i = 0, . . . , N + 1, are the piecewise cubicfunctions defined by

S0,0(x; 2;π) =

g1(1− `0(x)), x ∈ I1,

0, otherwise,

Si,0(x; 2;π) =

g1(`i−1(x)), x ∈ Ii,

g1(1− `i(x)), x ∈ Ii+1,

0, otherwise, 1 ≤ i ≤ N,

SN+1,0(x; 2;π) =

g1(`N (x)), x ∈ IN+1,

0, otherwise,

and the functions Si,1(x; 2;π), i = 0, . . . , N + 1, are the piecewise cubicfunctions

S0,1(x; 2;π) =

−h1g2(1− `0(x)), x ∈ I1,

0, otherwise,

Si,1(x; 2;π) =

hig2(`i−1(x)), x ∈ Ii,

−hi+1g2(1− `i(x)), x ∈ Ii+1,

0, otherwise, 1 ≤ i ≤ N,

SN+1,1(x; 2;π) =

hN+1g2(`N (x)), x ∈ IN+1,

0, otherwise.

26

Page 27: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

For notational convenience, we write

vi(x) = Si,0(x; 2;π), si(x) = Si,1(x; 2;π), i = 0, . . . , N + 1.(3.11)

The functions vi and si are known as the value function and the slope func-tion, respectively, associated with the point xi ∈ π. Note that if

uh(x) =N+1∑j=0

αjvj(x) + βjsj(x),

thenαj = uh(xj), βj = u′h(xj), j = 0, . . . , N + 1,

since

vj(xi) = δij , v′j(xi) = 0, sj(xi) = 0, s′j(xi) = δij , i, j = 0, . . . N + 1.

A basis for M3,01 (π) is obtained by omitting from (3.11) the functions v0(x)

and vN+1(x).

3.3 The Algebraic Problem

First we examine the structure of the coefficient matrix A of the Galerkinequations (3.6) in the case in which Sh = M1,0

0 (π). With the basis functionsgiven by (3.10), it is clear that

(w′i, w′j) + (pwi, w

′j) + (qwi, wj) = 0 if |i− j| > 1,

since wi(x) = 0, x ∈/(xi−1, xi+1). ThusA is tridiagonal. Since the tridiagonalmatrices

A = ((w′i, w′j)), B = ((wi, wj)),(3.12)

corresponding to p = 0, q = 1, occur quite often in practice, their elementsare given in Appendix A. If the partition π is uniform and

J = h−2

2 −1−1 2 −1

. . . . . . . . .−1 2 −1

. . . . . . . . .−1 2 −1

−1 2

,(3.13)

27

Page 28: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

then it is easy to see from Appendix A that

A = hJ, B = h

(I − h2

6J

)=h

6tridiag(1 4 1).(3.14)

Now consider the case in which Sh = M31(π). It is convenient to consider

the basis wi2N+3i=0 , where

w2i = vi, w2i+1 = si, i = 0, . . . , N + 1.(3.15)

For this ordering of the basis functions (3.11), it follows that, since vi andsi vanish outside [xi−1, xi+1], 1 ≤ i ≤ N , the quantities (w′k, w

′l), (pwk, w

′l)

and (qwk, wl) are nonzero only if wk and wl are basis functions associatedwith the same node or adjacent nodes of the partition π. Consequently

a(wk, wl) = 0 if |k − l| > 3.

Thus the matrix A whose (i, j) element is a(wi, wj) is a band matrix ofbandwidth seven. More precisely, in the 2ith and (2i+ 1)th rows of A, onlythe elements in columns 2i+ j− 2, j = 0, 1, . . . , 5, can be nonzero. Thus wecan partition the (2N + 4)× (2N + 4) matrix A in the form

A =

A00 A01

A10 A11 A12

· · · · · · · · ·

· · · · · · · · ·

· · · · · · AN,N+1

AN+1,N AN+1,N+1

;(3.16)

that is, A is block tridiagonal, the submatrices Aii and Ai,i+1 being 2 × 2.The matrices A = ((w′i, w

′j)), B = ((wi, wj)) are given in Appendix B for the

case in which the partition π is uniform. If Sh = M3,01 (π), the corresponding

(2N+2)×(2N+2) matrix A is obtained from (3.16) by omitting the first andthe (2N + 3)th rows and columns. It is easy to see that if Sh = M2m−1

m−1 (π)and the basis given by (3.9) is chosen, then A is block tridiagonal with m×mdiagonal blocks.

3.4 Treatment of Inhomogeneous Dirichlet Boundary Con-ditions and Flux Boundary Conditions

The Galerkin method can be easily modified to treat boundary conditionsother than the homogeneous Dirichlet conditions. Consider, for example,

28

Page 29: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

the inhomogeneous Dirichlet boundary conditions

u(0) = g0, u(1) = g1, g0g1 6= 0,(3.17)

and choose Sh = M3,01 (π). If we set

w = u− g

whereg(x) = g0v0(x) + g1vN+1(x)

thenw(0) = w(1) = 0,

and, from (3.2),

(p(w + g)′, v′) + (q(w + g), v) = (f, v), v ∈ H10 (I).

The Galerkin approximation U ∈M31(π) is defined uniquely by

(pU ′, v′) + (qU, v) = (f, v), v ∈M3,01 (π),

and the condition that if

U(x) =N+1∑i=0

αivi(x) + βisi(x)

then α0 = g0 and αN+1 = g1.With the basis functions ordered as in (3.15), the coefficient matrix of the

Galerkin equations in this case is the matrix A of (3.16) with the elementsof rows 1 and 2N + 3 replaced by

a1j = δ1j , j = 1, . . . , 2N + 4

anda2N+3,j = δ2N+3,j , j = 1, . . . , 2N + 4,

respectively, and the right hand side vector has as its first and (2N + 3)th

components, g0 and g1, respectively.In the case in which the general linear boundary conditions

λ0u(0)− p(0)u′(0) = ν0, λ1u(1) + p(1)u′(1) = ν1,(3.18)

are prescribed then as in derivation of (3.2),we have,

(pu′, v′)− [pu′v]|10 + (qu, v) = (f, v),(3.19)

29

Page 30: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

for v ∈ H1(I). Using (3.18) in (3.19), we obtain

(pu′, v′) + λ1u(1)v(1) + λ0u(0)v(0) + (qu, v)= (f, v) + ν0v(0) + ν1v(1), v ∈ H1(I).

If Sh is now a subspace of H1(I), then the Galerkin approximation U ∈ Sh

is defined by

(pU ′, v′)+λ1U(1)v(1)+λ0U(0)v(0)+(qU, v) = (f, v)+ν0v(0)+ν1v(1), v ∈ Sh.(3.20)Thus, if Sh = M3

1(π) and the basis functions are again ordered as in (3.15),the matrix of coefficents of the Galerkin equations is obtained from (3.16)by adding λ0 and λ1 to the (1, 1) element and the (2N +3, 2N +3) element,respectively. The term ν0v(0)+ν1v(1) on the right hand side of (3.20) merelyadds ν0 to the first component and ν1 to the (2N + 3)–th component of theright hand side vector.

3.5 The Uniqueness of the Galerkin Approximation

We now show that there exists a unique solution uh ∈ Sh of (3.4) for hsufficiently small. Since the problem under consideration is linear, it issufficient to show uniqueness.

Let U1 and U2 be two solutions of (3.4) and set Y = U1 − U2. Then

a(Y, v) = 0, v ∈ Sh.(3.21)

Let ψ ∈ H10 (I) be the solution of

L∗ψ = Y (x), x ∈ I,ψ(0) = ψ(1) = 0,

where L∗ denotes the formal adjoint of L,

L∗u = −u′′ − (pu)′ + qu.

From the hypotheses on p and q, it follows that ψ ∈ H2(I) ∩H10 (I) and

‖ψ‖H2 ≤ C‖Y ‖L2 .(3.22)

Thus, using integration by parts and (3.21), we have

‖Y ‖2L2 = (L∗ψ, Y ) = a(Y, ψ) = a(Y, ψ − χ),

30

Page 31: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

where χ ∈ Sh, and since

a(φ, ψ) ≤ C‖φ‖H1‖ψ‖H1 ,(3.23)

for φ, ψ ∈ H1(I), it follows that

‖Y ‖2L2 ≤ C‖Y ‖H1‖ψ − χ‖H1 .

From Theorem 3.1, we can choose χ ∈ Sh such that

‖ψ − χ‖H1 ≤ Ch‖ψ‖H2 .

Thus‖Y ‖2

L2 ≤ Ch‖Y ‖H1‖ψ‖H2 ≤ Ch‖Y ‖H1‖Y ‖L2 ,

where in the last inequality we have used (3.22), and we obtain

‖Y ‖L2 ≤ Ch‖Y ‖H1 .(3.24)

Since Y ∈ Sh,a(Y, Y ) = 0,

from which it follows that

‖Y ′‖2L2 = −(pY ′, Y )− (qY, Y ) ≤ C‖Y ′‖L2 + ‖Y ‖L2‖Y ‖L2 .

Using this inequality and (3.24), we obtain

‖Y ‖2H1 ≤ Ch‖Y ‖2

H1 .

Thus, for h sufficiently small,

‖Y ‖H1 = 0,

and hence from Sobolev’s inequality,

Y = 0.

For the self-adjoint boundary value problem

Lu ≡ −(pu′)′ + q(x)u = f(x), x ∈ I.u(0) = u(1) = 0,

the uniqueness of the Galerkin approximation uh ∈ Sh satisfying

(pu′h, v′) + (quh, v) = (f, v), v ∈ Sh,

is much easier to prove. In this case, the coefficient matrix A of the Galerkinequations is positive definite, from which it follows immediately that theGalerkin approximation is unique. This result is proved in the followingtheorem.

31

Page 32: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

Theorem 3.2. The matrix A is positive definite.

Proof — Suppose β ∈ Rs and β 6= 0. Then, if A = (aij) and w =∑sj=1 βjwj , then

βTAβ =s∑

i,j=1

aijβiβj

=s∑

i,j=1

(pβiw′i, βjw

′j) + (qβiwi, βjwj)

= (pw′, w′) + (qw,w)= ‖p1/2w′‖2

L2 + ‖q1/2w‖2L2

≥ p∗‖w′‖2L2 .

The proof is complete if ‖w′‖ > 0. Suppose ‖w′‖ = 0. Then w′ = 0 and

w = C, where C is a constant. Since w ∈ Sh, w(0) = w(1) = 0, fromwhich it follows that C = 0. Therefore w = 0; that is

∑sj=1 βjwj = 0.

Since w1, . . . , ws is a basis for Sh and is therefore linearly independent,this implies that βj = 0, j = 1, . . . , N , which is a contradiction. Therefore‖w′‖ > 0 and A is positive definite.

3.6 The Accuracy of the Galerkin Approximation

3.6.1 Optimal H1 and L2 error estimates

In the following, an estimate of the error u − uh in an Hk–norm (or anLp–norm) will be called optimal in Hk (or Lp) if the estimate has the samepower of h as is possible by the approximation properties of the subspaceSh with the same smoothness assumptions on u.

Throughout this and subsequent analyses, C denotes a generic constantwhich is independent of u and h.

Theorem 3.3. Suppose u ∈ Hr+1(I) ∩ H10 (I). Then, for h sufficiently

small,‖u− uh‖L2 + h‖u− uh‖H1 ≤ Chr+1‖u‖Hr+1 .(3.25)

Proof — Let φ ∈ H10 (I) satisfy

L∗φ = e(x), x ∈ I,φ(0) = φ(1) = 0,

32

Page 33: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

where e = u− uh. Then, as in section 3.2 but with φ and e replacing ψ andY , respectively, we have

‖e‖L2 ≤ Ch‖e‖H1 ,(3.26)

sincea(e, v) = 0, v ∈ Sh.

Sincea(e, e) = a(e, u− χ), χ ∈ Sh,

it follows that

‖e′‖2L2 = −(pe′, e)− (qe, e) + a(e, u− χ)

≤ C[‖e′‖L2 + ‖e‖L2 ]‖e‖L2 + ‖e‖H1‖u− χ‖H1

on using Schwarz’s inequality and (3.23). On using (3.26), we have, for hsufficiently small,

‖e‖H1 ≤ C‖u− χ‖H1 .(3.27)

Since u ∈ Hr+1(I) ∩H10 (I), from Theorem 3.1, χ can be chosen so that

‖u− χ‖H1 ≤ Chr‖u‖Hr+1 ,

and hence‖e‖H1 ≤ Chr‖u‖Hr+1 .(3.28)

The use of this estimate in (3.26) completes the proof.

3.6.2 Optimal L∞ error estimate

In this section, we derive an optimal L∞ error estimate when Sh = Mr,0k (π)

and the partition π of I is from a quasi–uniform collection of partitions,which we now define.

Definition 3.1. Let

π : 0 = x0 < x1 < . . . < xN < xN+1 = 1

denote a partition of I and let Π(I) denote the collection of all such partitionsπ of I. As before, set

hi = xi − xi−1, i = 1, . . . , N + 1,

33

Page 34: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

and h = maxi hi. A collection of partitions C ⊂ Π(I) is called quasi–uniformif there exists a constant σ ≥ 1 such that, for all π ∈ C,

max1≤j≤N+1

hh−1j ≤ σ.

The following lemma, which is proved in [14], plays an important role inthe derivation of the L∞ estimate.

Lemma 3.1. Let Pu be the L2 projection of u into Mrk(π), that is,

(Pu− u, v) = 0 v ∈Mrk(π),

where −1 ≤ k ≤ r − 1. Then, if u ∈W r+1∞ (I),

‖Pu− u‖L∞(I) ≤ Chr+1‖u‖W r+1∞ (I),

provided π is chosen from a quasi–uniform collection of partitions of I.

We now prove the following theorem.

Theorem 3.4. Suppose Sh = Mr,0k (π), with π chosen from a quasi–uniform

collection of partitions. If u ∈W r+1∞ (I) ∩H1

0 (I), then

‖u− uh‖L∞ ≤ Chr+1‖u‖W r+1∞

.(3.29)

Proof — Let W ∈ Sh satisfy

(W ′, v′) = (u′, v′), v ∈ Sh.

Sincea(u− uh, v) = 0, v ∈ Sh,

it follows that

((W − uh)′, v′) + (u− uh, qv − (pv)′) = 0,

for all v ∈ Sh. If we set v = W − uh, then

‖(W − uh)′‖2L2 ≤ C‖u− uh‖L2‖W − uh‖H1 .

Therefore, since ‖ · ‖H10

and ‖ · ‖H1 are equivalent norms on H10 ,

‖W − uh‖H1 ≤ C‖u− uh‖L2

34

Page 35: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

and from Sobolev’s inequality, we obtain

‖W − uh‖L∞ ≤ C‖u− uh‖L2 .

Then, from Theorem 3.3, it follows that

‖W − uh‖L∞ ≤ Chr+1‖u‖Hr+1 .(3.30)

Since‖u− uh‖L∞ ≤ ‖u−W‖L∞ + ‖W − uh‖L∞ ,(3.31)

we need to estimate ‖u−W‖L∞ to complete the proof.Note that since

((u−W )′, 1) = 0,

it follows that W ′ is the L2 projection of u′ into Mr−1k−1(π), and hence, from

Lemma 3.1, we obtain

‖(u−W )′‖L∞ ≤ Chr‖u′‖W r∞ ≤ Chr‖u‖W r+1

∞.(3.32)

Now suppose g ∈ L1 and define G by

G′′ = −g(x), x ∈ I,G(0) = G(1) = 0.

Then

‖G‖W 21≤ C‖g‖L1 ,(3.33)

and, for χ ∈ Sh,

(u−W, g) = −(u−W,G′′) = ((u−W )′, (G− χ)′).

On using Holder’s inequality, we obtain

(u−W, g) ≤ ‖(u−W )′‖L∞‖(G− χ)′‖L1 .

From Theorem 3.1, we can choose χ so that

‖(G− χ)′‖L1 ≤ Ch‖G‖W 21.

Hence, on using (3.33), it follows that

|(u−W, g)| ≤ Ch‖(u−W )′‖L∞‖g‖L1 .

On using (3.32) and duality, we have

‖u−W‖L∞ ≤ Chr+1‖u‖W r+1∞

.(3.34)

The desired result now follows from (3.30), (3.31) and (3.34).

35

Page 36: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

3.6.3 Superconvergence results

The error estimates of Theorems 3.3 and 3.4 are optimal and consequentlyno better global rates of convergence are possible. However, there can beidentifiable points at which the approximate solution converges at rates thatexceed the optimal global rate. In the following theorem, we derive one suchsuperconvergence result.

Theorem 3.5. If Sh = Mr,00 (π) and u ∈ Hr+1(I) ∩ H1

0 (I), then, for hsufficiently small,

|(u− uh)(xi)| ≤ Ch2r‖u‖Hr+1 , i = 0, . . . , N + 1.(3.35)

Proof — Let G(x, ξ) denote the Green’s function for (3.1); that is,

u(x) = −(Lu,G(x, ·)) = a(u,G(x, ·))

for sufficiently smooth u. This representation is valid for u ∈ H10 (I) and

hence it can be applied to e = u− uh. Thus, for χ ∈ Sh,

e(xi) = a(e,G(xi, ·)) = a(e,G(xi, ·)− χ),

sincea(e, χ) = 0, χ ∈ Sh.

Thus,|e(xi)| ≤ C‖e‖H1‖G(xi, ·)− χ‖H1 .(3.36)

From the smoothness assumptions on p and q, it follows that

G(xi, ·) ∈ Hr+1([0, xi]) ∩Hr+1([xi, 1]),

and‖G(xi, ·)‖Hr+1([0,xi]) + ‖G(xi, ·)‖Hr+1([xi,1]) ≤ C.

Hence there exists χ ∈ Sh (obtained, for example, by Lagrange interpolationon each subinterval) such that

‖G(xi, ·)− χ‖H1 ≤ Chr.(3.37)

From Theorem 3.3, we have

‖e‖H1 ≤ Chr‖u‖Hr+1 ,(3.38)

36

Page 37: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

for h sufficiently small, and hence combining (3.36)–(3.38), we obtain

|e(xi)| ≤ Ch2r‖u‖Hr+1 ,

as desired.A method which involves very simple auxiliary computations using the

Galerkin solution can be used to produce superconvergent approximationsto the derivative, [24]. First we consider approximations to u′(0) and u′(1).Motivated by the fact that

(f, (1− x)) = (−u′′ + pu′ + qu, 1− x) = u′(0) + a(u, 1− x),

we define an approximation Γ0 to u′(0) by

Γ0 = (f, 1− x)− a(uh, 1− x),

where uh is the solution to (3.4). Also, with 1−x replaced by x in the above,we find that

u′(1) = a(u, x)− (f, x),

and hence we define an approximation ΓN+1 to u′(1) by

ΓN+1 = a(uh, x)− (f, x).

It can be shown that that if u ∈ Hr+1(I), then

|Γj − u′(xj)| ≤ Ch2r‖u‖Hr+1 ,(3.39)

j = 0, N + 1, when for example, Sh = Mr,0k (π).

If Sh = Mr,00 (π), a procedure can be defined which at the nodes xi, i =

1, . . . , N , gives superconvergence results similar to (3.39). Specifically, if Γj ,an approximation to u′(xj), j = 1, . . . , N , is defined by

Γj =a(uh, x)I′j

− (f, x)I′j

xj,

where the subscript I ′j denotes that the inner products are taken over I ′j =(0, xj), then (3.39) holds for j = 1, . . . , N . The approximation Γj is moti-vated by the fact that

(Lu, x)I′j= (f, x)I′j

,

and, after integration by parts,

−u′(xj)xj + a(u, x)I′j= (f, x)I′j

.

37

Page 38: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

3.6.4 Quadrature Galerkin methods

In most cases, the integrals occurring in (3.6) cannot be evaluated exactlyand one must resort to an approximation technique for their evaluation. Inthis section, the effect of the use of certain quadrature rules the Galerkinmethod is discussed. To illustrate the concepts, we consider the problem

−(pu′)′ = f, x ∈ I,u(0) = u(1) = 0,

where we shall assume that there exists a constant p0 such that

0 < p0 ≤ p(x), x ∈ I.

The Galerkin method in this case takes the form

(pu′h, v′) = (f, v), v ∈ Sh,(3.40)

where we shall take Sh to be Mr,00 (π). Let

0 ≤ ρ1 < ρ2 < . . . < ρν ≤ 1,

and setρij = xi−1 + hiρj , i = 1, . . . , N + 1, j = 1, . . . , ν.

Suppose ωj > 0, j = 1, . . . , ν, and denote by

〈α, β〉i = hi

ν∑j=1

ωj(αβ)(ρij),

a quadrature rule on Ii which is exact for αβ ∈ Pt(Ii).Set

〈α, β〉 =N+1∑i=1

〈α, β〉i.

If this quadrature formula is used in (3.40), the problem becomes that offinding zh ∈ Sh such that

〈pz′h, v′〉 = 〈f, v〉, v ∈ Sh.(3.41)

If t is sufficiently large, the existence and uniqueness of zh follow from thenext lemma.

38

Page 39: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

Lemma 3.2. If t ≥ 2r − 2,

〈pV ′, V ′〉 ≥ p0‖V ′‖2L2 , V ∈ Sh.

Proof — Since the weights ωj , j = 1, . . . , q are positive, and (V ′)2 ∈P2r−2(Ii),

〈pV ′, V ′〉 ≥ p0〈V ′, V ′〉= p0‖V ′‖2

L2 .

From this lemma, it follows that, if t ≥ 2r − 2, there exists a uniquesolution zh ∈ Sh of (3.41). In order to have t ≥ 2r − 2, it is necessary tohave at least r quadrature points; the use of fewer leads to non–existence,as is shown by Douglas and Dupont (1974).

It can be shown (cf. Douglas and Dupont, 1974) that if t ≥ 2r − 1 then

‖u− zh‖L2 ≤ Chr+1‖f‖Hr+1 .

In addition, the superconvergence indicated in Theorem 3.5 is preserved.More specifically

|(u− zh)(xi)| ≤ Ch2r‖f‖Ht+1 .

Notice that the use of an r–point Gaussian quadrature rule produces thedesired accuracy and satisfies the condition of Lemma 3.2.

3.7 The Galerkin Method for Nonlinear Problems

Consider the boundary value problem

−u′′ + f(x, u) = 0, x ∈ I,u(0) = u(1) = 0.

It is easy to show that the weak form of this boundary value problem is

(u′, v′) + (f(u), v) = 0, v ∈ H10 (I).(3.42)

As before, let Sh be a finite dimensional subspace of H10 (I) with basis

w1, . . . , ws. The Galerkin approximation to u is the element uh ∈ Sh

such that(u′h, v

′) + (f(uh), v) = 0, v ∈ Sh,(3.43)

and if

uh(x) =s∑

j=1

αjwj(x),

39

Page 40: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

we obtain from (3.43) with v = wi, the nonlinear system

s∑j=1

(w′i, w′j)αj +

(f(

s∑ν=1

ανwν), wi

)= 0, i = 1, . . . , s,(3.44)

for α1, . . . , αs. Newton’s method for the solution of (3.44) can be easilyderived by linearizing (3.43) to obtain

(u(n)h

′, v′) + (f(u(n−1)

h , v) + (fu(u(n−1)h )(u(n)

h − u(n−1)h ), v) = 0, v ∈ Sh,

where u(0)h is arbitrary. If

u(k)h =

s∑j=1

α(k)j wj ,

then(A+Bn)α(n) = −Fn +Bnα(n−1),(3.45)

where

A = ((w′i, w′j)), α(n) = (α(n)

1 , . . . , α(n)N )T ,

Bn = ((fu(s∑

ν=1

α(n−1)ν wν)wi, wj)),

Fn = ((f(u(n−1)h ), wi)).

Note that (3.45) may be written in the form

(A+Bn)(α(n) −α(n−1)) = −(Aα(n−1) + Fn).

A comprehensive account of the Galerkin method for second order nonlinearproblems is given by Fairweather (1978).

40

Page 41: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

4 The Orthogonal Spline Collocation Method

4.1 Introduction

Consider the linear second order two-point boundary value problem

Lu ≡ −u′′ + p(x)u′ + q(x)u = f(x), x ∈ I,(4.1)

µ0u(0) + ν0u′(0) = g0, µ1u(1) + ν1u

′(1) = g1,(4.2)

where the functions p, q and f are smooth on I. Let

π : 0 = x0 < x1 < . . . < xN < xN+1 = 1

denote a partition of I, and set hi = xi−xi−1. In the orthogonal spline collo-cation method for (4.1)–(4.2), the approximate solution uh ∈Mr

1(π), r ≥ 3.If wjs

j=1 is a basis for Mr1(π), where s = (N +1)(r− 1)+2, we may write

uh(x) =s∑

j=1

ujwj(x).(4.3)

Then the coefficients ujsj=1 in (4.3) are determined by requiring that uh

satisfy (4.1) at the points ξjs−2j=1, and the boundary conditions (4.2):

µ0uh(0) + ν0u′h(0) = g0,

Luh(ξj) = f(ξj), j = 1, 2, . . . , s− 2,µ1uh(1) + ν1u

′h(1) = g1,

where

ξ(i−1)(r−1)+k = xi−1 + hiσk, i = 1, 2, . . . , N + 1, k = 1, 2, . . . , r − 1,(4.4)

and σkr−1k=1 are the nodes for the (r− 1)-point Gauss-Legendre quadrature

rule on the interval [0, 1].As an example, consider the case when r = 3, for which the collocation

points are

ξ2i−1 = xi−1+12

(1− 1√

3

)hi, ξ2i = xi−1+

12

(1 +

1√3

)hi, i = 1, 2, . . . , N+1.

(4.5)With the usual basis (3.15) for M3

1(π), we set

uh(x) =N+1∑j=0

αjvj(x) + βjsj(x).

41

Page 42: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

Since only four basis functions,

vi−1, si−1, vi, si,(4.6)

are nonzero on [xi−1, xi], the coefficient matrix of the collocation equationsis of the form

D0

S1 T1

S2 T2

. . . . . .SN+1 TN+1

D1

,(4.7)

with Da = [µ0 ν0], Db = [µ1 ν1], and, for i = 1, 2, . . . , N + 1,

Si =

(Lvi−1(ξ2i−1) Lsi−1(ξ2i−1)Lvi−1(ξ2i) Lsi−1(ξ2i)

), Ti =

(Lvi(ξ2i−1) Lsi(ξ2i−1)Lvi(ξ2i) Lsi(ξ2i)

).

For r > 3, with the commonly used B-spline basis [8], the coefficientmatrix has the form

D0

W11 W12 W13

W21 W22 W23

. . .WN+1,1 WN+1,2 WN+1,3

D1

,(4.8)

with Wi1 ∈ R(r−1)×2,Wi2 ∈ R(r−1)×(r−3),Wi3 ∈ R(r−1)×2. The matrices(4.7) and (4.8) are called almost block diagonal and the collocation equationsform an almost block diagonal linear system. There exist efficient algorithmsfor solving such systems based on the idea of alternate row and columnelimination; see section 5.

For the case in which the partition π is uniform, the values at the colloca-tion points of the basis functions (4.6) and their first and second derivativesare given in Appendix C.

4.2 The Existence and Uniqueness of the Collocation Ap-proximation

To demonstrate analytical tools used in the proof of the existence anduniqueness of the collocation approximation and in the derivation of error

42

Page 43: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

estimates, we consider the linear two-point boundary value problem

Lu = −u′′ + p(x)u′ + q(x)u = f(x), x ∈ I,

u(0) = u(1) = 0.

(4.9)

Then, for v ∈ H2(I) ∩H10 (I), there exists a constant C0 such that

‖v‖H2(I) ≤ C0‖Lv‖L2(I).(4.10)

We define the discrete inner product 〈·, ·〉 by

〈φ, ψ〉 =N+1∑i=1

〈φ, ψ〉i,

where

〈φ, ψ〉i =r−1∑k=1

hiωjφ(ξ(i−1)(r−1)+k)ψ(ξ(i−1)(r−1)+k), i = 1, . . . .N + 1,

and ξjs−2j=1 are the collocation points given in (4.4) and ωkr−1

k=1 are theweights for the (r−1)-point Gauss-Legendre quadrature rule on the interval[0, 1].

In order to demonstrate the existence and uniqueness of the collocationapproximation uh ∈ Sh ≡Mr,0

1 (π) satisfying

Luh(ξj) = f(ξj) j = 1, . . . , s− 2,

we use the following lemma, [13].

Lemma 4.1. There exists a constant C such that

|〈LY,LY 〉 − ‖LY ‖2L2(I)| ≤ Ch‖Y ‖2

H2(I),(4.11)

for all Y ∈ Sh.

From (4.10) and this lemma, we find that, for Y ∈ Sh,

‖Y ‖2H2(I) ≤ C0‖LY ‖2

L2(I) ≤ C0(〈LY,LY 〉+ Ch‖Y ‖2H2(I)).

Thus, for h sufficiently small,

‖Y ‖2H2(I) ≤ C〈LY,LY 〉.(4.12)

IfLY (ξj) = 0, j = 1, . . . , s− 2,

then (4.12) implies that Y = 0, which proves the uniqueness and henceexistence of the collocation approximation uh.

43

Page 44: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

4.3 Optimal H2 Error Estimates

We now derive an optimal H2 error estimate.

Theorem 4.1. Suppose u ∈ Hr+1(I)∩H10 (I). Then there exists a constant

C such that, for h sufficiently small,

‖u− uh‖H2(I) ≤ C hr−1‖u‖Hr+1(I).(4.13)

Proof — Let Tr,h be the interpolation operator from C1(I) to Sh deter-mined by the conditions

(i) (Tr,hv)(xi) = v(xi), (Tr,hv)′(xi) = v′(xi), i = 0, . . . , N + 1,

(ii) (Tr,hv)(ηij) = v(ηij), i = 1, . . . , N + 1, j = 1, . . . , r − 3,

for v ∈ C1(I), where the points ηij , j = 1, . . . , r − 3, are certain specificpoints in Ii; see [13] for details. Clearly Tr,h is local in the sense that Tr,hvis determined on Ii by v on Ii. (Note that when r = 3, Tr,hv is the Hermitecubic interpolant of v.) It can be shown that, if W = Tr,hu and u ∈ Hr+1(I),

2∑k=0

hk‖(u−W )(k)‖L∞(Ii) ≤ Chr+ 1

2i ‖u(r+1)‖L2(Ii), i = 1, . . . , N + 1,(4.14)

and2∑

k=0

hk‖(u−W )(k)‖L2(I) ≤ C hr+1‖u(r+1)‖L2(I).(4.15)

Then, using (4.14), it follows that

|L(uh−W )(ξj)| = |L(u−W )(ξj)| ≤ C2∑

k=0

‖(u−W )(k)‖L∞(Ii) ≤ C hr− 32 ‖u(r+1)‖L2(Ii).

Hence

〈L(uh −W ), L(uh −W )〉 =N+1∑i=1

r−1∑j=1

hiωj [L(uh −W )(xij)]2

≤N+1∑i=1

r−1∑j=1

h2r−2i ωj‖u(r+1)‖2

L2(Ii)

≤ C h2r−2‖u(r+1)‖2L2(I),

and, from (4.12), it follows that

44

Page 45: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

‖uh −W‖H2(I) ≤ C hr−1‖u(r+1)‖L2(I).(4.16)

The required result is now obtained from the triangle inequality, (4.15) and(4.16).

4.4 Superconvergence Results

Douglas and Dupont [13] construct a “quasi–interpolant”, u ∈ Sh, whichhas the properties that, for u ∈W 2r−1

∞ (I),

|(u− u)(j)(xi)| ≤ C h2r−2‖u‖W 2r−1∞ (I), j = 0, 1,(4.17)

and, if u ∈W 2r∞ (I),

L(uh − u)(ξj) = ε(ξj), j = 1, . . . , s− 2,(uh − u)(0) = (uh − u)(1) = 0,

(4.18)

where|ε(ξj)| ≤ C h2r−2‖u‖W 2r

∞ (I).

From (4.12) and (4.18), we obtain

‖uh − u‖H2(I) ≤ C〈L(uh − u), L(uh − u)〉12

= C〈ε, ε〉12

≤ C h2r−2‖u‖W 2r∞ (I),

and hence, using Sobolev’s inequality,

‖uh − u‖W 1∞(I) ≤ C h2r−2‖u‖W 2r

∞ (I);(4.19)

that is, the quasi–interpolant u differs from the collocation solution uh alongwith first derivatives, uniformly by O(h2r−2). For r ≥ 3, this represents ahigher order in h than is possible for u − uh. However, using the triangleinequality, (4.17) and (4.18), we see that a superconvergence phenomenonoccurs at the nodes, namely

|(u− uh)(j)(xi)| ≤ C h2r−2‖u‖W 2r∞ (I), j = 0, 1.(4.20)

45

Page 46: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

4.5 L2 and H1 Error Estimates

From the inequality,(∫Ii

g(x)2 dx) 1

2

≤ Ch12i (|g(xi−1|+ |g(xi)|) + h2

i

(∫Ii

g′′(x)2 dx) 1

2

,

valid for any g ∈ C2(Ii), the H2 estimate (4.13) and the superconvergenceresult (4.20), it follows that

‖u− uh‖L2 ≤ C(h2r−1‖u‖W 2r

∞ (I) + hr+1‖u‖Hr+1(I)

).

From this inequality, (4.13) and the inequality [1],∫Ii

|g′(x)|2 dx ≤ 54(h−2

i

∫Ii

|g(x)|2 dx+ h2i

∫Ii

|g′′(x)|2 dx),

we obtain the estimate

‖u− uh‖H1(I) ≤ C(h2r−2‖u‖W 2r

∞ (I) + hr‖u‖Hr+1(I)

).

In these estimates, the exponent of h is optimal but the smoothness re-quirements on u are not minimal. It is shown in [13] that, by modifying thecollocation procedure, the smoothness requirements on the solution can bereduced to the minimal ones required for approximation when global esti-mates are sought. To describe this modification, let f denote the standardL2–projection of f into M r−2

−1 . The smoothed collocation method consistsof finding uh ∈ Sh such that

(Luh)(ξj) = f(ξj), j = 1, . . . , s− 2.

Douglas and Dupont [13] prove that, for h sufficiently small,

‖u− uh‖Hj(I) ≤ Cjhr+1−j‖f‖Hr−1(I), j = 0, 1, 2,

and|(u− uh)(xi)| ≤ Ch2r‖f‖Hr−1(I), i = 1, . . . , N.

46

Page 47: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

Figure 1: Structure of a general ABD matrix

5 Algorithms for Solving Almost Block DiagonalLinear Systems

5.1 Introduction

In several numerical methods for solving two-point boundary value problems,there arise systems of linear algebraic equations with coefficient matriceswhich have a certain block structure, almost block diagonal (ABD), [8]; see,for example, section 4.1.

The most general ABD matrix, shown in Figure 1, has the followingcharacteristics: the nonzero elements lie in blocks which may be of differentsizes; each diagonal entry lies in a block; any column of the matrix intersectsno more than two blocks (which are successive), and the overlap betweensuccessive blocks (that is, the number of columns of the matrix common totwo successive blocks) need not be constant. In commonly used methods forsolving two-point boundary value problems, the most frequently occurringABD structure is shown in Figure 2, where the blocks W (i), i = 1, 2, . . . , N ,are all of equal size, and the overlap between successive blocks is constantand equal to the sum of the number of rows in TOP and BOT.

In this section, we outline efficient methods for the solution of systemswith coefficient matrices having this structure. These algorithms are allvariants of Gaussian elimination with partial pivoting, and are more efficientthan Gaussian elimination primarily because they introduce no fill–in.

We describe the essential features of the algorithms using a simple ex-ample with a coefficient matrix of the form in Figure 2, namely the matrixof Figure ?? in which there are two 4× 7 blocks W (1),W (2), and TOP and

47

Page 48: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

Figure 2: Special ABD structure arising in BVODE solvers

Figure 3: Fill-in introduced by SOLVEBLOK

BOT are 2×3 and 1×3, respectively. The overlap between successive blocksis thus 3.

5.2 Gaussian Elimination with Partial Pivoting

The procedure implemented in solveblok [9, 10] uses conventional Gaussianelimination with partial pivoting. Fill-in may be introduced in the positionsindicated ∗ in Figure 3. The possible fill-in, and consequently the possibleadditional storage and work, depends on the number of rows, NT , in theblock TOP.

48

Page 49: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

Figure 4: Structure of the reduced matrix

5.3 Alternate Row and Column Elimination

This stable elimination procedure, based on the approach of Varah [23],generates no fill-in for the matrix A of Figure ??. Suppose we choose apivot from the first row. If we interchange the first column and the columncontaining the pivot, there is no fill-in. Moreover, if instead of perform-ing row elimination as in conventional Gaussian elimination, we reduce the(1, 2) and (1, 3) elements to zero by column elimination, the correspondingmultipliers are bounded in magnitude by unity. We repeat this process inthe second step, choosing a pivot from the elements in the (2, 2) and (2, 3)positions, interchanging columns 2 and 3 if necessary and eliminating the(2, 3) element. If this procedure were adopted in the third step, fill-in couldbe introduced in the (i, 3) positions, i = 7, 8, 9, 10. To avoid this, we switchto row elimination with partial pivoting to eliminate the (4, 3), (5, 3), (6, 3)elements, which does not introduce fill-in. We continue using row elimina-tion with partial pivoting until a step is reached when fill-in could occur, atwhich point we switch back to the “column pivoting, column elimination”scheme. This strategy leads to a decomposition

A = PLBUQ,

where P,Q are permutation matrices recording the row and column inter-changes, respectively, the unit lower and unit upper triangular matrices L,U contain the multipliers used in the row and column eliminations, respec-tively, and the matrix B has the structure shown in Figure 4, where · denotesa zeroed element. Since there is only one row or column interchange at eachstep, the pivotal information can be stored in a single vector of the order of

49

Page 50: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

Figure 5: The coefficient matrix of the reordered system

the matrix, as in conventional Gaussian elimination. The nonzero elementsof L,U can be stored in the zeroed positions in A. The pattern of row andcolumn eliminations is determined by NT and the number of rows NW ina general block W (i) (cf. Figure 2). In general, a sequence of NT columneliminations is alternated with a sequence of NW −NT row eliminations.

To solve Ax = b, we solve

PLz = b, Bw = z, UQx = w.

The second step requires particular attention. If the components of w areordered so that those associated with the column eliminations precede thoseassociated with the row eliminations, and the equations are ordered accord-ingly, the system is reducible. In our example, if we use the ordering

w = [w1, w2, w5, w6, w9, w10, w3, w4, w7, w8, w11]T ,

the coefficient matrix of the reordered equations has the structure in Figure5. Thus, the components w1, w2, w5, w6, w9, w10, are determined by solving alower triangular system and the remaining components by solving an uppertriangular system.

5.4 Modified Alternate Row and Column Elimination

The alternate row and column elimination procedure can be made moreefficient by using the fact that, after each sequence of row or column elimi-nations, a reducible matrix results. This leads to a reduction in the numberof arithmetic operations because some operations involving matrix-matrix

50

Page 51: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

multiplications in the decomposition phase can be deferred to the solutionphase, where only matrix-vector multiplications are required. After the firstsequence of column eliminations, involving a permutation matrix Q1 andmultiplier matrix U1, say, the resulting matrix is

B1 = AQ1U1 =

C1 O

M1

A1

O

,where C1 ∈ R2×2 is lower triangular, M1 ∈ R4×2 and A1 ∈ R9×9. Theequation Ax = b becomes

B1x = b, x = U−11 QT

1 x =

(x1

x2

),

and x1 ∈ R2. Setting b =

(b1

b2

), where b1 ∈ R2, we obtain

C1x1 = b1, A1x2 = b2 −(M1x1

0

)≡ b2.(5.1)

The next sequence of eliminations, that is, the first sequence of row elimi-nations, is applied only to A1 to produce another reducible matrix

L1P1A1 =

(R1 N1 O

O A2

),

where R1 ∈ R2×2 is upper triangular, N1 ∈ R2×3 and A2 ∈ R7×7. If

x2 =

(x1

x2

), L1P1b2 =

(b1

b2

),

where x1, b1 ∈ R2, system (5.1) becomes

A2x2 = b2, R1x1 = b1 − [N1 O] x2,(5.2)

and the next sequence of eliminations is applied to A2, which has the struc-ture of the original matrix with one W block removed. Since row operationsare not performed on M1 in the second elimination step, and column op-erations are not performed on N1 in the third, etc., there are savings in

51

Page 52: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

arithmetic operations [11]. The decomposition phase differs from that in al-ternating row and column elimination in that if the rth elimination step is acolumn (row) elimination, it leaves unaltered the first (r−1) rows (columns)of the matrix. The matrix in the modified procedure has the same structureand diagonal blocks as B, and the permutation and multiplier matrices areidentical.

In [11, 12], this modified alternate row and column elimination procedurewas developed and implemented in the package colrow for systems withmatrices of the form in Figure 2, and in the package arceco for ABD systemsin which the blocks are of varying dimensions, and the first and last blocksprotrude, as shown in Figure 1.

A comprehensive survey of the occurrence of, and solution techniquesfor, ABD linear systems is given in [3].

6 First Order Systems

6.1 Introduction

Consider the two–point boundary value problem for the first order systemgiven by

u′ = f(x,u), x ∈ I,(6.1)

subject to the separated boundary conditions

g0(u(0)) = 0, g1(u(1)) = 0,(6.2)

where u and f are vector functions of order m, g0 and g1 are vector func-tions of order m1 and m2 respectively, with m1 + m2 = m, and the primedenotes differentiation with respect to x. In most practical applications, fis nonlinear while g0 and g1 are both linear.

Equations of the form (6.1) may arise in practice or they may be derivedfor one or more higher order equations of the form

u(k) = f(x, u, u′, . . . , u(k−1)),

subject to appropriate boundary conditions. By introducing

u1 = u, u2 = u′, . . . , uk = u(k−1),

the first order system corresponding to such an equation is

u′i = ui+1 i = 1, 2, . . . , k − 1,u′k = f(x, u1, . . . , uk),

52

Page 53: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

subject to rewritten boundary conditions. In many cases, it is desirable toreduce the higher–order equations in this way and to use a solution approachthat is suitable for first order systems. However, it should be noted thatthere are situations where such a reduction is not necessary and approachesapplicable to the original higher–order equation are more appropriate.

6.2 The Trapezoidal Rule

Ifπ : 0 = x0 < x1 < . . . < xN+1 = 1

is a partition of I and hi = xi − xi−1, then the trapezoidal rule for thesolution of (6.1)–(6.2) takes the form

φi(U) ≡ Ui −Ui−1 − 12hif(xi,Ui) + f(xi−1,Ui−1) = 0, i = 1, . . . , N + 1,

φL(U) ≡ g0(U0) = 0, φR(U) ≡ g1(UN+1) = 0,

where Ui ≈ u(xi), i = 0, 1, . . . , N + 1. The determination of the approxi-mation

U ≡ [UT0 ,U

T1 , · · · ,UT

N+1]T

requires the solution of a system of m(N + 2) equations of the form

Φ(U) ≡

φL(U)

φ0(U)

...

φN+1(U)

φR(U)

= 0.(6.3)

This system is usually solved by Newton’s method (or a variant of it), whichtakes the form

(J (Uν−1)∆Uν = −Φ(Uν−1),

Uν = Uν−1 + ∆Uν , ν = 1, 2, . . . ,(6.4)

where U0 is an initial approximation to U and the (block) matrix

J (U) ≡ ∂Φ(U)∂U

=

(∂φi

∂Uj

)

53

Page 54: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

is the Jacobian of the nonlinear system (6.3). In this case, this matrix hasthe ABD structure

A

L1 R1

L2 R2

· ·

· ·

LN+1 RN+1

B

(6.5)

where

A =∂g0

∂U0is m1 ×m,

Li = −[I +

12hi

∂f∂Ui−1

(xi−1,Ui−1)]

is m×m,

Ri = I − 12hi

∂f∂Ui

(xi,Ui) is m×m,

and

B =∂g1

∂UN+1is m2 ×m.

It can be shown that the approximation U determined from the trapezoidalrule satisfies

u(xi)−Ui = O(h2),

where h = maxj hj . Under sufficient smoothness conditions on u, higher–order approximations to u on the mesh π can be generated using the methodof deferred conditions (cf. Section 1.5.1), which we shall describe briefly;full details are given by Lentini and Pereyra (1977).

The local truncation error of the trapezoidal rule is defined as

τπ,i[u] = φi(u)− [u′ − f(x,u)]|x=xi−1/2, i = 1, . . . , N + 1,

where xi−1/2 = xi−1 + 12hi. By expanding in Taylor series about xi−1/2, it

is easy to show that

τπ,i[u] =L∑

ν=1

Tν(xi−1/2)h2νi +O(h2L+2), i = 1, . . . , N + 1,(6.6)

54

Page 55: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

where

Tν(xi−1/2) = − ν

22ν−1(2ν + 1)!d2ν

dx2νf(x,u)|x=xi−1/2

.

If Sk(U(k−1)) is a finite difference approximation of order h2k+2 to the first kterms in the expansion (6.6) of the local truncation error, then the solutionU(k) of the system

φ(v) = Sk(U (k−1)), k = 1, 2, . . . , L,(6.7)

where U(0) = U, the solution of (6.3), is an order h2k+2 approximation to uon the mesh π. Details of the construction of the operators Sk by numericaldifferentiation are given by Lentini and Pereyra (1974).

Note that the systems (6.3) and (6.7) are similar, the right hand side of(6.7) being simply a known vector. Once the first system (6.3) is solved, theremaining systems (6.7) are small perturbations of (6.3) and considerablecomputational effort can be saved by keeping the Jacobian, and thereforeits LU decomposition, fixed during the iteration (6.4). Of course, if themesh π is changed, the Jacobian must be recomputed and refactored. Inprinciple, this modification degrades the quadratic convergence of Newton’smethod, but because of the accuracy of the approximate Jacobian and theinitial approximation, convergence is still rapid.

6.3 Orthogonal Spline Collocation

This method produces an approximation to u which is a piecewise poly-nomial vector function U(x) = (U1(x), . . . , Um(x)), where, for each i, i =1, . . . ,m, Ui ∈ Mr

0(π). Each piecewise polynomial Ui is represented in theform

Ui(x) =s∑

l=1

αliBl(x),(6.8)

where s = r(N + 1) + 1, and Bl(x)Ml=1 is a convenient basis, commonly

known as the B–spline basis (de Boor, 1978), for the piecewise polynomialspace Mr

0(π). The coefficients αli are determined by requiring that theapproximate solution U satisfy the boundary conditions, and the differentialequation at r specific points, the collocation points, in each subinterval.These points are the Gauss points defined by

ξ(i−1)(r−1)+k = xi−1 + hiρk, i = 1, . . . , N + 1, k = 1, . . . , r,

where ρkrk=1 are the r zeros of the Legendre polynomial of degree r on

the interval [0, 1] (cf., Section 4.1). The collocation equations then take the

55

Page 56: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

form

φ0(U) ≡ g0(U(0)) = 0,

φl(U) ≡ U′(ξj)− f(ξj ,U(ξj)) = 0, j = 1, 2, . . . , s− 2,

φs(U) ≡ g1(U(1)) = 0.

(6.9)

Again a variant of Newton’s method is usually used to solve the nonlinearsystem (6.9). Certain properties of the B–splines ensure that the Jacobianof (6.9) is almost block diagonal but having a more general structure thanthat of the matrix (6.5). These properties are (de Boor, 1978):

• On each subinterval [xi−1, xi] only r+1 B–splines are non–zero, namely

Br(i−2)+1, . . . , Br(i−1)+1.

• At x = 0, Bi(0) = δi,1 and at x = 1, Bi(1) = δi,s where δij denotes theKronecker delta.

If the unknown coefficients in (6.8) are ordered by “columns”, that is, firstwith respect to i and then with respect to l, the Jacobian of (6.9) has theform

A

W11 W12 W13

W21 W22 W23

·

·

·

WN+1,1 WN+1,2 WN+1,3

B

where the matrices A and B are m1 × m and m2 × m respectively, andWi1,Wi2 and Wi3 are mr ×m, mr ×m(r − 1) and mr ×m, respectively.

Under sufficient smoothness conditions, it can be shown (see, for exampleAscher et al., 1979) that the error in U for x ∈ [xi−1, xi] is given by

uj(x)− Uj(x) = c(x)u(r+1)j (xi−1)hr+1

i +O(hr+2), j = 1, . . . ,m,

where c(x) is a known bounded function of x, and at each mesh point xi,

uj(xi)− Uj(xi) = O(h2r), j = 1, . . . ,m, i = 0, 1, . . . , N + 1.

56

Page 57: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

6.4 Multiple Shooting

This procedure involves finding an approximation Ui to u(xi), i = 0, 1, . . . , N+1, in the following way (cf. Stoer and Bulirsch, 1980). Let ui(x;Ui) denotethe solution of the initial value problem

u′ = f(x,u), u(xi) = Ui.(6.10)

Then the method consists in finding the vectors Ui, i = 0, 1, . . . , N+1, suchthat

a) the function defined by

u(x) = ui(x;Ui) on [xi, xi+1], i = 0, 1, . . . , N,u(1) = UN

is continuous on I, and is thus a solution of the differential equation(6.1), and

b) the function u satisfies the boundary conditions (6.1).

These requirements lead to the equations

g0(U0) = 0,(6.11)Ui+1 − ui(xi+1;Ui) = 0, i = 0, . . . , N,(6.12)

g1(UN+1) = 0,(6.13)

where equations (6.12) arise from the continuity constraints, and (6.11) and(6.13) from the boundary conditions. When Newton’s method is used tosolve this system, the Jacobian is again almost block diagonal, and, in thiscase, takes the form

A

L0 I

L1 I

· ·

· ·

LN I

B

57

Page 58: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

where

A =∂g0

∂U1is m1×m, Li = −∂ui(xi+1;Ui)

∂Uiis m×m, B =

∂g1

∂UNis m2×m,

and I denotes the m × m unit matrix. In order to determine the m × mmatrices Li, i = 0, 1, . . . , N , one must solve the initial value problems

L′i(x) = J(x,ui(x;Ui))Li(x), Li(xi) = I, i = 0, . . . , N,(6.14)

where J is the Jacobian matrix J =∂f∂u

. Then Li = Li(xi+1). Giventhe function f and its Jacobian J , one can solve the initial value problems(6.10) and (6.14) simultaneously using the values of U calculated from theprevious Newton iteration. In practice, these initial value problems aresolved numerically using an initial value routine, usually a Runge–Kuttacode.

6.5 Software Packages

**(THIS SECTION IS OUTDATED)** In recent years, several general–purpose, portable software packages capable of handling nonlinear boundaryvalue problems involving mixed order systems of ordinary differential equa-tions have been developed. The use of these packages has led to the solutionof problems which were difficult and/or costly to solve, if not intractable,using standard solution procedures, and has removed much of the guessworkfrom the problem of solving boundary value problems for ordinary differen-tial equations, (see, for example, Davis and Fairweather (1981), Denison etal. (1983), Muir et al. (1983), Fairweather and Vedha–Nayagam, (1987),Bhattacharya et al. (1986), Akyurtlu et al. (1986), Vedha–Nayagam et al.(1987)).

In this section, we shall discuss three packages which are widely availableand are now in common use. In addition to implementing one of the basicnumerical methods described in Sections 8.2-8.4, each package incorporatesa sophisticated nonlinear equations solver, which involves a linear equationssolver, and a complex, fully–automatic mesh selection and error estimationalgorithm upon which the success of the code largely depends.

The IMSL package, DVCPR, called BVPFD in the new IMSL Library,implements a variable order finite difference method based on the trapezoidalrule with deferred corrections. Like the package DD04AD in the HarwellSubroutine Library (1978) and D02RAF in the NAG Library, this package

58

Page 59: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

is a version of the code PASVA3 (Pereyra, 1979). The code PASVA3 hasfairly long history and a series of predecessors has been in use for severalyears. The package COLSYS (Ascher et al., 1981), which implements splinecollocation at Gauss points, and IMSL’s multiple shooting code, DTPTB ,called BVPMS in the new IMSL Library, are of a more recent vintage. Allthree codes are documented elsewhere, and in this section we shall onlybriefly mention some of their similarities and differences, and other note-worthy features of the codes. It should be noted that Bader and Ascher(1987) have developed a new version of COLSYS called COLNEW whichdiffers from COLSYS principally in its use of certain monomial basis func-tions instead of B-splines. This new code, which is reputed to be somewhatmore robust that its predecessor, shares many of its features, and, in theremainder of this section, any comments referring to COLSYS apply equallywell to COLNEW .

The codes DVCPR and DTPTB require that the boundary value prob-lem be formulated as a first–order system, and they can handle nonseparatedboundary conditions, i.e. boundary conditions for (1.1), for example, of theform

g(u(0),u(1)) = 0,

where g is a vector function of order m. On the other hand, COLSYS canhandle a mixed–order system of multipoint boundary value problems with-out first reducing it to a first–order system, but requires that the boundaryconditions be separated. This restriction is not a serious one as a boundaryvalue problem with nonseparated boundary conditions can be reformulatedso that only separated boundary conditions occur, as we shall see in Section10.2. However this reformulation does increase the size of the problem.

The algebraic problem arising in the codes are similar but there is nouniformity in the manner in which they are solved. Each code uses somevariant of Newton’s method for the solution of the nonlinear systems, and aGauss elimination–based algorithm to solve the almost block diagonal linearsystems. Since the codes were written, the package COLROW , (Diaz et al.,1983) has been developed specifically for the solution of such linear systems;see Section 9.4. Its use in the existing codes has not been investigated, butmay improve their efficiency significantly.

Both DVCPR and COLSYS solve the boundary value problem on a se-quence of meshes until user–specified error tolerances are satisfied. Detaileddescriptions and theoretical justification of the automatic mesh–selectionprocedures used in DVCPR and COLSYS are presented by Lentini andPereyra (1974) and Ascher et al. (1979), respectively. It is worth noting

59

Page 60: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

that DVCPR constructs meshes in such a way that each mesh generatedcontains the initial mesh, which is not necessarily the case in COLSYS .

In DTPTB , the “shooting points” x1, . . . , xN , are chosen by the user or,for linear problems, by the code itself. Any adaptivity in the code appearsin the initial value solver, which in this case is IMSL’s Runge–Kutta code,DVERK .

In DVCPR, one can specify only an absolute tolerance, ε, say, which isimposed on all components of the solution. The code attempts to find anapproximate solution U such that the absolute error in each of its compo-nents is bounded by ε. On the other hand, COLSYS allows the user tospecify a different tolerance for each component of the solution, and in factallows one to impose no tolerance at all on some or all of the components.This code attempts to obtain an approximate solution U such that

maxx∈[xi,xi+1]

|ul(x)− Ul(x)| ≤ εl + maxx∈[xi,xi+1]

|Ul(x)|εl

for lth component on which the tolerance εl is imposed. This criterion hasa definite advantage in the case of components with different order of mag-nitude or when components differ in magnitude over the interval of defini-tion. Moreover, it is often more convenient to provide an array of tolerancesthan to rescale the problem. On successful termination, both DVCPR andCOLSYS return estimates of the errors in the components of approximatesolution.

In DTPTB , the same absolute tolerance is imposed on all componentsof the solution. In addition, a boundary condition tolerance is specified,and, on successful termination, the solution returned will also satisfy theboundary conditions to within this tolerance.

Each code offers the possibility of providing an initial approximation tothe solution and an initial subdivision of the interval of definition (the shoot-ing points in the case of DTPTB). With COLSYS , one may also specify thenumber of collocation points per subinterval. In many instances, parame-ter continuation is used to generate initial approximations and subdivisions.That is, when solving a parameterized family of problems in increasing or-der of difficulty, the subdivision and initial approximation for a particularproblem are derived from the final mesh and the approximate solution cal-culated in the previous problem. This is an exceedingly useful techniquewhich improves the robustness of the packages. It should be noted thatCOLSYS requires a continuous initial approximation, which is sometimesan inconvenience.

60

Page 61: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

It has been the authors’ experience that DTPTB is the least robust ofthe three codes. However W. H. Enright and T. F. Fairgrieve (private com-munication) have made some modifications to this code which have greatlyimproved its performance, making it competitive with both DVCPR andCOLSYS . In general, there is little to choose between DVCPR and COL-SYS , and it is recommended that both be used to solve a given boundaryvalue problem. Driver programs for these codes are straightforward to write.Moreover, there are definite advantages to using both codes. For example,each provides its own insights when solving a problem, simple programmingerrors can be detected more quickly by comparing results, and considerableconfidence can be attached to a solution obtained by two different methods.With the wide availability of these software packages, it is rarely advisableor even necessary for the practitioner to develop ad hoc computer programsfor solving boundary value problems ordinary differential equations.

7 Reformulation of Boundary Value Problems intoStandard Form

7.1 Introduction

Boundary value problems arising in various applications are frequently not inthe form required by existing general purpose software packages. However,many problems can be easily converted to such a form thus enabling theuser to take advantage of the availability of this software and avoid the needto develop a special purpose code.

In this section, we describe the conversion to “standard” form of somerather common “nonstandard” problems which the author has encounteredin applications in chemical engineering and mechanical engineering. Someof the material presented is adapted from the excellent survey article [5],where additional examples may be found.

7.2 Nonseparated Boundary Conditions

The standard form required by several boundary value problem codes is

u′ = f(x,u), x ∈ I,

g(u(0),u(1)) = 0,(7.1)

where u, f and g have m components and f and/or g may be nonlinear.The boundary conditions in (7.1) are said to be nonseparated.

61

Page 62: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

While COLSYS/COLNEW can handle mixed order systems directly,it does not accept nonseparated boundary conditions. A simple way toconvert a BVP of the form (7.1) with nonseparated boundary conditions toone with separated boundary conditions, where each condition involves onlyone point, is the following. We introduce the constant function v such that

v(x) = u(1), x ∈ I,

and obtain the equivalent boundary value problem

u′ = f(x,u), v′ = 0, x ∈ I,

g(u(0),v(0)) = 0, u(1) = v(1).

The penalty in this approach is that the transformed problem is twice theorder of the original problem. On the other hand, as we have seen, the ap-plication of standard techniques to boundary value problems with separatedboundary conditions involves the solution of almost block diagonal linearsystems. The algebraic problem is significantly more involved when thesetechniques are applied directly to the boundary value problem (7.1).

7.3 Singular Coefficients

In problems involving cylindrical or spherical geometry, ordinary differentialequations with singular coefficients often occur. A prime example is theproblem of diffusion and reaction in porous catalytic solids which gives riseto a boundary value problem of the form

u′′ +s

xu′ = R(u), x ∈ I,(7.2)

subject to the boundary conditions

u′(0) = 0, u(1) = 1.(7.3)

In (7.2), the quantity s, is a geometric factor which depends on the ge-ometry of the catalyst pellets: s = 0, 1, 2 for rectangular, cylindrical, andspherical geometries, respectively. When s = 1 or 2, codes such as DVCPRand DTPTB cannot be used directly because of the presence of the sin-gular coefficient in the differential equation (7.2). Since u′(x) = 0, and

limx→0u′(x)x

= u′′(0), the differential equation at x = 0 is replaced by

u′′ =R(u)

(1 + s).

62

Page 63: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

With this modification, the codes DVCPR (a predecessor of BVPFD) andDTPTB (a predessor of BVPMS) have been used successfully to solve prob-lems of the form (7.2)–(7.3); see, for example, [15, 18].

7.4 Continuation

Many problems arising in engineering applications depend on one (or more)parameters. Consider, for example, the boundary value problem

u′ = f(x,u;λ), x ∈ I,

g(u(0),u(1);λ) = 0,(7.4)

where λ is a parameter. For the desired value of λ, λ1 say, the problem (7.4)might be exceedingly difficult to solve without a very good initial approx-imation to the solution u. If the solution of (7.4) is easily determined foranother value of λ, λ0 say, then a chain of continuation steps along the ho-motopy path (λ,u( · ;λ)) may be initiated from (λ0,u( · ;λ0)). At each stepof the chain, standard software can be used to determine an approximationto u( · ;λ) which is then used, at the next step, as the initial approximationto u( · ;λ+∆λ). This procedure can also be used when the solution of (7.4)is required for a sequence of values of λ, and the problems are solved inincreasing order of difficulty.

As an example, consider the boundary value problem

u′′ +s

xu′ + λeu = 0, x ∈ I,

u′(0) = 0, u(1) = 0,(7.5)

which describes the steady–state temperature distribution in the radial di-rection in the interior of a cylinder of unit radius when s = 1, and a sphereof unit radius when s = 2 with heat generation according to the exponentiallaw [21]. There is no solution for λ > 1.7 when s = 1 and for λ > 3.0 whens = 2. The quantity of interest in this problem is the dimensionless cen-ter temperature u(0). The boundary value problem (7.5) has been solvedfor various values of λ in [18] using DVCPR and DTPTB (replacing thedifferential equation in (7.5) by

u′′ +λeu

(1 + s)= 0,

when x = 0, as described in section 3) and COLSYS , and comparisons madewith results obtained in [21]. The results for the case s = 1 are presented in

63

Page 64: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

λ [21] COLSYS0.1 0.252(−1) 0.254822(−1)

0.2 0.519(−1) 0.519865(−1)

0.3 0.796(−1) 0.796091(−1)

0.4 0.109 0.108461

0.5 0.138 0.138673

0.6 0.169 0.170397

0.7 0.2035 0.203815

0.8 0.238 0.239148

1.0 0.65 0.316694

1.5 1.1 0.575364

1.7 2.0 0.731577

Table 1: Dimensionless center temperature u(0) for the case s = 1.

Table 1. For each value of λ, COLSYS converged without difficulty whileit was necessary to use continuation with the other codes; DVCPR was runstarting with λ = 1.7 and decreasing λ in steps of 0.1 to λ = 0.1 but didnot converge for λ = 1.7, while DTPTB was run with λ increasing in stepsof 0.1 from 0.1. In most cases, the results produced by DVCPR (except forλ = 1.7) and DTPTB agree to six significant figures with those obtained byCOLSYS , casting doubt on the validity of the results [21] for λ ≥ 1. Detailsof these numerical experiments and a discussion of the case s = 2 are givenin [18].

7.5 Boundary Conditions at Infinity

Consider the following boundary value problem on the semi–infinite interval[0,∞):

u′ = f(x,u), x ∈ [0,∞),

g(u(0), limx→∞ u(x)) = 0.(7.6)

Usually the components of the solution u approximate their asymptoticvalues quite accurately at a value x, L say, of moderate size. The boundary

64

Page 65: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

value problem (7.6) can then be replaced by the boundary value problem

u′ = f(x,u), x ∈ [0, L],

g (u(0),u(L)) = 0.

It is usually necessary to determine the value of L experimentally; this canbe facilitated by using the transformation

t =x

L

to map the interval [0, L] to the unit interval. The quantity L then appearsas a parameter in the differential equation and continuation as described inSection 7.4 can be used to determine an appropriate value for L.

Consider the following example. In the study conducted by Na andPop [20] of free–convective flow past a vertical plate embedded in a satu-rated porous medium, the governing equations reduce to the boundary valueproblem

f ′′′ + 13(m+ 2)ff ′′ − 1

3(2m+ 1)(f ′)2 = 0, x ∈ [0,∞),

f(0) = 0, f ′′(0) = −1, f ′(∞) = 0,(7.7)

where m is a parameter. The boundary condition at infinity is replaced byf ′(L) = 0, and, with z = x/L, (7.7) becomes

f ′′′ + 13L(m+ 2)ff ′′ − 1

3L(2m+ 1)(f ′)2 = 0, z ∈ I,

f(0) = 0, f ′′(0) = −L2, f ′(1) = 0,(7.8)

which can now be solved directly by COLSYS or reduced to a first ordersystem and solved by DVCPR or DTPTB . The quantity of interest in thisproblem is f ′(0) and the boundary value problem (7.8) is solved for anincreasing sequence of values of L until f ′(0) remains constant to the desirednumber of significant figures; see [18].

7.6 Eigenvalue Problems

As an example of problems in this category, consider the following boundaryvalue problem which arises in the study of squeezing flow of a viscous fluidbetween elliptic plates [6]:

f ′′′ + k = SF (f, g), η ∈ I,

g′′′ + βk = SF (g, f), η ∈ I,(7.9)

65

Page 66: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

S −0.5 0.0 1.0 25.0(a) 1.3023 3.0000 6.2603 73.8652

(b) 1.3036 3.0018 6.2778 74.0414

Table 2: Values of k for β = 1 and various values of S.

whereF (ϕ,ψ) = 2ϕ′ + ηϕ′′ +

12(ϕ′)2 − 1

2ϕ′′(ϕ+ ψ),

subject to the boundary conditions

f(0) = f ′′(0) = g(0) = g′′(0) = 0,

f(1) + g(1) = 2, f ′(1) = g′(1) = 0.(7.10)

In (7.9), the parameters β and S are prescribed, and k is an unknownconstant. If we add to the boundary value problem (7.9)–(7.10) the trivialdifferential equation

k′ = 0,

the augmented boundary value problem can be solved using standard soft-ware. In Table 2, we give the values of k for β = 1 and various values of Sdetermined by:

(a) solving the augmented boundary value problem using COLSYS andDVCPR;

(b) [6].

Since COLSYS and DVCPR produce the same results to the number offigures presented, it seems reasonable to assume that the results given by(a) are the correct values.

7.7 Integral Constraints

In many boundary value problems, a term of the form∫ 1

0G(u(t), t) dt = R,

where R is a given constant, occurs as a constraint. In such cases, we define

w(x) =∫ x

0G(u(t), t) dt,

66

Page 67: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

and replace the constraint by the differential equation

w′(x) = G(u(x), x)

and the boundary conditions

w(0) = 0, w(1) = R.

As an example, consider the following boundary value problem which arisesin a study of membrane separation processes:

u′′ +1xu′ + c1 + c2(1− e−ϕ(x,k)) [ψ(u, ϕ)− 1] + c3ψ(u, ϕ)u = 0,(7.11)

whereϕ(x, k) = k/(c4 − xk),

ψ(u, ϕ) = eu/[1 + c5eϕ(eu − 1)],

subject to

u′(0) = 0, u(1) = 0,(7.12) ∫ 1

0ψ(u, ϕ)x dx = c6

∫ 1

0ux dx,(7.13)

In the boundary value problem (7.11)–(7.13), the quantities ci, i = 1, . . . , 6,are prescribed constants and the constant k is to be determined in additionto the function u. To reformulate this problem, we first write (7.13) as∫ 1

0F (u, x, k)x dx = 0,

and setw(x) =

∫ x

0F (u, x, k)x dx.

Then to the original boundary value problem we add

w′ = xF (u, x, k),w(0) = 0, w(1) = 0,

and the trivial ordinary differential equation

k′ = 0.

This augmented boundary value problem was solved successfully by [7] usingCOLSYS .

67

Page 68: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

7.8 Interface Conditions

In problems involving layered media, for example, one has interface condi-tions at known interior points. Such a problem is

Lu = 0, x ∈ [0, β],

Lu− ϕ21u = 0, x ∈ [β, 1],

(7.14)

where

Lu =d2u

dx2+

1x+ α

du

dx,

and α and ϕ1 are known constants, subject to

u(0) = 1,du

dx(1) = 0, u(β−) = u(β+),

du

dx(β−) =

du

dx(β+).(7.15)

To obtain a standard boundary value problem, we map the problem (7.14)–(7.15) to [0, 1], by setting z = x/β to map [0, β] to [0, 1], and then z =(1− x)/(1− β) maps [β, 1] to [1, 0]. With

L0u =d2u

dz2+

β

β2 + α

du

dz,

and

L1u =d2u

dz2+

β − 1(β − 1)2 + 1 + α

du

dz,

we obtainL0u1 = 0 L1u2 − ϕ2

1(β − 1)2u2 = 0(7.16)

andu1(0) = 1,

du2

dz(0) = 0,

u1(1) = u2(1),1β

du1

dz(1) =

1β − 1

du2

dz(1).

(7.17)

An additional complication arises when the location of the interface, x = β,is unknown. Such a problem is discussed in [2], where in addition to (7.14),(7.15), we have

Lv − ϕ22u = 0, x ∈ [β, 1],(7.18)

and the boundary conditions

v(1) = 0,dv

dx(β) = 0, v(1) = 1,(7.19)

68

Page 69: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

where ϕ2 is a given constant. Equation (7.18) is transformed to

L1v1 − ϕ22(β − 1)2u2 = 0, z ∈ I,(7.20)

and (7.19) becomes

v1(0) = 1, v1(1) = 0,dv1dz

(1) = 0.(7.21)

Since β is unknown, to the boundary value problem consisting of (7.16),(7.17), (7.20) and (7.21), we add the trivial ordinary differential equation

dz= 0, z ∈ I.

69

Page 70: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

8 Matrix Decomposition Algorithms for Poisson’sEquation

8.1 Introduction

In this section, we consider the use of finite difference, finite element Galerkinand orthogonal spline collocation methods on uniform partitions for the so-lution of Poisson’s equation in the unit square subject to Dirichlet boundaryconditions:

−∆u = f(x, y), (x, y) ∈ Ω,(8.1)

u(x, y) = 0, (x, y) ∈ ∂Ω,

where ∆ denotes the Laplacian and Ω = (0, 1)2. Each method gives rise toa system of linear equations of the form

(A⊗B +B ⊗A)u = f ,(8.2)

where A and B are square matrices of order M , say, u and f are vectors oforder M2 given by

u = [u1,1, . . . , u1,M , . . . , uM,1, . . . , uM,M ]T ,(8.3)

f = [f1,1, . . . , f1,M , . . . , fM,1, . . . , fM,M ]T ,(8.4)

and ⊗ denotes the tensor product; see Appendix D for the definition of ⊗and its properties. A matrix decomposition algorithm is a fast direct methodfor solving systems of the form (8.2) which reduces the problem to one ofsolving a set of independent one-dimensional problems. We first develop aframework for matrix decomposition algorithms. To this end, suppose thereal nonsingular matrix E is given and assume that a real diagonal matrixΛ and a real nonsingular matrix Z can be determined so that

AZ = BZΛ(8.5)

andZTEBZ = I,(8.6)

where I is the identity matrix of order M . Premultiplying (8.5) by ZTEand using (8.6), we obtain

ZTEAZ = Λ.(8.7)

70

Page 71: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

The system of equations (8.2) can then be written in the form

(ZTE ⊗ I)(A⊗B +B ⊗A)(Z ⊗ I)(Z−1 ⊗ I)u = (ZTE ⊗ I)f ,(8.8)

which becomes, on using the properties of ⊗, (8.6) and (8.7),

(Λ⊗B + I ⊗A)(Z−1 ⊗ I)u = (ZTE ⊗ I)f .(8.9)

From the preceding, we obtain the following algorithm for solving (8.2):

MATRIX DECOMPOSITION ALGORITHM

1. Determine the matrices Λ and Z satisfying (8.5) and (8.6).

2. Compute g = (ZTE ⊗ I)f .

3. Solve (Λ⊗B + I ⊗A)v = g.

4. Compute u = (Z ⊗ I)v.

In the discretization methods considered in this section, the matrices Λand Z are known explicitly. If Λ = diagλiM

i=1, Step 2 reduces to a systemof independent problems of the form

(A+ λjB)vj = gj , j = 1, . . . ,M.

It is shown that, for each method, these systems correspond to discretiza-tions of two-point boundary value problems of the form

−u′′ + λu = f(x), x ∈ (0, 1),(8.10)

u(0) = u(1) = 0,

where λ is a positive constant. The elements of the matrix Z are sines and/orcosines and as a consequence multiplication by Z or ZT can be done usingfast Fourier transforms (FFTs). Moreover, the matrix E is sparse, so thatmultiplication by E can be done very efficiently. When all of these proper-ties are exploited, the operation count for any of the matrix decompositionalgorithms discussed in this section is O(N2 logN), where (N +1)h = 1 andh is the mesh parameter.

71

Page 72: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

8.2 Finite Difference Method

As a simple example, consider the basic five point difference approximationfor Poisson’s equation. To describe this method, suppose h = 1/(N + 1),where N is a positive integer, and set xm = mh, yn = nh. Denote by um,n

an approximation to u(xm, yn) defined by the usual second order differenceequations

−um−1,n − 2um,n + um+1,n

h2−um,n−1 − 2um,n + um,n+1

h2= f(xm, yn), m, n = 1, ..., N,

(8.11)where u0,n = uN+1,n = um,0 = um,N+1 = 0. If we set M = N and introducevectors u and f as in (8.3) and (8.4), respectively, with fm,n = f(xm, yn),then the finite difference equations (8.11) may be written in the form

(A⊗ I + I ⊗A)u = f ,(8.12)

with A = J , where J is the tridiagonal matrix of order N given by (3.13).It is well known that (8.5) and (8.6) are satisfied if B = E = I,

Λ = diag(λ1, . . . , λN ),(8.13)

whereλj =

4h2

sin2 jπh

2, j = 1, . . . , N,(8.14)

and Z is the symmetric orthogonal matrix given by

Z = S,(8.15)

where

S =(

2N + 1

)1/2 (sin

mnπ

N + 1

)N

m,n=1.(8.16)

From the Matrix Decomposition Algorithm with Λ and Z defined by (8.13)and (8.15), respectively, we obtain the following matrix decomposition algo-rithm for solving (8.12).

FINITE DIFFERENCE ALGORITHM

1. Compute g = (S ⊗ I)f .

2. Solve (Λ⊗ I + I ⊗A)v = g.

3. Compute u = (S ⊗ I)v.

72

Page 73: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

Note that Steps 1 and 3 can be carried out using FFTs at a cost ofO(N2 logN)operations. Step 2 consists of N tridiagonal linear systems each of whichcan be solved in O(N) operations so that the total cost of the algorithm isO(N2 logN) operations. Each tridiagonal system corresponds to the stan-dard finite difference approximation to (8.10).

8.3 Finite Element Galerkin Methods with Piecewise Bilin-ear Elements

To obtain the weak form of the Poisson problem (8.1), let H10 (Ω) denote

the space of all piecewise continuously differentiable functions defined on Ωwhich vanish on ∂Ω. Then, for all v ∈ H1

0 (Ω), u(x, y) satisfies

−∆u(x, y)v(x, y) = f(x, y)v(x, y),

and−∫Ω

∆u(x, y)v(x, y)dx dy =∫Ωf(x, y)v(x, y)dx dy.(8.17)

On applying Green’s formula to (8.17) and using the fact that v = 0 on ∂Ω,we obtain ∫

Ω

[∂u

∂x

∂v

∂x+∂u

∂y

∂v

∂y

]dx dy =

∫Ωf(x, y)v(x, y)dx dy.(8.18)

For convenience, we introduce the notation

(g1, g2) =∫Ωg1(x, y) · g2(x, y)dx dy

for functions g1 and g2 with the dot denoting scalar multiplication in thecase of vector functions. Then (8.18) can be written in the form

(∇u,∇v) = (f, v) for all v ∈ H10 (Ω).(8.19)

This is the weak form of (8.1) on which the finite element Galerkin methodis based.

Now let xkN+1k=0 be a uniform partition of the interval [0, 1], so that

xk = kh, k = 0, . . . , N + 1, where the stepsize h = 1/(N + 1). Let wnNn=1

denote the standard basis for the space of piecewise linear functions definedon this partition which vanish at 0 and 1; see (3.10). Then the C0 piecewisebilinear Galerkin approximation

uh(x, y) =N∑

m=1

N∑n=1

um,nwm(x)wn(y)

73

Page 74: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

to the solution u of (2.1) is obtained by requiring that

(∇uh,∇v) = (f, v),(8.20)

for all piecewise bilinear functions v which vanish on ∂Ω. If M = N , and uand f are as in (8.3) and (8.4), respectively, with fm,n = (f, φmφn), then weobtain the linear system (8.2) in which the matrices A and B are given by(3.14). If E = I,

Λ = diag(λj/[1− 1

6λj ])N

j=1,(8.21)

and

Z = S diag(h[1− 1

6λj ]−1/2

)N

j=1,(8.22)

where λj and S are given by (8.14) and (8.16), respectively, then (8.5)and (8.6) are satisfied. Thus, with Λ and Z defined by (8.21) and (8.22),respectively, we have the following matrix decomposition algorithm:

FINITE ELEMENT GALERKIN ALGORITHM

1. Compute g = (ZT ⊗ I)f .

2. Solve (Λ⊗B + I ⊗A)v = g.

3. Compute u = (Z ⊗ I)v.

As in the finite difference method, Step 2 involves the solution of N tridi-agonal systems and consequently the computational cost of the algorithmis O(N2 logN) operations. In this case, each tridiagonal system arises fromthe Galerkin approximation of (8.10).

8.4 Orthogonal Bicubic Spline Collocation Method

Again, let xkN+1k=0 be a uniform partition of the interval [0, 1], and let

φnMn=1 , where M = 2N + 2 be a basis for M3,0

1 given by

φn = vn, n = 1, . . . , N, φN+n+1 = sn, n = 0, . . . , N + 1.(8.23)

74

Page 75: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

This is the standard basis but the ordering of the basis functions is nonstan-dard. The Hermite bicubic orthogonal collocation approximation

uh(x, y) =M∑

m=1

M∑n=1

um,nφm(x)φn(y)

to the solution u of (2.1) is obtained by requiring that

−∆uh(ξm, ξn) = f(ξm, ξn), m, n = 1, . . . ,M.(8.24)

With u and f as in (8.3) and (8.4), respectively, where fm,n = f(ξm, ξn),equations (8.24) can be written in the form (8.2) with

A = (amn)Mm,n=1, amn = −φ′′n(ξm), B = (bmn)M

m,n=1, bmn = φn(ξm).(8.25)

Let

Λ = diag(λ−1 , . . . , λ−N , λ0, λ

+1 , . . . , λ

+N , λN+1),(8.26)

where

λ±j = 12

(8 + ηj ± µj

7− ηj

)h−2, j = 1, . . . , N, λ0 = 36h−2, λN+1 = 12h−2,

and

ηj = cos(

N + 1

), µj =

√43 + 40ηj − 2η2

j .

To describe the matrix Z, let Λ±α , Λ±β be diagonal matrices defined by

Λ±α = diag(α±1 , . . . , α±N ), Λ−β = diag(β−1 , . . . , β

−N ), Λ+

β = diag(1, β+1 , . . . , β

+N , 1/

√3),

where

α±j = (5 + 4ηj ∓ µj)ν±j , β±j = 18 sin(

N + 1

)ν±j ,

and

ν±j =[27(1 + ηj)(8 + ηj ∓ µj)2 + (1− ηj)(11 + 7ηj ∓ 4µj)2

]−1/2.

Then

Z = 3√

3

[SΛ−α 0 SΛ+

α 0CΛ−β CΛ+

β

],(8.27)

75

Page 76: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

where 0 is the N -dimensional zero column vector, S is given by (8.16), and

C =(

2N + 1

)1/2 (cos

mnπ

N + 1

)N+1

m,n=0, C =

(2

N + 1

)1/2 (cos

mnπ

N + 1

)N+1,N

m=0,n=1.

It can be shown that if A, B, Λ, Z are given by (8.25), (8.26), (8.27), andE = BT , then equations (8.5) and (8.6) are satisfied. Thus, from the MatrixDecomposition Algorithm, we obtain:

ORTHOGONAL SPLINE COLLOCATION ALGORITHM

1. Compute g = (ZTBT ⊗ I)f .

2. Solve (Λ⊗B + I ⊗A)v = g.

3. Compute u = (Z ⊗ I)v.

Since there are at most four nonzero elements in each row of the matrix BT

the matrix-vector multiplications involving the matrix BT in step 1 requirea total of O(N2) arithmetic operations. From (8.27), it follows that FFTroutines can be used to perform multiplications by the matrix ZT in step1 and by the matrix Z in step 3, the corresponding cost of each step beingO(N2 logN) operations. Step 2 involves the solution of M independentalmost block diagonal linear systems with coefficient matrices of the form(4.7) arising from the orthogonal spline collocation approximation of (8.10),which can be solved in a total of O(N2) operations. Thus the total cost ofthis algorithm is also O(N2 logN) operations.

76

Page 77: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

References

1. S. Agmon, Lectures on Elliptic Boundary–Value Problems, Van Nos-trand, Princeton, New Jersey, 1965.

2. A. Akyurtlu, J. F. Akyurtlu, C. E. Hamrin Jr. and G. Fairweather, Re-formulation and the numerical solution of the equations for a catalytic,porous wall, gas–liquid reactor, Computers Chem. Eng., 10(1986),361–365.

3. P. Amodio, J. R. Cash, G. Roussos, R. W. Wright, G. Fairweather, I.Gladwell, G. L. Kraut and M. Paprzycki, Almost block diagonal linearsystems: sequential and parallel solution techniques, and applications,Numerical Linear Algebra with Applications, to appear.

4. E. Anderson, Z. Bai, C. Bischof, J. Demmel, J. Dongarra, J. Du Croz,A. Greenbaum, S. Hammarling, A. McKenney, S. Ostrouchov and D.Sorensen, LAPACK Users’ Guide, SIAM Publications, Philadelphia,1995.

5. U. Ascher and R. D. Russell, Reformulation of boundary value prob-lems into ‘standard’ form, SIAM Rev., 23(1981), 238–254.

6. A. Aziz and T. Y. Na, Squeezing flow of a viscous fluid between ellipticplates, J. Comput. Appl. Math., 7(1981), 115–119.

7. D. Bhattacharyya, M. Jevtitch, J. T. Schrodt and G. Fairweather,Prediction of membrane separation characteristics by pore distributionmeasurements and surface force–pore flow model, Chem.Eng. Com-mun., 42(1986), 111–128.

8. C. de Boor, A Practical Guide to Splines, Applied Math. Sciences 27,Springer–Verlag, New York, 1978.

9. C. de Boor and R. Weiss, SOLVEBLOK: A package for solving almostblock diagonal linear systems, ACM Trans. Math. Software, 6(1980),80–87.

10. C. de Boor and R. Weiss, Algorithm 546: SOLVEBLOK, ACM Trans.Math. Software, 6(1980), 88–91.

11. J.C. Diaz, G. Fairweather and P. Keast FORTRAN packages for solv-ing certain almost block diagonal linear systems by modified alternaterow and column elimination, ACM Trans. Math. Software, 9(1983),358–375.

77

Page 78: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

12. J. C. Diaz, G. Fairweather and P. Keast, Algorithm 603 COLROW andARCECO: FORTRAN packages for solving certain almost block diag-onal linear systems by modified alternate row and column elimination,ACM Trans. Math. Software, 9(1983), 376–380.

13. J. Douglas Jr. and T. Dupont, Collocation Methods for ParabolicEquations in a Single Space Variable, Lecture Notes in Mathematics385, Springer–Verlag, New York, 1974.

14. J. Douglas Jr., T. Dupont and L. Wahlbin, Optimal L∞ error estimatesfor Galerkin approximations to solutions of two–point boundary valueproblems Math. Comp., 29(1975), 475–483.

15. K. S. Denison, C. E. Hamrin Jr. and G. Fairweather, Solution ofboundary value problems using software packages: DD04AD and COL-SYS, Chem. Eng. Commun., 22(1983), 1–9.

16. E. J. Doedel, Finite difference collocation methods for nonlinear twopoint boundary value problems, SIAM J. Numer. Anal., 16(1979), 173–185.

17. G. Fairweather, Finite Element Galerkin Methods for Differential Equa-tions, Lecture Notes in Pure and Applied Mathematics, Vol. 34, Mar-cel Dekker, New York, 1978.

18. G. Fairweather and M. Vedha–Nayagam, An assessment of numericalsoftware for solving two–point boundary value problems arising in heattransfer, Numerical Heat Transfer, 11(1987), 281-293.

19. H. B. Keller, Numerical Methods for Two–Point Boundary Value Prob-lems, Dover, New York, 1992.

20. T. Y. Na and I. Pop, Free convection flow past a vertical flat plateembedded in a saturated porous medium, Int.J. Engrg. Sci., 21(1983),517–526.

21. T. Y. Na and S. C. Tang, A method for the solution of conduction heattransfer with non–linear heat generation, Z. Angew. Math. Mech.,49(1969), 45–52.

22. R. S. Stepleman, Tridiagonal fourth order approximations to generaltwo–point nonlinear boundary value problems with mixed boundaryconditions, Math. Comp., 30(1976), 92–103.

78

Page 79: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

23. J. M. Varah, Alternate row and column elimination for solving certainlinear systems, SIAM J. Numer. Anal., 13(1976), 71–75.

24. M. F. Wheeler, A Galerkin procedure for estimating the flux for two–point boundary value problems, SIAM J. Numer. Anal., 11(1974), 764–768.

Appendix A. Galerkin Matrices for Piecewise Linear Functions.For the matrix, A = ((w′i, w

′j)):

(w′i, w′i) =

∫ 1

0[w′i(x)]

2 dx

=∫ xi

xi−1

1(xi − xi−1)2

dx+∫ xi+1

xi

1(xi+1 − xi)2

dx

=[

1(xi − xi−1)

+1

(xi+1 − xi)

]=

1hi

+1

hi+1;

(w′i, w′i+1) =

∫ xi+1

xi

−1(xi+1 − xi)2

dx = − 1(xi+1 − xi)

= − 1hi+1

;

(w′i, w′i−1) = − 1

hi.

For the matrix B = ((wi, wj)) :

(wi, wi) =∫ xi

xi−1

(x− xi−1

xi − xi−1

)2

dx+∫ xi+1

xi

(x− xi

xi − xi+1

)2

dx

=13[hi + hi+1];

(wi, wi+1) =∫ xi+1

xi

(xi+1 − x

xi+1 − xi

)(x− xi

xi+1 − xi

)dx

=1

h2i+1

12(xi+1 − x)(x− xi)2|xi+1

xi+

12

∫ xi+1

xi

(x− xi)2 dx

=16hi+1.

Similarly,

(wi, wi−1) =16hi.

79

Page 80: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

Appendix B. Galerkin Matrices for Piecewise Hermite CubicFunctions. We consider the case in which the partition π of I is uniformand let h = xi+1 − xi, i = 0, . . . , N . The conditioning of the matrices isimproved if the slope functions are normalized by dividing by h. Thus wedefine

si(x) = h−1si(x), i = 0, . . . , N + 1.

If the Galerkin solution uh is expressed in the form

uh(x) =N+1∑i=0

α(1)i vi(x) + α

(2)i si(x),

thenα

(1)i = uh(xi),

as before, but now

α(2)i = hu′h(xi), i = 0, . . . , N + 1.

If A and B are partitioned as in (3.16) then, with the normalization ofthe slope functions, we have, if A = ((w′i, w

′j)) ≡ (Aij) and B = ((wi, wj)) ≡

(Bij),

A00 = (30h)−1

[36 3

3 4

],

Aii = (15h)−1

[36 0

0 4

], i = 1, . . . , N,

AN+1,N+1 = (30h)−1

[36 −3

−3 4

],

Ai,i+1 = ATi+1,i = (30h)−1

[−36 3

−3 −1

], i = 0, . . . , N,

and

B00 =h

210

[78 11

11 2

],

Bii =h

105

[78 0

0 2

], i = 1, . . . , N,

80

Page 81: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

BN+1,N+1 =h

210

[78 −11

−11 2

],

Bi,i+1 = BTi+1,i =

h

420

[54 −13

13 −3

], i = 0, . . . , N.

Appendix C. Elements of Piecewise Hermite Cubic OrthogonalSpline Collocation Matrices. On [xi, xi+1], with xi = ih,

vi(x) = −2[xi+1 − x

h

]3+ 3

[xi+1 − x

h

]2,

v′i(x) =6h3

(xi+1 − x)2 − 6h2

(xi+1 − x),

v′′i (x) = −12h3

(xi+1 − x) +6h2.

From (4.5), ξ2i+1 = xi + 12h(1 + ρ1), where ρ1 = −1/

√3. Then, if x = ξ2i+1

xi+1 − x =12h(1− ρ1)

and

vi(ξ2i+1) = −14(1− ρ1)3 +

34(1− ρ1)2

=14(1− ρ1)2(2 + ρ1)

v′i(ξ2i+1) =32h

(1− ρ1)2 −3h

(1− ρ1)

= − 32h

(1− ρ21)

v′′i (ξ2i+1) = − 6h2

(1− ρ1) +6h2

=6ρ1

h2

Similar expressions hold for vi(ξ2i+2), v′i(ξ2i+2), v′′i (ξ2i+2), where ξ2i+2 =xi + 1

2h(1 + ρ2), ρ2 = −ρ1.On [xi, xi+1],

vi+1(x) = −2[x− xi

h

]3+ 3

[x− xi

h

]2,

v′i+1(x) = − 6h3

(x− xi)2 +6h2

(x− xi)

v′′i+1(x) = −12h3

(x− xi) +6h2.

81

Page 82: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

If x = ξ2i+1,

x− xi =12h(1 + ρ1).

Therefore,

vi+1(ξ2i+1) = −14(1 + ρ1)3 +

34(1 + ρ1)2

=14(1 + ρ1)2(2− ρ1),

v′i+1(ξ2i+1) = − 32h

(1 + ρ1)2 +3h

(1 + ρ1)

=32h

(1− ρ21),

v′′i+1(ξ2i+1) = − 6h2

(1 + ρ1) +6

h2i+1

= −6ρ1

h2,

with similar expressions for vi+1(ξ2i+2), v′i+1(ξ2i+2), v′′i+1(ξ2i+2).With si = h−1si as in Appendix B, on [xi, xi+1],

si(x) = −[

xi+1 − x

h

]3−[xi+1 − x

h

]2,

s′i(x) = −− 3h3

(xi+1 − x)2 +2h2

(xi+1 − x),

s′′i (x) = −

6h3

(xi+1 − x)− 2h2

.

With xi+1 − x = 12h(1− ρ1), we obtain

si(ξ2i+1) = −

18(1− ρ1)3 −

14(1− ρ1)2

= −1

8(1− ρ1)21− ρ1 − 2

=18(1− ρ1)2(1 + ρ1),

s′i(ξ2i+1) = −−3h

(1− ρ1)2

4+

1h

(1− ρ1)

= − 14h

(1− ρ1)−3(1− ρ1) + 4

82

Page 83: Numerical Methods for Two–Point Boundary Value Problems · Numerical Methods for Two–Point Boundary Value Problems Graeme Fairweather and Ian Gladwell 1 Finite Difference Methods

= − 14h

(1− ρ1)(1 + 3ρ1),

s′′i (ξ2i+1) = −

3h2

(1− ρ1)−2h2

= − 1

h2(1− 3ρ1),

with similar expressions for si(ξ2i+2), s′i(ξ2i+2), and s′′i (ξ2i+2).On [xi, xi+1],

si+1(x) =[x− xi

h

]3−[x− xi

h

]2,

s′i+1(x) =3h3

(x− xi)2 −2h2

(x− xi),

s′′i+1(x) =6h3

(x− xi)−2h2.

Then, with x− xi = 12h(1 + ρ1), we have

si+1(ξ2i+1) =

18(1 + ρ1)3 −

14(1 + ρ1)2

= −1

8(1 + ρ1)2(1− ρ1),

s′i+1(ξ2i+1) =

34h

(1 + ρ1)2 −1h

(1 + ρ1)

= − 14h

(1 + ρ1)(1− 3ρ1),

s′′i+1(ξ2i+1) =1h23(1 + ρ1)− 2

=1h2

(1 + 3ρ1),

with similar expressions for si+1(ξ2i+2), s′i+1(ξ2i+2), s′′i+1(ξ2i+2).

Appendix D. Properties of the Tensor Product. If A = (aij) isan M ×M matrix and B is an N ×N matrix, then the tensor product of Aand B, A⊗B, is the MN ×MN block matrix whose (i, j) element is aijB.The tensor product has the following properties:

A⊗ (B + C) = A⊗B +A⊗ C;(B + C)⊗A = B ⊗A+ C ⊗A;

(A⊗B)(C ⊗D) = AC ⊗BD, provided the matrix products are defined.

83