gregory e. fasshauer - wit presssince their introduction in the 1970s radial basis functions (rbfs)...

On the numerical solution of differential

equations with radial basis functions

Gregory E. FasshauerDepartment of Applied Mathematics, Illinois Institute of Technology,

E-Mail: [email protected]

Abstract

In this paper we report on two different experiments dealing with the nu-merical solution of differential equations by radial basis functions: 1) thesolution of a two-point boundary value problem; 2) the solution of a two-dimensional Poisson equation. In the second experiment we contrast a mul-tilevel collocation algorithm based on locally supported basis functions withtwo different direct solution approaches (one based on locally supportedbasis functions, the other on globally supported multiquadrics) . In bothexperiments the effects of a smoothing operation are studied.

1. Introduction

Since their introduction in the 1970s radial basis functions (RBFs) haveattracted a lot of attention from mathematicians working in the area ofapproximation theory. More recently, however, a large number of scien-tists and engineers have discovered their merits when working with multi-dimensional data. Furthermore, the main focus of the applications seems tohave shifted from scattered data approximation to the numerical solutionof partial differential equations (PDEs). It is this second class of problemswe will address in this paper.

The RBFs being used fall into two general categories: globally sup-ported, and locally supported functions. Globally supported RBFs (i.e.multiquadrics, Gaussians, thin plate splines, etc.) were the ones used earlyon, and for problems whose size does not exceed 400-500 pieces of data thesemethods should still be the method of choice. For larger problems the ex-act solution of the associated (dense) linear systems becomes too expensive.Therefore, researchers interested in continuing to work with globally sup-ported RBFs have turned to the construction of fast (approximate) solvers

Transactions on Modelling and Simulation vol 22, © 1999 WIT Press, www.witpress.com, ISSN 1743-355X

292 Boundary Element Technology

and evaluation algorithms. Most work in this area is being done by thegroup in Cambridge, England (see e.g. Powell [11]), as well as by Rick Beat-son (see e.g. Beatson & Light [1]).

It is the class of locally supported functions, however, which we willfocus our attention on in this paper. Those compactly supported RBFsused most widely were introduced by Wendland[13]. Simple direct formulasfor computing these functions are (see Fasshauer[3])

)r% + (3f + 6)r + 3] , (1)

= (1 - r) [( + 9 4- 23( 4- 15)

+ 36^ + 45)7*2 + (15f + 45)r -f 15] .

Here = denotes equality up to a constant factor, and r — \\x\\, where x £ B/,so that the composition of the univariate function fa^ with the norm indeedyields a radial function. Note that the cut-off function (•)+ ensures thecompact support of these functions. Moreover, it should be pointed outthat the fa^ are in fact piecewise polynomials. The two indices i and kare related to the space dimension d, and to the smoothness of the basisfunctions. More precisely, if one intends to work in B/\ then one shouldtake t = [fj 4- k + 1, where k is the value of the second index stating that0£,fc G C^^ , since the functions are positive definite for this choice, and thusthe associated system matrix is assured to be invertible.

The solution of partial differential equations with radial basis functionsis a subject which is still in its infant stages, even though some first attemptswere reported roughly 10 years ago (see Kansa[10]). The examples we areabout to discuss are linear elliptic problems which we solve numerically bya symmetric collocation approach. To this end we assume that the PDE isof the form

where C — [Z/i, . . . , LAT] is a vector of linear operators, and the right-hand side T - [/i, . - - , /jvF- Together they define the (system of) partialdifferential equations along with boundary conditions. We also assume weare given a set of centers X — {xi, . . . ,#„} at which we attempt to satisfythe PDE pointwise (collocation). The expansion one uses for the solutionis of the form

N #*„

44 = E Z 4 (11% - ajlW, % e if* . (2)*/=! j=l

Here the X» are subsets of X with cardinality #^ corresponding to thecomponents of C. The superscript W indicates that L acts on 0 as a function


Boundary Element Technology 293

of the second variable, i.e., the center location. Expansion (2) is motivatedby a connection to scattered Hermite interpolation which ensures the invert-ibility of the related collocation matrix (for more details see Fasshauer[2,4]or Franke & Schaback[7]).

The remainder of the paper is organized as follows. In Section 2 weoutline our algorithmic approach which is a Newton-like multilevel methodwith interlaced smoothing operations, and then discuss our two examplesin the following two sections. The last section is devoted to summarizingremarks.

2. Multilevel Collocation with Smoothing

It is not recommended that one use compactly supported functions in thesame way as one uses globally supported functions, i.e., in a direct, single-grid approach. The advantage that compactly supported functions add interms of computational efficiency is offset by their modest approximationproperties (see Example 2 in Section 4). To obtain the best results withcompactly supported functions (for large data sets) it is recommended thata multilevel algorithm be employed (a more detailed discussion of this issuecan be found in Fasshauer[4] or in Schaback[12]).

The basic multilevel algorithm for the solution of the problem Cu = Ton fi C R^ by collocation can be described as follows:

Multilevel Collocation Algorithm.

1. Generate a nested sequence of computational grids X\ C • • • C XK C H,and let UQ = 0.

2. For k from 1 to K doa. Compute the residual r^ = T — Cuk-i on the grid Xi~.b. Solve Cuk = rk on X with u as in Eqn. (2).c. Update the level-k approximation as

By linearity and a simple telescoping argument it is clear that UK willsatisfy the differential equation at all points of the finest grid XK •

In Fasshauer & Jerome [5] it was shown how a multilevel algorithm ofthis kind can be interpreted as Newton iteration at the operator level. Thisimplies that step 2 above is viewed as

WA-WA-l ==-7%F(%6-l). (3)

Here the operator F computes the residual, i.e., F(u) — Cu - T, and theoperator TX (which encompasses the solution of the differential equationvia collocation in our examples) can be interpreted as an approximation tothe inverse of the (Freeh et) derivative of F.



It is well known (see e.g. Jerome[9]) that - unlike the classical Newtonmethod - the operator version does in general not converge quadratically,but suffers from a so-called loss of derivatives. The main result of Jerome[9],which was adapted for Besov spaces in Fasshauer & Jerome[5], is that theaddition of an appropriate smoothing operation 5& (defined via convolutionwith an appropriate convolution kernel -0) at each step of the Newton iter-ation almost restores quadratic convergence. In other words, the iterativescheme

uj, - Uk-i = -SkTx>F(uk-i) (4)

converges superlinearly. The precise statement of the convergence of the al-gorithm (in a variety of Besov spaces) is given in Theorem 3.1 of Fasshauer &Jerome[5]. The two most important hypotheses required for the superlinearconvergence result to hold are:

• The smoothing operation needs to be an approximate identity satisfyinga Bernstein and a Jackson inequality in a very general scale of Besovspaces (assumption (A2) in Fasshauer & Jerome[5]).

• The meshsize of each computational grid has to be chosen adaptivelyaccording to the size of the most recent residual ((3.4) in Fasshauer &Jerome[5]).

Thus, the addition of an appropriate smoothing operation in step 2.c ofthe multilevel collocation algorithm will generate a (superlinearly) conver-gent scheme, provided that the sequence of computational grids satisfies themeshsize constraint mentioned above. Another way to interpret the resultsof the smoothing operation is that the smoothing restores the convergenceproperties of the numerical inversion operator TX (see (3.2) and (3.6) ofFasshauer & Jerome[5]).

3. Smoothing with Spline Kernels

The main properties required in the proof of the Jackson inequality for thesmoothing operation in Fasshauer & Jerome[5] were:

. e Cg°(B/),

• ijj = 0 in a neighborhood #e(0) of the origin.

Here denotes the d-variate Fourier transform of . Smoothing kernelssatisfying all of these properties are not very easily found. One example,however, was listed already in Jerome[9]. Since that kernel is not easy toimplement, other possibilities (which satisfy only some of the theoretical re-quirements) were suggested in Fasshauer & Jerome[5]. One such possibilityare kernels whose Fourier transforms are compactly supported on ]R/*, andwhich also satisfies the condition $ = 1 in #e(0). However, the regularityof the Fourier transform is assumed to be limited. The simplest kernel ofthis sort is the sine function

sin(z)*(x) = —,



whose Fourier transform is the characteristic function of the interval [—1,1].For general d, the inverse Fourier transform of the characteristic function ofthe unit ball in H^ is given by

where J^ is the Bessel function of the first kind of order v.Reverting to the case d = 1, kernels whose Fourier transform is smoother

can be constructed using Hermite interpolation conditions at ±1 and ±e.Therefore the kernel whose Fourier transform is given by the continuousfunction

(I M < 1/2,^M= < 2(1-w) 1/2 < H <1,

I 0 otherwise,

is given by

%/,(%) = 2

or the C* Fourier transform

cos(|) - cos(x)

1 |w| < 1/2,9w^ + 6w-l) 1/2 < |w|

otherwise,

(cos(f ) - cos(aQ) - x (s'm(x) + sin(f ))

Of course it is also possible to construct kernels whose transform is smoother.

Example 1. Consider the two point boundary- value problem

-u"(x) + rPu(x) = 271-2 g a:, x G (0, 1),

16(0) = 1i(l) = 0, ^

with solution u(x) = sin Tnr. As computational grids X\~ we take 2 "*" + 1uniformly spaced points on [0, 1] as indicated in Table 1. Even though we aredealing with a one-dimensional problem we use the radial basis function 05 ,3as defined in Eqn. (1), which is six times continuously differ entiable. Notethat since the differential equation (6) is of second order, the RBF chosenneeds to be at least C* for use with the symmetric collocation approach(2). The linear systems arising at each level are solved with the conjugategradient method and Jacobi preconditioning. In order to have a meaningfulinitial approximation the support of the basis functions for the coarsest gridis chosen so large that the matrix is a dense one. At subsequent levels the



mesh

17336512925751310252049

w/o smoothing^2-error

5.408189 10-G1.16028010-71.548268 10-*1.118532 10-*1.09340210-s1.09267510-»1.093092 10-*1.09335110-8

rate

5.542.910.470.030.000.000.00

w/ smoothing(.% -error

5.681042 10-G5.590327 lO-*6.070907 10-95.538264 10~*9.82359910-91.08711110-81.26192810-81.35782210-8

rate

6.673.200.13-0.83-0.15-0.22-0.11

% nonzero

10078.046.625.313.26.713,391.70

Table 1: Errors and convergence rates with and withoutC* -spline smoothing for BVP (6).

support size for the new basis functions is taken to be half that of the one forthe previous level. We do this in order to keep the bandwidth of the systemmatrices constant (note that the mesh size is also halved in each iteration).The rightmost column of Table 1 shows the percentage of nonzero entriesin the system matrix at each level.

Columns 2 and 3 of Table 1 show how the multilevel collocation al-gorithm performs without the recommended smoothing. Clearly, it ceasesto converge after the first 4 steps. The smoothing which is used for theresults presented in columns 4 and 5 is based on the (7* spline kernel (5).The smoothing operation Sk in Eqn. (4) depends on the level &, in facti/jtk(%) — tfyfrkx)- In Fasshauer & Jerome[5] the smoothing speeds tk weredefined as tk = p^ , where the three parameters p, 0 and (3 can be chosenby the user (with some dependency on the smoothness of the problem). Forthe results of Table 1 we have used p = 10, 0 = 1.1, and 0 = 1.2. Thereader can see at least some benefits of the smoothing in this example sincethe errors (as well as the convergence rates) are better for the first few stepsof smoothing. Later on, however, the performance of the algorithm withsmoothing is even worse than without. This may be attributable to thelack of smoothness, or possibly to a stronger influence of the oscillations inthe smoothing kernel on these finer grids.

4. A Boundary-Value Problem in 2D

We now give an example of a Poisson equation in two dimensions. Thesolution by multilevel collocation with compactly supported RBFs is, toour knowledge, the first such attempt. Besides discussing the influence ofsmoothing on the performance of the algorithm, we will also compare themultilevel results with direct, single-grid results based on multiquadrics andcompactly supported RBFs.



Example 2. Consider the Poisson equation

((x - - 1200(x - i)'° + (y- |)< - 1200(7, - i)

inside the unit square with Dirichlet boundary conditions(7)

u(x,y) = = 0 or x = 1 or y = 0 or y = 1.

A plot of the solution is given on the left in Figure 1.For this example the computational grids X^ are given by (2*+* + 1) x

(2 +i + 1) uniformly spaced points in the unit square. Again, we use theRBF 05,3. The arrangement of the information in Table 2 is the same as inTable 1. However, the smoothing operation is now performed by convolvingwith the Gauss-Weierstrass kernel

(4?r) ' ^ '

The parameters determining the smoothing speeds are p — 1.2, 9 = 1.5,and 0 = 1.9. It should be pointed out that the Gauss-Weierstrass kerneldoes not satisfy all the theoretical requirements for the kernel either. Inparticular, as a positive kernel it is saturated (i.e., the required Jacksoninequality does not hold). However, the results below clearly show that itsuse improves the performance of the multilevel collocation algorithm. Plotsof the two different fits on the 33 x 33 grid are shown in Figure 1.

mesh

5x59x917 x 1733x3365x65

w/o smoothing2 -error

1.65688610°4.47631610-14.008961 10-i4.08899910-14.14984310-1

rate

1.890.16-0.03-0.02

w/ smoothing^2-error

2.584879 10~^3.72251810-21.87379710-21.26935410-2

1.25518810-2

rate

2.800.990.560.02

% nonzero

10072.629.59.352.63

Table 2: Errors and convergence rates for multilevel collocationwithout and with Gauss-Weierstrass smoothing for BVP (7).

Finally, we would like to offer the following interesting comparison withcollocation on a single grid. We do this with both the compactly supportedRBF 05,3 used above, and with globally supported multiquadrics, </>(r) ==

The values of c used for the multiquadrics were chosen on an



Figure 1: Plot of exact solution, and multilevel fits on 1089 pointswithout and with smoothing for BVP (7).

ad-hoc basis, aided by a small amount of trail and error, and can thereforeprobably be improved upon. The NA entries for the multiquadrics in Table 3indicate that the resulting dense linear systems (of size 1089 x 1089, and4425 x 4225) were too large to solve on a desktop PC. The support size (andtherefore the bandwidth of the matrices for the locally supported resultswere kept constant throughout resulting in the same percentage of nonzeroentries as for the multilevel method. Even though the second entry for thecompactly supported functions indicates a relatively good fit, the graph ofthe approximate solution in Figure 2 shows otherwise.

mesh

5x59x917 x 1733x3365x65

MQ2 -error

7.571929 10-*3.954529 KT*5.816741 lO-*

NANA

c0.20.150.1NANA

CS-RBF£2 -error

1.65688610"3.322141 lO-^1.98078310-12.492955 10"*2.66576210-

% nonzero10072.629.59.352.63

Table 3: Errors for single-grid collocation with multiquadricsand compactly supported RBFs for BVP (7).

The comparison with single-grid collocation shows that globally sup-ported functions should be preferred on small data sets, and that compactly



Figure 2: CS-RBF fit on 81 and 1089 points,and MQ fit on 289 points for BVP (7).

supported functions should probably be avoided altogether for direct com-putations on a single grid. Moreover, the solutions obtained with multi-quadrics were better than those obtained with the multilevel method (evenif up to 4225 points were used instead of 81). Of course, the support size ofthe compactly supported functions could be increased, but that would con-tradict the philosophy of wanting sparse matrices for efficient computation.

5. Concluding Remarks

In this paper we have described some numerical experiments with radialbasis functions using the multilevel collocation method for partial differen-tial equations. The application of a spline kernel smoothing delivers somelimited improvements over the unsmoothed algorithm, but the difficultiescaused by the oscillations of these kernels seem to outweigh their benefits.The Gauss-Weierstrass smoothing applied to a 2D problem, on the otherhand, performed in a rather stable way, and yielded clear improvements overthe unsmoothed method. The best results, however, were obtained using adirect approach with globally supported multiquadrics and a rather smallset of collocation points. The use of globally supported functions shouldtherefore not be ignored since they are also much easier to implement thana multilevel algorithm with added smoothing.

References

1. Beat son, R. K. and W. A. Light, Fast evaluation of radial basis func-tions: Methods for 2-dimensional polyharmonic splines, IMA J. Numer.Anal. 17 (1997), 343-372.

2. Fasshauer, G. E., Solving partial differential equations by collocationwith radial basis functions, in Surface Fitting and Multiresolution Meth-ods, A. Le Mehaute, C. Rabut, and L. L. Schumaker (eds), VanderbiltUniversity Press, Nashville TN, 1997, 131-138.



3. Fasshauer, G. E., On smoothing for multilevel approximation with ra-dial basis functions, in Approximation Theory IX, Charles K. Chui andLarry L. Schumaker (eds), Vanderbilt University Press,, Nashville, TN,1999, to appear.

4. Fasshauer, G. E., Solving differential equations with radial basis func-tions: multilevel methods and smoothing, Adv. in Comput. Math., toappear.

5. Fasshauer, G. E. and J. W. Jerome, Multistep approximation algo-rithms: Improved convergence rates through postconditioning withsmoothing kernels, Adv. in Comput. Math., to appear.

6. Floater, M. S. and A. Iske, Multistep scattered data interpolation usingcompactly supported radial basis functions, J. Comput. Applied Math.73 (1996), 65-78.

7. Franke, C. and R. Schaback, Solving partial differential equations bycollocation using radial basis functions, Appl. Math. Comp. 93 (1998),73-82.

8. Franke, C. and R. Schaback, Convergence orders of meshless collocationmethods using radial basis functions, Adv. in Comput. Math. 8 (1998),381-399.

9. Jerome, J. W., An adaptive Newton algorithm based on numerical in-version: regularization as postconditioner, Numer. Math. 47 (1985),123-138.

10. Kansa, E. J., Multiquadrics - A scattered data approximation schemewith applications to computational fluid-dynamics - II: Solutions toparabolic, hyperbolic and elliptic partial differential equations, Com-put. Math. Appl. 19 (1990), 147-161.

11. Powell, M. J. D., Recent research at Cambridge on radial basis func-tions, preprint, 1998.

12. Schaback, R., On the efficiency of interpolation by radial basis func-tions, in Surface Fitting and Multiresolution Methods, A. Le Mehaute,C. Rabut, and L. L. Schumaker (eds), Vanderbilt University Press,Nashville TN, 1997, 309-318.

13. Wendland, H., Piecewise polynomial, positive definite and compactlysupported radial functions of minimal degree, Adv. in Comput. Math.4 (1995), 389-396.

14. Wendland, H., Meshless Galerkin approximation using radial basis func-tions, Math. Comp., to appear.

15. Wendland, H., Numerical solution of variational problems by radialbasis functions, in Approximation Theory IX, Charles K. Chui andLarry L. Schumaker (eds), Vanderbilt University Press,, Nashville, TN,1999, to appear.


gregory e. fasshauer - wit presssince their introduction in the 1970s radial basis functions (rbfs)...

Documents