speeding up numerical computations via conformal...

Speeding up numerical computationsvia conformal maps

Nick Trefethen, Oxford University

Thanks to Nick Hale,Nick Higham and Wynn Tee

1/35

SIAM 1997 SIAM 2000 Cambridge 2003

2/35

Princeton 2005 Bornemann et al., SIAM 2004

Suppose f is analytic, bounded, and 2-periodicin the strip Sa = {z: -a < Im z < a} .

Sample f in equally spaced points

x

a

PERIODIC STRIPS, INFINITE STRIPS, AND ELLIPSES

3/35

Sample f in equally spaced points

Error in trigonometric interpolation: O(ea/x)

Error in trapezoid rule quadrature: O(e2a/x)

(Poisson 1826, Davis 1959)

If f is nonperiodic on the whole real line (but integrable):

Same results under mild assumptions (sinc interpolation)(Turing 1943, Goodwin 1949, Milne 1953, Martensen 1968, Stenger 1970s)

Now suppose f is analytic and bounded in the ellipse Eρwith foci ±1, ρ = semimajor + semiminor axis lengths > 1.

cosh(a)

sinh(a)

11

ρ = exp(a)

4/35

cosh(a)

Error in polynomial interpolation inChebyshev or Gauss-Legendre points: O(n)

Error in Gauss quadrature: O(2n)

(Bernstein 1919)

1. New formulas for quadrature on [−1,1]

2. Evaluating f(A), A = matrix or operator

3. Tee’s adaptive spectral method

PLAN OF THE TALK:

WE’LL APPLY THESE RESULTS TO THREE PROBLEMS,EACH INVOLVING A CONFORMAL CHANGE OF VARIABLES

5/35

4. Double Exponential quadrature

5. Analytic continuation

6. Inverse Laplace transforms

RELATED TOPICS WE WON’T HAVE TIME FOR:

1. New formulas for quadrature on [−1,1]

JOINT WORK WITH NICK HALE, OXFORD U.

6/35

SIAM J. Numer. Anal., to appear

Analyticity in an ellipse is a strange condition.

- It entails more smoothness in the middle than near the ends.

- A Gauss or Chebyshev grid is /2 times coarser in the middlethan an equispaced grid.

1 1

7/35

than an equispaced grid.

- pts per wavelength are needed in total to resolve a sine wave.

“Gauss quad. is /2 times less efficient than the trapezoid rulefor periodic integrands.”

“Chebyshev spectral methods need /2 times as many grid pointsas Fourier spectral methods — or in 3D, (/2)34 times as many.”

1 1

Q: Where do ellipses come from?

A: From using polynomials to derive the quadrature formula.

8/35

Q: Why do we have to use polynomials?

A: We don’t!

Our solution: conformally map the ρ-ellipse to a region withstraighter sides. For example, map it to an infinite strip:

1 11 1

gsx

Gauss quadrature here… …gives us a non-polynomialtransplanted quadrature rule here

strip is π/2 times narrower than ellipse

9/35

Transplanted integral: f(x) dx = f(g(s)) g’(s) ds1

1

1

1

THM: If f is analytic in the strip, the transplantedGauss formula has error O( ρ 2n ) for any ρ < ρ .

transplanted quadrature rule here

~ ~

Conformal map from ellipse to infinite strip

sin−1 tanh−1

10/35

sin−1

sn

tanh

GAUSS vs. TRANSPLANTED GAUSS quadrature points

(for a typical choice of parameter ρ )

N=16

1 1

11/35

N=32

N=64

1 1

1 1

Convergence for f(x) = 1/(cosh(1)cos(16x))

(analytic in the strip of half-width a = 1/16)

error

12/35

Gauss quadrature

TransplantedGauss quadrature

n

error

Nine more examples (strip map with ρ=1.4)

Gauss

transplantedGauss

13/35

Standard theorems for Gauss quadrature New theorems for transplanted Gauss quadrature.

E.G.: Suppose f is analytic and bounded in the ε-nbhdof [−1,1] for any ε < 0.05, and we use the ρ=1.1 strip map.

THEOREMS

14/35

THM: Gauss quadrature: error O( (1+ε)−2n )

Transplanted Gauss: error O( (1+ε)−3n )

Gauss

A wilder example

integrand quadrature error

15/35

transplantedGauss

RELATED WORK

“Gregory formulas”: trapezoid rule with endpoint corrections

Bakhvalov 1967: theoretical results on conformal maps & quadrature

Kosloff & Tal-Ezer 1993: arcsine change of vars. for spectral methods

Beylkin, Boyd, Rokhlin & others: prolate spheroidal wave functions

16/35

Alpert 1999: hybrid trapezoid/Gauss quadrature formulas

The last three seem roughly as effective as our method in practice.But they come with no thms about geometric convergence for analytic f.

2. Evaluating f(A), A = matrix or operator

JOINT WORK WITH NICK HALE AGAIN AND ALSO NICK HIGHAM, U. OF MANCHESTER

17/35

SIAM J. Numer. Anal., submitted

Aim: compute f(A) , A = operator or large matrix(e.g. of dimension 106)

or f(A)b for various vectors b

Examples: A , A , log(A) , exp(A) , . . .

18/35

Examples: A , A , log(A) , exp(A) , . . .

Applications: anomalous diffusion, finance, semigroups, . . .

Higham has written a book about f(A) problems.

where C encloses a and liesin the region of analyticity of f .

For a matrix or operator A ,

For a scalar a ,

Cauchy integrals

19/35

where C encloses spec(A ).

For a matrix or operator A ,

If C is a circular contour,equally spaced pointsshould be perfect —periodic trapezoid rule!

ASSUMPTIONS

f is analytic in the complex plane except (-, 0].

A has spectrum in [m,M] , M» m > 0 .

E.G.:

A

A

log(A)

tanh(A )

(A)...

20/35

0 m M

singularities of f spectrum of A

A BAD IDEA

Take the contour C to be a circle surrounding the spectrum.

For this you’ll need a very large numberof sample points: » M/m .

Reason: annulus of analyticity is narrow.Insteadwe wantto mapa much

21/35

0

singularities of f spectrum of A

m M

a muchthickerannulusonto theWHOLELIGHTGRAYREGION.

MAP FROM THE ANNULUS(equivalently could use periodic strip)

g

22/35

f(z) (z-A)-1 dz = f(g(s)) (g(s)-A)-1 g’(s) ds

As always we use a change of variables:

CONFORMAL MAP FROM ANNULUS(plots show the upper half)

log

23/35

sn (Jacobi elliptic function again)

Möbius

% method1.m - evaluate f(A) by contour integral. The functions% ellipkkp and ellipjc are from Driscoll's SC Toolbox.

f = @sqrt; % change this for another function fA = pascal(6); % change this for another matrix AX = sqrtm(A); % change this if f is not sqrtI = eye(size(A));e = eig(A); m = min(e); M = max(e);k = (sqrt(M/m)-1)/(sqrt(M/m)+1);L = -log(k)/pi;[K,Kp] = ellipkkp(L);for N = 5:5:50

MATLAB TEST CODE FOR MAP 1 , f =

>> method1

RESULTS

24/35

for N = 5:5:50t = .5i*Kp - K + (.5:N)*2*K/N;[u,cn,dn] = ellipjc(t,L);z = sqrt(m*M)*((1/k+u)./(1/k-u));snp = cn.*dn./(1/k-u).^2;S = zeros(size(A));for j = 1:NS = S + f(z(j))*inv(z(j)*I-A)*snp(j);

endS = -4*K*sqrt(m*M)*imag(S)/(k*pi*N);error = norm(S-X)/norm(X);fprintf('%4d %16.12f\n', N, error)

end

>> method15 5.983430140320

10 0.37194156608715 0.01748713246020 0.00074193428025 0.00002971644430 0.00000114669035 0.00000004310840 0.00000000159045 0.00000000005850 0.000000000002

A more practical example

A = negative of 5050 discrete Laplacian (sparse, dimension 2500)

b = random vector of same dimension

Compute A1/2 b :

Contour integral & conformal map: 0.76 secs. on this laptop

25/35

Matlab “sqrtm”: 4 min. 48 secs.

Comments about conformal mapping methods for f(A)

• Further improvements get a further factor of 2 speedup

• We have reduced f(A)b to a dozen or two “backslashes”

• Competitor for small A: Schur reduction, Padé approx.

• Competitor for large A: Krylov subspace compressions

26/35

• Competitor for large A: Krylov subspace compressions

• This technique is very general, applicable to many f and A

• Deeper understanding: link with rational approximation

3. Tee’s adaptive spectral method

JOINT WORK WITH WYNN TEE, OXFORD DPhil 2007

27/35

SIAM J. Sci. Comp., 2006

This final topic is the most complex.

I told Wynn it would never work. But it did!

The aim: adaptive spectral method for PDEs —for problems with spikes, fronts, rapid variation…

RELATED WORKBayliss, Matkowsky and others `87,`89,`90,`92,`95Guillard and Peyret `88Augenbaum `89

28/35

Augenbaum `89Kosloff and Tal-Ezer `93Mulholland, Huang, Sloan, Qiu `97,`98Weideman `99Berrut, Baltesnsperger, Mittelmann `00,`01,`02,`04,`05

Good ideas here. But no method that can handle extreme cases.

Why not? None of them thought in terms of conformal maps.

Tee’s new method combines:

1. Padé/Chebyshev-Padé location of complex singularities

2. Conformal mapping onto domains with slits

3. Spectral differentiation by rational barycentric formulas

29/35

At each time step we construct conformal map

from ellipse… …to plane minus slits endingat estimated singularities

30/35

Examples of adaptively constructed irregular grids

For these computations we achieve 10-digit accuracy with gridsof <100 points (spectral in x, 9th or 13th order in t)

31/35

32/35

33/35

demonstrations to 10-digit accuracywith <100 grid points in x

burgersallencahnblowup

34/35

blowup

1. New formulas for quadrature on [-1,1]

2. Evaluating f(A), A = matrix or operator

3. Tee’s adaptive spectral method

RECAP OF OUR PROBLEMS

35/35

MORAL OF THE STORY

It’s not enough for a grid to “look good”.

It must correspond to a transplantationwith a wide region of analyticity. And if it

does, you get exponential convergence.

Speeding up numerical computationsvia conformal maps

Nick Trefethen, Oxford University

Thanks to Nick Hale,Nick Higham and Wynn Tee

36/35