spectral methods for mantle convection with 3d viscosity ... · spectral methods for mantle...

Post on 23-Jul-2020

9 Views

Category:

Documents

0 Downloads

Preview:

Click to see full reader

TRANSCRIPT

Spectral Methods for

Mantle Convection with 3D

Viscosity Variations

Robert Moucha

Alessandro Forte

Workshop for Advancing Numerical Modeling of Mantle

Convection & Lithospheric Dynamics

July 10, 2008

Outline

Introduce the variational functional for Stoke’s flow

Spectral formulation of the variational functional

Solving the system – practical considerations

Benchmarks

Introduction

A complete treatment of the mathematical details may be found in Forte & Peltier (1994),

Advances in Geophysics 36.

For the geodynamical impact of lateral viscosity variations see Moucha et al. (2007) , GJI.

Introduction (cont.)

Incompressible fluid Anelastic-liquid Newtonian rheology Infinite Prandtl number approximation

Boussinesq-like fluid approximation

Assumptions utilized in the buoyancy induced flow problem (Stoke's flow) in the mantle.

2410Pr ≈=ρκ

η

Introduction (cont.)

Cartesian representation of non-hydrostatic dynamical equations:

0=∂ kkuConservation of Mass :

ijijki EP ηδσ 21 +−=Stress tensor:

011 =∂+∂+∂ oiiokik φρφρσConservation of Momentum :

( )ijjiij uuE ∂+∂=

2

1Strain-rate tensor:

Introduction (cont.)

The boundary conditions imposed on the flow field at the outer and inner surfaces are:

0ˆ =ii un

0ˆˆ =jijinh σ

Zero Radial Velocity:

Free Slip:

Other boundary conditions imposed on the flow field at the outersurface can also be imposed, such as rigid boundary conditions.

Introduction (cont.)

Different methods exist for obtaining a solution to the coupled system of equations, such as finite-difference/volume (STAG3D), finite-element (TERRA ; CitComS), or spectral methods.

There are numerous spectral methods and pseudo (or hybrid) spectral methods (e.g. Zhang & Christenson, 1993; Čadek et al. 1993; Martinec et al., 1993; Forte & Peltier, 1994).

Advantages of spectral methods:

•Meshless – circumvents challenges with choosing grid types anddomain decomposition

•Quasi-analytical solution

Disadvantages:

•Computationally and storage-wise expensive for 3D viscosity

•Non-Newtonian rheology requires an iterative approach (Čadek et al. 1993

The Variational Formulation

1. Assume that ui satisfies the governing equations and the BCs. Now

consider a flow perturbation δui such that:

0=∂ kk uδ

2. Take the inner product of the flow perturbation δui with the momentum conservation equation:

( ) ( ) ( ) ( ) 0011 =∂+∂+∂−∂ iiioiikkiikik uuuu δφρδφρδσδσ

[ ] [ ] 0ˆˆ 1001 =−+−∂ ∫∫ Skkkkik

Vkikiii dSunundVEu δφρδσδσδφρ

kikiikki Eu δσδσ =∂

Integrating over the volume V occupied by the medium by virtue of Gauss'

theorem, and by symmetry of the stress tensor :

We get:

The Variational Formulation (cont.)

4. Since both the solution ui and the perturbed flow ui + δui satisfy the same BCs on the surface S we therefore have:

[ ] [ ] 0ˆˆ 1001 =−+−∂ ∫∫ Skkkkik

Vkikiii dSunundVEu δφρδσδσδφρ

which implies that δui must be tangential to S; thus

0ˆ =kk un δ

0ˆ =ikik un δσ

The Variational Formulation (cont.)

[ ] 0201 =−∂∫V kikiii dVEEu δηδφρ

5. Because of the incompressibility assumption

011 == kkkiki EPEP δδδ

The Variational Formulation (cont.)

[ ] 0201 =−∂∫V kikiii dVEEu δηδφρ

[ ]∫ ∂−==V

iiijij dVuEEWW 01;0 φρηδ

Rate of energy released by buoyancy

Rate of viscous dissipation energy

Assuming that δη = 0, i.e. η does not depend on the flow velocity ui

(otherwise see Čadek et al., 1993)

The Variational Calculation using spherical harmonics

qp ∇×+∇××∇=

=•∇vvv

v

rru

u 0since

( ) ( ) ( )∑=m

mmYrprp

,

,,,l

llφθφθ

( ) ( ) ( )∑=m

mmYrqrq

,

,,,l

llφθφθ

Expand the flow field u in terms of poloidal (p) and toroidal (q) flow generating scalars as follows:

Represent the poloidal and toroidal flow fields in terms of spherical harmonic basis functions:

We allow for explicit 3-D viscosity variations by also expanding the viscosity:

( ) ( ) ( )∑=m

mmYrr

,

,,,l

llφθηφθη

The Variational Calculation using spherical harmonics (cont.)

[ ]∫ ∂−=V

iiijij dVuEEW 01 φρη

By expressing the tensor inner product EijEij in terms of the so-called contravariant canonical components using generalized spherical harmonics

described in Phinney & Burridge (1973), the viscous dissipation energy integral becomes:

The Variational Calculation using spherical harmonics (cont.)

( )( )( )

( ) ( ) ( )

( )

( )drr

dr

dq

dr

dir

r

p

dr

pd

dr

dq

dr

dir

r

p

dr

pdJs

iqr

p

dr

dpiq

r

p

dr

dp

r

Js

r

p

dr

dp

r

p

dr

dp

r

Jsr

tmtm

JsJsdVEE

t

s

t

s

st

s

mmms

t

s

t

s

t

smmmss

a

b

t

s

t

s

mmstm

J

s

sJm tsV

ijij

2

2

2

2

2

2

2

2

2

2

2

11

2

2121

2

2

2

2

1

, ,

2

2

011

2

022

6

000

1212124

+

Ω+×

Ω+ΩΩ

−−

++

−+

ΩΩΩΩ

−+

ΩΩ

×

−−+++=

∑∑∑∫

−−

+

−=

ll

l

ll

l

ll

ll

ll

l

l

ll

l

l

l

ll

η

πη

[ ]∫ ∂−=V

iiijij dVuEEW 01 φρηWigner 3-j symbols

( ) ( )( )2

21,

2

121

+−=Ω

+=Ω

llll ll

Where;

The Variational Calculation using spherical harmonics (cont.)

( ) ( ) ( ) ( ) ( )

( )∗

+

−=

×+++= ∑

φθ

φθφθ

,

121212,,

21

21

21

21

21

21

21

22

2

11

1

Nm

mNmN

YmmmNNN

YY

l

ll

lll

ll

llllll

lll

A summary of useful aspects of spherical harmonic coupling rules, in the context of fluid dynamics in spheres, can be found in Forte & Peltier (1994, Appendix II).

The Variational Calculation using spherical harmonics (cont.)

( )( )( )

( ) ( ) ( )

( )

( )drr

dr

dq

dr

dir

r

p

dr

pd

dr

dq

dr

dir

r

p

dr

pdJs

iqr

p

dr

dpiq

r

p

dr

dp

r

Js

r

p

dr

dp

r

p

dr

dp

r

Jsr

tmtm

JsJsdVEE

t

s

t

s

st

s

mmms

t

s

t

s

t

smmmss

a

b

t

s

t

s

mmstm

J

s

sJm tsV

ijij

2

2

2

2

2

2

2

2

2

2

2

11

2

2121

2

2

2

2

1

, ,

2

2

011

2

022

6

000

1212124

+

Ω+×

Ω+ΩΩ

−−

++

−+

ΩΩΩΩ

−+

ΩΩ

×

−−+++=

∑∑∑∫

−−

+

−=

ll

l

ll

l

ll

ll

ll

l

l

ll

l

l

l

ll

η

πη

[ ]∫ ∂−=V

iiijij dVuEEW 01 φρη

The Variational Calculation using spherical harmonics

( ) ( )

bar

m

bar

m

dr

rpdrp

,

2

2

,0

==

== l

l

One possible set of radial basis functions that satisfy these boundary conditions are the following modified Fourier basis:

One can chose any set of radial basis functions, but they must satisfy the following poloidal and toroidal (free-slip) boundary conditions:

( ) ( )

( )

−=

=∑

ba

arkrf

rfprp

k

k

k

m

k

m

πsin

ll( ) ( )

( )

−=

=∑

ba

arkrrg

rgqrq

k

k

k

m

k

m

πcos

ll

( )0

,

=

= bar

m

r

rq

dr

dl

The Variational Calculation using spherical harmonics (cont.)

The functional W will be at a minimum when the following conditions are satisfied:

( ) ( )[ ] 001 =∂−

∂=

∂∫V iiijijt

sn

t

sn

dVuEEpp

Wφρη

( ) ( )[ ] 0=

∂=

∂∫V ijijt

sn

t

sn

dVEEqq

The Variational Calculation using spherical harmonics (cont.)

( ) ( )( )∫∑∑

∗+

=+a

bn

t

s

mk

m

k

mk

nst

mk

m

k

mk

nst drrgrfr

rssqBpA

2

0

1

0,,,,

)1( ρ

ηl

l

l

l

l

l

0,,,,

=− ∑∑mk

m

k

mk

nst

mk

m

k

mk

nst pDqCl

l

l

l

l

l

Minimizing the functional W will yield the following couples set of algebraic equations:

The Variational Calculation using spherical harmonics (cont.)

The coefficients involve a numerical radial integration of the viscosity and the flow basis functions, for example:

( )( )( )

( ) ( ) ( )

( )

( )drr

r

f

dr

fd

r

f

dr

fdJs

r

f

dr

df

r

f

dr

df

r

Js

r

f

dr

df

r

f

dr

df

r

Jsr

tmtm

JsJsA

n

s

n

kks

nnkk

ss

a

b

nnkk

stm

J

s

sJ

mk

nst

2

2

2

2

2

2

2

2

2

2

2

11

2

2121

2

2

2

2

1

0

2

22

011

4

022

12

000

121212

Ω+×

Ω+ΩΩ

−−

+

+

ΩΩΩΩ

−+

ΩΩ

×

−−+++=

−−

+

−=

l

l

ll

l

l

l

l

l

l

l

ll

η

η

The Variational Calculation using spherical harmonics (cont.)

( ) ( )( )∫∑∑

∗+

=+a

bn

t

s

mk

m

k

mk

nst

mk

m

k

mk

nst drrgrfr

rssqBpA

2

0

1

0,,,,

)1( ρ

ηl

l

l

l

l

l

0,,,,

=− ∑∑mk

m

k

mk

nst

mk

m

k

mk

nst pDqCl

l

l

l

l

l

=

− 0

d

q

p

CD

BA

bSx =

The Variational Calculation using spherical harmonics (cont.)

( ) ( )( )∫∑∑

∗+

=+a

bn

t

s

mk

m

k

mk

nst

mk

m

k

mk

nst drrgrfr

rssqBpA

2

0

1

0,,,,

)1( ρ

ηl

l

l

l

l

l

0,,,,

=− ∑∑mk

m

k

mk

nst

mk

m

k

mk

nst pDqCl

l

l

l

l

l

=

− 0

d

q

p

CD

BA

bSx =

= 0 for 1D viscosity

S is for super matrix

11 ××× = nnnn bxS

( ) flowflowr ssnn 22 +=

nr = number of radial basis functions

sflow = spherical harmonic degree of flow scalars

where

Currently we use:

nr = 40

sflow = 32

n = 87040 !!

Equivalent Spatial Resolution:

36 km radially

600 km at the surface

56 GB of storage (double precision)

S is for super matrix (cont.)

Very dense matrix

Badly conditioned!

But, consistent!

1510−≈− bSy

row

s

columns100 200 300 400 500

100

200

300

400

5000

1

2

3

4

5

6

7

8

log10|S|

For example;nr=40, sflow = 6

1/cond(S) = 10-7

Residual;

S is for super matrix (cont.)

Very dense matrix

Badly conditioned!

But, consistent!

2110−≈− bSy

log10|diag(1/R)*S|

For example;nr=40, sflow = 6

1/cond(S) = 10-5

Residual;

row

s

columns100 200 300 400 500

100

200

300

400

500-8

-7

-6

-5

-4

-3

-2

-1

0

Row Scaling => maximum in a row

Getting a solution

Use a direct method for solving the system of equations

Use a cluster (130 Opterons (1.6GHz) with 3 GB of RAM/CPU)

Use ScaLapack, which is a linear algebra library for parallel computers that implements block-oriented LAPACK routines

1. Distribute matrix and RHS on a processor grid using 2D block-cyclic distribution

2. Solve the system of equations using LU factorization (PDGETRF) and solver (PDGETRS)

row

s

columns100 200 300 400 500

100

200

300

400

5000

1

2

3

4

5

6

7

8

Getting a solution (cont.)

2D block cyclic distribution in ScaLapack on a 2x2 processor grid

log10|S|

0000 0000 0000

0000 0000 0000

0000 0000 0000

1111 1111 1111

1111 1111 1111

1111 1111 1111

2222 3333 2222 3333 2222 3333

2222 3333 2222 3333 2222 3333

2222 3333 2222 3333 2222 3333

0000 1111

2222 3333

1 2 3 4 5 6 7 8 9

x 104n

S gen.

Solve

Total

Getting a solution (cont.)

Obtaining a solution is expensive!

100

101

102

103

101

102

103

104

Number of Processors

Ela

ps

ed

Tim

e (

s)

S gen.

Solve

Total

Perfect Efficiency

nr=40, sflow = 12, sviscosity = 12n = 13440

nr=40, sflow = 12-32, sviscosity = 12# of processors = 121

Benchmarking

Variational calculations for

n = 87040

nr = 40, sflow = 32, sviscosity=32, sdensity = 12

Compare with a finite-element Stokes’ flow solution in spherical geometry using CitcomS v1.1 (Zhong et al., 2000).

12 spherical caps, each with 65x65x65 nodes.

The buoyancy forces in the mantle are derived from a seismic tomography model using velocity-to-density conversion profile.

Benchmarking (cont.)

The spatial variations in viscosity are expressed as:

where;

and ν(r,θ,φ) are estimated on the basis of homologous-temperature scaling using seismic-tomography derived temperature anomalies for the mantle.

( ) ( ) ( )[ ], , 1 , ,r r rη θ φ η ν θ φ= +

( )10

0

ar

rη η

=

η0 = 1.0x1021 Pa s

Benchmarking (cont.)

Viscosity at 500 km

η0 = 1.0x1021 Pa s

Viscosity at 2740 km

Temperature at 500 km

Temperature at 2740 km

∆T = 1500 K, T0 = 0.5

Benchmarking (cont.)

Spatial Finite-element

Spectral VariationalSpectral Variational

Difference (Spectral – CitComS)

8 (m)

-8

80 (m)

-80

Currently in development Efficient Wigner 3-j symbol storage and look-up

Use B-spline radial basis functions

B-spline radial basis offer local support

Local support reduce the Super matrix into a block-tridiagonal form in the case of a cubic B-spline

Block-tridiagonal form offers a huge speed up by reducing the number of blocks in the Super matrix that need to be calculated

Massive reduction in storage requirements for the Super matrix

A direct block-triadiagonal solver reduces the number of FLOP by over 2 orders of magnitude vs. traditional direct dense matrix solvers (impact on number of communications should be also less)

Degree 64 (~310 km) gives a matrix size of n = 337920

Dense super matrix = 850GB vs. B-spline Block-Tridiagonal compact Matrix = 190 GB

Impact of 3D Viscosity

Impact of 3D Viscosity (cont.)

Impact of 3D Viscosity (cont.)

Impact of 3D Viscosity (cont.)

top related