LBNLLBNLGXTGXTBRBR FY2001 Oil and Gas Recovery Technology Review Meeting

Diagnostic and Imaging

High Speed 3D Hybrid Elastic Seismic Modeling

Lawrence Berkeley National Laboratory, GX Technology, Burlington Resources

Contacts: Valeri Korneev 510-486-7214, VAKorneev@lbl.gov;

Mike Hoversten 510-486-5085, GMHoversten@lbl.gov

LBNLLBNLGXTGXTBRBR Why do we need 3D elastic modeling

• Heterogeneous 3D media, complex topography, 3-component data, strong converted S- waves

• Survey design

• Hypothesis testing

• AVO evaluation

• Wave field interpretation

• Synthetic data sets for depth migration and full waveform inversion testing

• “Engine” for inversion

• It is cheaper and faster than real data acquisition

LBNLLBNLGXTGXTBRBR

Society of Exploration Geophysicists and European Association of Exploration Geophysicists 3D modeling

project for seismic imaging testing.

8000 Gflop hours. Acoustic.Started in 1994. Still not completed.

This project needs larger model and elastic code.

LBNLLBNLGXTGXTBRBR

0 2000 4000 6000 8000 10000 12000 14000 16000 18000

X [m ]

0

2000

4000

6000

Z

[m]

0

0

0

0

0

1

1

0 2000 4000 6000 8000 10000 12000 14000 16000 18000

x [m ]

0

2000

4000

6000

8000

Z [

m]

1 . 0

2 . 0

3 . 0

4 . 0

5 . 0

6 . 0

Source 51 receivers with 200 m spacing

0 2000 4000 6000 8000 10000 12000 14000 16000 18000

x [m ]

0

2000

4000

6000

8000

Z [

m]

Source 51 receivers with 200 m spacing

Comparizon of acoustic and elastic m odelingfor Mahogany salt body - Gulf of Mexico

Acoustic Elastic

Velocity m odel forMahogany salt body

Vp- velocity [km/s]

0 1 2 3 4 5Tim e [s]

0

5

10

Re

lativ

e o

ffse

t [k

m]

0 1 2 3 4 5T im e [s]

0

5

10

Re

lativ

e o

ffse

t [k

m]

LBNLLBNLGXTGXTBRBR

Deep water Gulf of Mexico Regional Seismic Line

Sub-salt structures can not be seen using acoustic inversion. Elastic propagator is needed to image details.

LBNLLBNLGXTGXTBRBR

Where 3D modeling stands today

• Industry primarily uses acoustic uniform grid

• Need of “smart” users who are experts in the method

• Massive parallel computing is expensive and “slow”

• Model building is a problem

• Modeling results are difficult to interpret

LBNLLBNLGXTGXTBRBR

Requirements

• Full elastic modeling

• Attenuation

• Anisotropy

• Topography

• Effectively exploit computational resources

• High fidelity numerical modeling

• Hybrid methodology: Ray-tracing coupled with finite difference

• Local resolution algorithms

• Massively parallel super computers and clusters

LBNLLBNLGXTGXTBRBR What is our goal?

• To build a 3D elastic modeling software tool capable to compute

• realistic (10 km * 10 km *4 km) models

•

• at seismic exploration frequencies (up to 100 Hz)

•

• on local networks

• at reasonable time (overnight)

• by any geophysical software user (with no special method-oriented training).

LBNLLBNLGXTGXTBRBR

0 1000 2000 3000 4000 5000 6000 7000 8000 9000

X [m ]

0

1000

2000

3000

4000

5000

6000

7000

Z [

m]

0

0

0

0

0

1

1

W ater

Shallow sediments

SALT

Subdomain 1Acoustic code

Subdomain 2Acoustic - Elastic contact code

Subdomain 3Elastic code

1 . 0

2 . 0

3 . 0

4 . 0

5 . 0

6 . 0

Vp velocity [km/s]

Main subdomain types for optimal hybrid seismic wave propagation modeling

LBNLLBNLGXTGXTBRBR

0 50 100 150 200 250 300 350 400 450 500 550 600

x [m ]

0

50

100

150

200

250

300

350

400

450

500

Z [

m]

0 50 100 150 200 250 300 350 400 450 500 550 600

x [m ]

0

50

100

150

200

250

300

350

400

450

500

z [m

]

Vp = 1400 m/s

Vp =3400 m /s

Vp = 1400 m/s

Vp =3400 m /s

SourceSource

Horizontal interface Dipping interface

Finite difference elastic simulation snapshots for two halfspaces m odel

- regular grid nodes

- LBC interface grid nodes

- desired interface specification

- effective interface specification for a regular gridV 1

V 2

n

Local Boundary Conditions for high contrast interfaces

LBNLLBNLGXTGXTBRBR

0 2000 4000 6000 8000 10000 12000

Horiz ontal d istance [ft]

0

2000

4000

Dep

th [

ft]Y u cca M ou n ta in v e lo c ity m o d e l E x p 85

R id ge

T im e = s0 .54

T u n n el

0 .0 0 .1 0 .2 0 .3 0 .4 0 .5 0 .6 0 .7 0 .8 0 .9 1 .0 1 .1 1 .2 1 .3 1 .4 1 .5 1 .6 1 .7 1 .8 1 .9 2 .0

T im e [s]

0

1000

2000

3000

4000

X - (h o r izo n ta l) co m p o n en t

0 .0 0 .1 0 .2 0 .3 0 .4 0 .5 0 .6 0 .7 0 .8 0 .9 1 .0 1 .1 1 .2 1 .3 1 .4 1 .5 1 .6 1 .7 1 .8 1 .9 2 .0

T im e [s]

0

1000

2000

3000

4000

Offs

et [f

t]

Z - (v ertica l) co m p o n en t

0 .0 0 .1 0 .2 0 .3 0 .4 0 .5 0 .6 0 .7 0 .8 0 .9 1 .0 1 .1 1 .2 1 .3 1 .4 1 .5 1 .6 1 .7 1 .8 1 .9 2 .0

T im e [s]

0

1000

2000

3000

4000

Offs

et [f

t]

C o m p ressio n a l co m p o n en t

0 .0 0 .1 0 .2 0 .3 0 .4 0 .5 0 .6 0 .7 0 .8 0 .9 1 .0 1 .1 1 .2 1 .3 1 .4 1 .5 1 .6 1 .7 1 .8 1 .9 2 .0

T im e [s]

0

1000

2000

3000

4000

Offs

et [f

t]

S h ea r co m p o n en t

2 D -> 3D g eom etry co n versio n

V p

LBNLLBNLGXTGXTBRBR Issues of 3D modeling performance

• Accuracy. Improves by using higher order differencing and finer gridding.

• Model size. No less then 5 grid points per shortest wavelength.

• • Acoustic code requires 3.5 *N cells, where N = Nx*Ny*Nz number of cells for model

parameters sampling. Elastic code needs 6 times more cells.

• CPU time. Acoustic code requires 5*K operations per grid cell. Elastic code requires 5 times more operations per grid cell, where K= 6*m, m - an order of differential operator.

• Stability. Requires integration time step

• Optimization. Avoid over sampling and too small time integration steps. Use parallel computing. Avoid computing in undisturbed cells.

• Numerical artifacts. Contrast contacts. Step sampling noise of dipping interfaces. Boundary reflections. Liquid-elastic interfaces.

5fV

max

minx

cVc ,

max

mm

xt 0.5

LBNLLBNLGXTGXTBRBR 3D hybrid elastic seismic modeling

• Parallel based upon overlapping subdomain decomposition variable order 3D finite difference code.

• Grid spacing depends on model parameters to provide local optimal computational regime.

• Wave propagation in the water will be computed by acoustic code.

• Contrast dipping interfaces will be handled with Local Boundary Conditions approach.

• Computation is performed for subdomains with non-zero wave field only.

• Option of conditional computation restarting at any given lapse time.

LBNLLBNLGXTGXTBRBR E q u a t i o n s o f m o t i o n f o r e l a s t i c a n i s o t r o p i c a t t e n u a t i v e m e d i a

e l a s t i c f o r c e s + s o u r c e = d e n s i t y * a c c e l e r a t i o n

ti k k

f id u id t,

2

2 , i 1 2 3, ,

t dd t

d i v dd t

u l u1 123 2 1 1 3 3

( ) , ,u , t t d

d tu u1 2 2 1 1 2 2 1

( , , )

t dd t

d i v dd t

u l u2 223 2 2 2 3 3

( ) , ,u , t t m d

d tu u2 3 3 2 2 3 3 2

( , , )

t l dd t

d i v l p dd t

u3 323 2 2 3 3

( ) ,u , t t m d

d tu u1 3 3 1 1 3 3 1

( , , )

G e n e r a l c a s e – a n i s o t r o p y , 3 v e l o c i t i e s , v e c t o r f i e l d - n o a t t e n u a t i o n ,

- i s o t r o p y , 2 v e l o c i t i e s , v e c t o r f i e l d - a c o u s t i c c a s e , 1 v e l o c i t y , s c a l a r f i e l d

M o s t o f c o n t e m p o r a r y s i m u l a t i o n s a r e a c o u s t i c .

W e n e e d m o r e r e a l i s t i c m o d e l i n g t o s o l v e p r o b l e m s .

0l m p 00 ,0

LBNLLBNLGXTGXTBRBR FY2000 Results

• Stair step gridding problem resolvedNonuniform grid algorithm tested

• New stable topography algorithm tested

• Parallel interface library applied to 2D

• 4-th order in time scheme tested

• GXT ray tracer installed

LBNLLBNLGXTGXTBRBR

• BoxLib Foundation Library– Domain specific library: support PDE solvers on structured,

hierarchical adaptive meshes (AMR)

– Support for heterogeneous workloads

– BoxLib based programs run on serial, distributed memory and shared memory supercomputers and on SMP clusters

– Parallel implementation

• MPI standard based, ensuring portability

• Dynamic load balance achieved using dynamic-programming approach to distribute sub grids

– Hybrid C++ / Fortran programming model• C++ flow control, memory management and I/O

• Fortran numerical kernels

Parallel Software Infrastructure

LBNLLBNLGXTGXTBRBR

Discretization Methodology

• Improved finite difference schemes– Fourth order in space and time

– Reduce computational times and memory requirements by more than an order of magnitude for realistic geologic models

– Improved parallel performance by reducing communication to computation ratio

• Absorbing boundary conditions– Use non-local pseudo-differential operators to represent one-way

wave propagation

– Expand with PADE approximations to obtain local representation

– Add graded local damping to minimize evanescent reflections

LBNLLBNLGXTGXTBRBR

Grating effect reduction by spatial filtering

0 50 100 150 200 250 300 350 400 450 500 550 600

x [m ]

0

50

100

150

200

250

300

350

400

450

500

Z [

m]

0 50 100 150 200 250 300 350 400 450 500 550 600

x [m ]

0

50

100

150

200

250

300

350

400

450

500

z [m

]

Vp = 1400 m/s

Vp =3400 m/s

Vp = 1400 m/s

Vp =3400 m/s

No correction Linear gradient correction

- regular grid nodes

- desired interface specification

- effective interface specification for a regular grid

V 1

V 2

- linear velocity gradient zone

V 1

V 2

LBNLLBNLGXTGXTBRBR

Nonuniform grid FD computations

Savings factors

Resource 2D tested 3D projectedMemory 1.6 1.8CPU time 3.2 3.6

0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80-0.80

-0.70

-0.60

-0.50

-0.40

-0.30

-0.20

-0.10

0.00

Subdomain 1

Subdomain 2

0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80-0.80

-0.70

-0.60

-0.50

-0.40

-0.30

-0.20

-0.10

0.00

Subdomain 2

Subdomain 1

Two half-spaces with 100% velocity contrast

No velocity contrast test

LBNLLBNLGXTGXTBRBR Numerically stable steep topography

modeling

0 . 0 0 . 2 0 . 4 0 . 6 0 . 8 1 . 0 1 . 2 1 . 4 1 . 6

X [km ]

-0.8

-0.6

-0.4

-0.2

0.0

Dep

th [

km]

0 . 0 0 . 2 0 . 4 0 . 6 0 . 8 1 . 0 1 . 2 1 . 4 1 . 6

X [km ]

-0.8

-0.6

-0.4

-0.2

0.0

Dep

th [

km]

0 . 0 0 . 2 0 . 4 0 . 6 0 . 8 1 . 0 1 . 2 1 . 4 1 . 6

X [km ]

-0.8

-0.6

-0.4

-0.2

0.0

Dep

th [

km]

LBNLLBNLGXTGXTBRBR

LBNL has a fast and accurate wave propagation algorithm for moderately heterogeneous elastic media:

We are going to apply it

LBNLLBNLGXTGXTBRBR

Hybrid Ray Tracing and FD approachspeeds up computations up to 10 times

W a v e f i e l d a t p o i n t x a t t i m e t .

u x t m x sS

u g x x s t d S( , ) ( ) ( ) ( , , )

S – d a t a e x c h a n g e s u r f a c e , m – m o d e l p a r a m e t e r s ,u - i n c i d e n t w a v e f i e l d , g – G r e e n ’ s f u n c t i o n

Fast code throughslow simple media

FD code throughcomplex media

LBNLLBNLGXTGXTBRBR LBNL PC cluster

• Vendor - SGI

• 8 dual Pentium III CPU 800 MHz

• 512 SDRAM per node

• Myrinet LAN - fast net card

• Lynux based

• Price $60K

• Completion by the end of 2000

LBNLLBNLGXTGXTBRBR Year 2001 efforts

Requested budget - 250K

• Parallel 3D elastic code

• Nonuniform grid for 3D

• 4-th order in time scheme for 3D

• Hybrid (ray tracing + FD) algorithm

• Topography for 3D

LBNLLBNLGXTGXTBRBR BoxLib Parallelism

• Hybrid C++/Fortran Programming environment.

• Library supports parallel PDE solvers on rectangular meshes.

• MPI portability: distributed and shared memory supercomputers, clusters of engineering workstations.

LBNLLBNLGXTGXTBRBR Discretization

• Fourth order accuracy in space and time based on a modified equation approach.

• 2D: fourth order scheme gives 2 times the performance of conventional second order schemes.

• 3D: fourth order 4 times as efficient.

LBNLLBNLGXTGXTBRBR

Parallel Performance

0

1

2

3

4

5

6

7

8

9

1 2 4 8 16 32 64

IBM 2

IBM 4

T3E 2

T3E 4

Wall clock run time

Number of CPUs

4-th order scheme performs better as number of CPUs increases.