UTAM Reproducible Research

Package and C++ Seismic LibrarySamuel Brown

November 5, 2008

OutlineOutline

URRP MotivationURRP Motivation Create a centralized, version-controlled,

software release with regression tests, promoting code reuse by students and sponsors.

Provide the ability to reproduce results from all UTAM reports in a uniform and comprehensible manner.

URRP DistributionURRP Distribution URRP is available through a secure

subversion repository.

Reproducible scripts have an interface for downloading images and intermediate results through sftp.

Firewall? No svn, no Firewall? No svn, no sftp?sftp?

Compressed repository snapshots for annual and midyear releases will be available on the UTAM website.

Compressed images and intermediate results reside in a directory tree on the UTAM website, which mirrors the URRP reports directory.

Source Source Code/CompilationCode/Compilation

bash and csh scripts for environment setup.

Top-level SConstruct file with autoconf functionality for C/C++/F90 programs.

Matlab library.

ReproducibilityReproducibility Report directories contain:

paper.tex – latex report run.py - python script other scripts, directories, etc.

run.pyrun.py run.py: uniform interface for reproducing

results python only, does not use scons imperative, not declarative generates and runs shell scripts interactively simple mechanisms for downloading data and

controlling computation with sources and targets

can interface with PBS

run.pyrun.py run.py consists of a number of individual

processes

A ‘process’ is a call to the URRP python function process( )

1 process for downloading images

1 process for compiling latex paper

1 or more processed for reproducing results or downloading intermediate results

A Simple ProcessA Simple Process

Process(cmds=[’matlab –nosplash –nodisplay < xcorr.m’]

)

A More Involved A More Involved ProcessProcess

Process(cmds=[‘ ucsl_fdac par=mod1.par’],

sources=[ ‘vp.rsf’, ‘recv_coord.txt’], targets=[ ‘csg1.su’], docmds=1, wdir=‘./csgs’, bdir=‘./batch’, sdir=‘./mod’, pbs=1, nodes=4, ppn=2, walltime=0:30:00))

OutlineOutline

UCSL MotivationUCSL Motivation Provide library for rapid development of

flexible, robust, high-performance research codes.

Find an appropriate balance between imperative and object-oriented programming.

Provide a high level of abstraction to enable performance and flexibility, ie task-specific file objects with optional MPI I/O.

ApplicationsApplications Development initially driven by:

Flexible 2D/3D modeling/RTM. Ray tracing and interferometric imaging of

earthquake data.

Forward Modeling – Forward Modeling – PMLPML

Problem: When implementing PMLs, there are up to 26

regions requiring a different combination of fields/damping.

To get the best result, all valid regions should be implemented, with full ghost region communication.

For simplicity and performance in the FD kernel, PML regions should reside in separate volumes.

This greatly complicates domain decomposition and communications.

cart_decompcart_decomp C++ domain decomposition object.

Computes balanced domain decomposition distributed along any combination of axes using a boundary condition cost function.

Uses MPI topologies. Builds arrays of subdomain volumes for each

PE. FD application only has to ask for as many

volume arrays as are necessary for a given implementation.

cart_halocart_halo C++ ghost region communication object.

Each subdomain group requiring ghost region communication creates a halo object. For 2nd order time acoustic modeling, the subdomain group would be a 2 x nsubdomain volume array.

cart_halo handles all communication between local and remote subdomains with two functions: start_update(int tslice, int half) finish_update()

Ability to overlap communication and computation with option to split subdomains along the z axis.

Flexible Flexible ImplementationsImplementations

Source and receiver groups are also abstracted as objects.

Implementing a new parallel modeling code is simply a matter of: Providing kernels for interior and boundary

regions. Providing a time-stepping loop. Writing a small amount of initialization code.