Initial Results from the Integration of Earth and Space Frameworks

Cecelia DeLuca/NCAR, Alan Sussman/University of Maryland, Gabor Toth/University of MichiganAbstract

There is increasing call to couple Earth and space models, as the interplay between the regions is an active topic of basic research and a key factor in operational prediction systems. Two pilot projects to combine Earth and space software frameworks have produced initial technical results. The projects, which were initiated under NASA and NSF funding, demonstrate 1) transfers of data between models using the Earth System Modeling Framework (ESMF) and those that are part of the Center for Integrated Space Weather Modeling (CISM) and 2) transfers of data between models using ESMF and those running under the Space Weather Modeling Framework (SWMF). We describe technical approaches, capabilities achieved, and possible next steps.

GEOS-5

surface fvcore gravity_wave_drag

history agcm

dynamics physics

chemistry moist_processes radiation turbulence

infrared solar lake land_ice data_ocean land

vegetation catchment

coupler

coupler coupler

coupler

coupler

coupler

coupler

• The ESMF architecture is based on components with standard interfaces and standard methods

• An ESMF application runs as a single executable• New components can be added to the application

systematically• Data in and out of components are state types that

contain fields, bundles of fields, arrays, or nested states• Toolkits include regridding and redistribution methods and

utilities such as time management• ESMF is free for download at http://www.esmf.ucar.edu

ESMFComponent 1

(e.g. land)

ESMFComponent 2

(e.g. atm)

ESMF App Driver

ESMF Coupler

Configuration InterComm is used for data transfers between an ESMF

application and a CISM model (or other non-ESMF code) These are in separate executables Data transfers can be from anywhere, to anywhere Minimal changes to codes Each code doesn’t need to know anything about the internals of

the other

CISM Model

Multiple executables

InterComm data transfer and sequencing

1. ESMF and CISM / InterComm Coupling

ESMF EarthSystem

Component

ESMF App Driver

SWMF Driver

ESMF SWMF Component

ESMF Coupler

Configuration The entire SWMF is wrapped as an ESMF component The SWMF component can communicate data to one or more Earth

system components The application is run using an ESMF driver The application runs as a single executable

Single Executable

2. ESMF and SWMF Coupling

Conclusions

Space Weather Modeling Framework

Center for Integrated Space

Weather Modeling

Earth System Modeling Framework

TECHNICAL APPROACH

Mode of execution Single executable Multiple executable Single executable or multiple executable

Mode of component concurrency

Sequential, concurrent, mixed

Concurrent Sequential, concurrent, mixed

Mode of inter-component data exchange

Argument list before/after component run

Put/get anywhere Argument list before/after component run

Standard data structure for inter-component data exchanges

No, although interfaces are generic

Yes Yes

Framework support for nested components

No No Yes

Parallelism Components may use MPI or OpenMP

Components may use MPI or OpenMP

Components may use MPI or OpenMP

Execution environment Command line or GUI interface

Command line Command line

Supported platforms Linux cluster with NAG, PGF90, and Lahey compilers, OSF1, Altix, SGI 3000, Mac OSX with NAG and XLF

Linux, AIX, and Solaris AIX, SGI IRIX64, OSF1, Linux with Intel, PGI, NAG, Absoft, and Lahey compilers, Mac OSX with XLF and Absoft, Altix, Cray X1

COMPUTATIONAL APPROACH

Supported grids and grid methods

Supported grids and methods: any 1D, 2D, or 3D logically Cartesian grid, and any 2D or 3D block adaptive grid; 2nd order interpolation methods are provided, and user supplied interpolation schemes can also be used

Any grids and methods supported by Overture

Intrinsic support only for simple logically rectangular grids, bilinear and first order conservative regrid

• In CISM applications, the InterComm library (University of Maryland) is used for parallel data transfers between models that are in separate executables

• The Overture framework (LLNL) for solving PDEs in complex moving geometries is used to write coupling code, which may be put in its own separate executable or within a subroutine of one of the models being coupled

• The approach requiress minimal modification to existing programs

• InterComm and Overture are both general purpose, freely downloadable packages

Global Magnetosphere

Ionosphere Electrodynamics

Inner Magnetospher

e

Radiation belts

Solar Energetic Particles

Upper Atmosphere

Solar Corona

Eruptive Event Generator

Inner Heliosphere

• The SWMF architecture is based on components with generic interfaces

• A SWMF application runs as a single executable• Many different configurations may be run• The SWMF is freely available at

http://csem.engin.umich.edu

Polar Wind

Plasmasphere

Photosphere to Corona

SWMF

Physics Module Layer

Infrastructure Layer

Superstructure Layer

User Interface Layer

Web-based Graphical User Interface conf iguration, problem setup, job submission, job monitoring, result v isualization

Framework Services component registration, input/output, execution control, coupling toolkit

Component Interface component wrappers and couplers

Utilities phy sics constants, coordinate transf ormation, parameter reading, time/date conv ersion, f ield line tracing, input/output units, prof iling

SC solar

corona

UA upper atm

IM inner

magnet.

Physics Module Layer

Infrastructure Layer

Superstructure Layer

Application Driver e xecution and sequencing control, hierarchical composition

Component Interface component w rappers and couplers

Utilities regridding and communication toolkits, time management, parameter reading, input/ output, logging and error handling, prof iling

ATM OCN LND

Modifications Required

3 new ESMF functions required• 1 to translate ESMF array descriptors into InterComm descriptors

ESMF_ArrayExternalICTranslate() • 2 to export/import ESMF sub-arrays to another program

ESMF_ArrayExternalICSend()ESMF_ArrayExternalICRecv()

Status• Implementation of prototype complete and tested• Currently limited to block array distributions• InterComm functions will appear as option in ESMF release• Prototype coupling of ESMF model to NCAR space weather

model is being discussed

Future WorkWe will demonstrate in a simple model the ability to execute a data transformation between an ESMF component and a CISM component using InterComm, with the ESMF component using ESMF interfaces and the CISM component using native InterComm interfaces.

MotivationThe main motivations for interfacing ESMF-based Earth simulations to space simulations and vice versa are the desire to better express boundary conditions for these domains, and the desire to improve representation of the boundary region. End goals are to deepen understanding of physical processes and to produce better forecasts. The ability to exchange component versions between the CISM and SWMF frameworks would afford space weather researchers more opportunities for experimentation and collaboration. There are also a number of upper atmosphere researchers who are interested in implementing their component versions in an ESMF-compatible way.

The initial results from these pilot projects are proof of concept that SWMF components, CISM components, and ESMF components can be connected to each other to create applications.

A natural next step is to explore the possibility of component exchanges between the CISM and SWMF frameworks, perhaps mediated by ESMF interfaces.

Example:GEOS-5 AGCM

FRAMEWORK: Earth System Modeling Framework (ESMF)

FRAMEWORK: Space Weather Modeling Framework (SWMF)

FRAMEWORK: Center for Integrated Space Weather Modeling (CISM) / InterComm

Framework Comparison TablePilot Projects

Status• Implementation of prototype complete and tested• ESMF functions appear as options in the SWMF distribution

Similarities Between ESMF and SWMF

ESMF and SWMF are layered and structured in a similar fashion (see diagrams below). This made it rather straightforward to modify the SWMF so that it could be run as an ESMF component. Problems that occurred during integration were due largely to the immaturity of aspects of the ESMF software at the time of the implementation.

ESMF SWMF