1

Cactus in a nutshell ...Cactus in a nutshell ... Cactus facilitates parallel code design, it enables platform independent

computations and encourages collaborative code development between different groups.

Cactus was developed as a 3D parallel high-performance tool for Numerical Relativity which would allow easy development and maintenance especially for collaborations.

Modular structure: user code (a “thorn”) plugs into compact core code (“flesh”) through an extensible interface. Cactus allows a programmer to concentrate on his area of expertise.

User friendly: thorns can be written in C, C++ or Fortran 77/90, communication programming is optionally abstracted away from the user.

Cactus offers access to a range of computational capabilities as e.g. parallel I/O, checkpoiting, remote steering & visualization, performance analysis and provides easy access to many cutting edge software packages as Globus HDF5, PETSc.

Developed, supported and expanded by the Albert-Einstein-Institute in collaboration with many other institutes and universities worldwide, released under GNU Opensource license.

Information & Download: http://www.cactuscode.org

2

“Big mesh sizes”

Collaboration technology needed!Collaboration technology needed!

A scientist’s view on a large scale computation problem:

Initial DataEvolution Algorithms

Analysis routines

(Bettter be Fortran)

“Easy job submission”

“Large Data Output”

“Parallel would be great”

Scientist cannot be required to become experts in IT technology.

3

Collaboration technology needed!Collaboration technology needed! An IT expert’s view on a large scale computation problem:

Next Gen.Highspeed Comm.

Layers

High-performanceparallel I/O

Code instrumentation + steering

Load schedulingInteractive

Viz.Metacomputing

“Programmers, use this!”

IT experts cannot write the applications that make use of their technology

4

Large Scale Computations Require a mix of Technologies and ExpertiseLarge Scale Computations Require a mix of Technologies and Expertise

Scientific/Engineering ComponentsThe “physics” in the game

Numerical Algorithm Components

Finite differences? Finite elements? Structured meshes? Different Computational Components

Architectures and plattforms Parallelism (MPI, PVM, OpenMP, ???) parallel I/O, Data Management, Queueing Systems Visualization of all that comes out!

How can experts work together effectively?

Need a code environment that encourages collaborative HPC.

5

Cactus “glue”Cactus “glue”

Cactus „glue“

Initial Data

Evolution Algorithms

Elliptic Equation Solver

AMR/FMR

ParallelisationMPI/PVM/

OpenMP/Shmem

(High Performance-)Input/Output

Checkpointing

Code Instrumentation / Bencharking

Interactive Steering

Interactiv Visualisation

Cactus’ goal is to provide a modular framework that allows each technology to be of impact to other unrelated fields.

6

Cactus “glue” in more detailCactus “glue” in more detail Cactus “Flesh” (core) written in C

User “thorns” (modules) grouped in arrangements, can be written in F77, F90,

C, C++, (Java) Thorn -- Flesh interface fixed in 3 files written in CCL (Cactus Configuration

Language): interface.ccl: Grid Variables, Arrays, Scalars (integer, real, logical, complex) param.ccl: Parameters and their allowed values schedule.ccl: Scheduling of routines, dynamic memory allocation

Object orientated features (public, private, protected variables, inheritance)

define scope of objects and ensure non-contaminating modularity.

Flesh provides standard interfaces for Parallelization, Interpolation, Reduction,

I/O, etc.

Underlying technology can be replaced invisibly to the user

7

How to use Cactus How to use Cactus

Develop thorns, according to the interface rules interface.ccl + schedule.ccl + param.ccl Specify calling sequence of the routines for different problems and

algorithms (schedule.ccl)

For a given code: strip off parameter parsing / memory allocation / output

At compile time: Specify which thorns are to be compiled into executable. Redundancy allowed: Multiple thorns providing the same functionality. E.g.

different Evolutions systems, IO methods.

At run time: In the parameter file, turn on the thorns needed for your simulation.

8

Cactus architectures & environmentsCactus architectures & environments

Cactus applications developed on a standard workstation or laptop can beseamlesly run on a cluster or supercomputer. Supported architectures:

SGI Origin Cray T3E Compaq DEC Alpha Linux (IA32/64,PPC,NOWs) Windows NT IBM SP2 Hitachi HP SUN Ultra

.

Homogenous + heterogenous networks:

“What is a supercomputer and what is not ? “

Computing in a resource continuum

9

Cactus, Egrid, KDI Cactus, Egrid, KDI

The European Grid Forum (Egrid) aims at the cooperative use of the distributed computing resources that are accessible via wide area networks.

18 participating organisations of the 1st Egrid workshop in Poznan chose to install Cactus as a first testbed application.

10

Cactus@work HPC in Numerical RelativityCactus@work HPC in Numerical Relativity

SC 97: “Distributed Spacetime”, Intercontinental Metacomputing at AEI / Argonne / Garching / NCSA(512 node T3E calculation with live visualization San Jose)

SC98: “Colliding Neutron Stars across the Atlantic”,Connecting T3E’s in Berlin, Garching, San Diego

SC99: Cactus used by several independent demos as underlying framework.

In Fall 99: