FY 2003 Technology Master Plan

for

Programming Environment and Training Supportto the

DoD High Performance Computing Modernization Program

Submitted by:The MOS Consortium

7 November 2002

High Performance Computing Modernization OfficeArlington, Virginia

TABLE OF CONTENTS

PARAGRAPH PAGE

Introduction 1.0 2

Goals for the TMP 1.1 2

DoD Approach to Technology Planning 1.2 2

FY04 Project Task Process 2.0 2

Solicitation/Recommendation Plan 2.1 3

Integrated Master Schedule for FY03 2.2 3

FA Roadmaps 3.0 4

CWO 3.1 4

EQM 3.2 9

CE 3.3 12

SIP 3.4 16

IMT 3.5 19

FMS 3.6 22

ET 3.7 29

CFD 3.8 31

CSM 3.9 36

OKC 3.10 34

EOTC 3.11 38

Multidisciplinary and Cross-Component Roadmaps

3.12

39

Summary 4.0 40

1

1.0 Introduction. The Technology Master Plan (TMP) is a MOS consortium document that

looks at the relevant technologies in the Programming Environment and Training (PET) program

with respect to what the current state-of-the-art is, where technology development is headed, and

what additional development is required to implement these technologies for the DoD High

Performance Computing (HPC) user base. The TMP lays out the perceived needs within each

Functional Area (FA) as well as across multiple FAs. It will continue to evolve as the PET

program progresses. This is the Year 2 version of the plan.

1.1 Goals for the TMP. The overarching goal for the TMP is to capture the evolving “system

of plans” for all the MOS PET FAs to ensure that DoD HPC users have access to and can

implement the most current technologies to benefit the warfighter. This “system of plans” forms

the primary basis each year for the development and selection of PET Project Tasks. As such, it

captures the thinking and ideas of a large number of Government PET leadership, Government

Computational Technology Area (CTA) leads, DoD users, Functional Area Points of Contact

(FAPOCs), on-site PET personnel, etc. — i.e., the full set of stakeholders for PET. Since those

thoughts and ideas are constantly being upgraded as technical and scientific progress is made, the

TMP is, by definition, a “living” document. But the intent is to provide the MOS team as well as

the Government with a solid rationale for proceeding along certain paths of research,

development and technology transfer that are most relevant to critical DoD needs in HPC.

1.2. DoD Approach to Technology Planning. Over a number of years, the majority of DoD

laboratories have adopted a generic “roadmap” approach to both foresee progress in technology

development programs as well as to justify current and out-year budget requests. This approach

uses progressive roadmaps that succinctly describe sequential or parallel tasks to be

accomplished (and funded) in order to reach broad-based research goals. Such plans typically

look forward 3-5 years (the standard DoD funding cycle) and are updated at least annually.

The PET TMP uses the roadmap format to forecast core and project efforts for the “fiscal

year after next” based on a knowledge of the core and project effort for the current fiscal year.

2.0 FY04 Project Task Process. The FY04 Project Task process has been initially delineated

in the Government’s Selection/Evaluation Plan (SEP), released to the PET contractors on 24

October 2002. It follows the same general process as that for the previous year. Based on the

schedule outlined in the SEP, the contractors’ Solicitation/Recommendation Plans (SRPs) are

due back to the Government on 12 November 2002. The white papers are due to the Government

2

on 7 February 2003. The Government will select white papers and request full proposals on 4

March 2003. Final proposals are due back to the Government on 6 May 2003.

2.1 Solicitation/Recommendation Plan. The SRP will reflect the views of the MOS

leadership on how we will solicit white papers, the process by which we will recommend white

papers to the Government for full proposals, and the process by which we will recommend

proposals to the Government for funding. This process will be funding limited, and MOS will

use an allocation spreadsheet to support our recommendations. The full SRP will be delivered to

the Government on 12 November.

2.2 Integrated Master Schedule for FY03. Based on the dates in the SEP, MOS has updated the

Integrated Master Schedule (IMS) for this fiscal year. It includes all key dates of which we are currently aware.

Start End PET Activity Action OPR01-Oct-02 01-Oct-03 FY04 Projects    

01-Oct-02 28-Oct-02 PreparationGovernment Evaluation/Selection Plan (White Papers and Proposals)

Govt

28-Oct-02 28-Oct-02   Distribution of Govt selection plan to KTRs COR, KO

28-Oct-02 12-Nov-02  Preparation of Solicitation/Recommendation Plan (SRP)

KTRs

12-Nov-02 12-Nov-02   SRP Due KTRs13-Nov-02 21-Nov-02   Govt review of SRP Govt21-Nov-02 21-Nov-02   Govt feedback to SRP COR09-Dec-02 04-Mar-03 Phase I: White Papers    

09-Dec-02 07-Feb-03   Preparation of White Papers (WPs) KTRs

07-Feb-03 07-Feb-03   WPs due to HPCMO KTRs10-Feb-03 21-Feb-03   Govt WP Review KTRs24-Feb-03 28-Feb-03   Selection of WPs for 2nd phase Govt

03-Mar-03 04-Mar-03  Notification to KTRs of selections of WPs for development of full proposals

COR, KO

05-Mar-03 28-Jul-03 Phase II: Proposals    

05-Mar-03 25-Apr-03   FAPOCs and PIs prepare full proposals

FAPOCs, PIs

25-Apr-03 25-Apr-03   Proposals due to MOS Mgt FAPOCs, PIs

25-Apr-03 06-May-03   MOS Mgt prepares proposal book and recommendation MOS Mgt

06-May-03 06-May-03   Proposals due to HPCMO KTRs

06-May-03 06-May-03   Recommendations from KTRs due KTRs

07-May-03 13-Jun-03   Govt Review of Proposals GovtStart End PET Activity Action OPR

16-Jun-03 19-Jun-03   Decision on Proposals COR, KO10-Jul-03 28-Jul-03   Cost negotiations with FAPOCs,

3

Government PIs, MSU

28-Jul-03 28-Jul-03   Govt issues request for task orders to KTRs COR, KO

23-Sep-03 30-Sep-03   MSU issues contracts mods to subcontractors MOS Mgt

01-Oct-03 30-May-04 Phase III: Implementation FY03 project tasks are conducted PIs

31-Oct-02 31-Oct-02 TMP for KY2 due   DPD, FAPOCs

16-Nov-01 22-Nov-01 SC02 @ Baltimore All03-Feb-03 07-Feb-02 PET Technical Review @ CAU All

19-May-02 23-May-03 PET Annual Contract Review @ ARL All

09-Jun-03 13-Jun-02 UGC 2003 @ Bellevue, WA All         

         Acronyms: KTR = Contractor, COR = Contracting Officer's Representative (Leslie Perkins), KO = Contracting Officer (Brenda Spence for MOS), PD = Program Director (Joe Thompson), DPD = Deputy Program Director (Dick Pritchard), PI = Principal Investigator (on a project)PM = Program Manager (Zak Kozak)    

3.0 FA Roadmaps. The MOS FA roadmaps were updated by the FAPOCs, in conjunction with

their User Advisory Panels (UAPs) and other resources.

3.1 Climate/Weather/Oceans Modeling (CWO). MOS CWO efforts are organized to meet

these general strategic objectives:

Improve the performance and scalability of strategic ocean and atmospheric models

to enable higher resolution simulations and minimize time to produce forecasts.

Improve the HPC user environment and tools to enable rapid porting and tuning of

strategic CWO codes and of visualization/analysis of CWO data.

Evaluate and develop new technologies and techniques in code coupling, data

assimilation, ensemble calculations, and meshes and coordinate systems for

enhancing CWO simulation capabilities and accuracy.

The CWO FAPOC works with the CWO CTA leader, the UAP, and on-site staff to compile

and update a list of the highest priority specific strategic issues that DoD CWO researchers face.

This list is used to develop a tactical execution plan for on-site activities, develop and deliver

training classes, and guide potential PET project PIs in formulation of proposals. Many of these

issues are dependent on cross-cutting technologies (CE, ET) and other CTAs (EQM, CFD).

These inter-FA relationships are noted and these issues discussed with other relevant FAPOCs to

ensure that PET activities to address these inter-FA issues are coordinated.

4

The use of on-sites to support CWO users will expand in Year 2 to encompass users not

located at the three locations with on-sites. For example, users at AFRL and AFWA have

requested assistance. John Romo, the on-site at Monterey, will also support these users because

of their overlapping applications area (atmospheric modeling/weather prediction). CWO needs at

other locations will be assessed and addressed with current on-sites.

There are a number of top-level strategic issues that CWO is addressing [NOTE: The issues

below are not ranked in relative importance. However, the list is used to develop priorities for

each type of PET activity: for on-site support activities, for training classes, and for new projects.

Most issues can be addressed via multiple types of PET activities (training, on-site support,

and/or projects), but for each the greatest potential for impact might come from a different

activity. These activities are listed in order of expected impact for each issue. In general, the first

four issues are most amenable to CE and ET projects, CE or CWO on-site support, and CE

training. The fifth issue is most amenable to CWO on-site support and CWO training. The

remaining issues are most amenable to CWO projects.]:

Porting and optimizing CWO codes to/for new HPC systems

Inter-FA relationships/dependencies: CE

PET Activities: CWO on-site support, CWO and CE projects, CWO training

CWO codes are among the largest consumers of HPC cycles, so CWO users grab

cycles on whichever systems they can get allocations. However, efficient

utilization of these systems requires making codes portable as well as efficient:

time is often lost learning how to execute in a new environment, and once ported

the code must take advantage of the new architecture efficiently to scale to large

problems.

Easier programming tools to enhance porting and optimizing (profilers, debuggers,

numerical libraries, parallel libraries, etc.)

Inter-FA relationships/dependencies: CE

PET Activities: CE projects, CE training (also CWO or CE on-site support)

Common and consistent programming tools (see #2) and user environments (queue

structures, batch queue systems, data handling systems, etc.).

Inter-FA relationships/dependencies: CE

PET Activities: CE projects, CE training (also CWO or CE on-site support)

5

This issue and the previous one are not the same. CWO users would prefer both

easier and consistent tools and environments, but achieving either is would be a

big success. Both of the first two issues are thus very important.

Enhanced capability for executing ensemble simulations—scheduling jobs/locating

resources, moving data, collecting results, etc.

Inter-FA relationships/dependencies: CE

PET Activities: CWO and CE projects, CWO training (also CWO on-sites)

Ensemble calculations are used to improve accuracy and bound predictions, but

are currently most often performed as independent simulations and then data is

collected at the end. Computational grids, meta-scheduling, automated data

handling tools, etc. would enhance execution of ensemble simulations.

More scalable programming algorithms/solvers, tools, and systems (lower latency

computing systems programming libraries, 2D/3D elliptic solvers, etc.)

Inter-FA relationships/dependencies: CE, ET

PET Activities: CE and ET projects, MSRC/ADC/DDC coordination, and on-site

support

High-resolution simulations are critical to making accurate predictions and must

execute rapidly to produce useful forecasts. Thus, CWO is even more dependent

than most FAs on highly scalable components.

Enhanced visualization capabilities of model output for remote users.

Inter-FA relationships/dependencies: ET

PET Activities: ET projects, on-site support

Code coupling between models (e.g. deep-ocean and coastal, ocean and atmosphere

models, ocean and ice models, and including biology/chemistry models, aerosols,

contaminant/drift transport, etc.).

Inter-FA relationships/dependencies: ET, EQM and others (CEA, SIP, etc.)

PET Activities: CWO and ET projects

Investigations/evaluations of non-uniform meshes for estuaries/coastlines

Inter-FA relationships/dependencies: EQM, ET

PET Activities: CWO, EQM and ET projects, on-site support

Faster (esp. more scalable and better load-balanced) variational data assimilation

techniques including automatic adjoint compilers

6

Inter-FA relationships/dependencies: EQM, ET, CE

PET Activities: CWO, ET, CE, EQM projects (other CTAs?)

Investigations/evaluations of vertical coordinate systems for accuracy

Inter-FA relationships/dependencies: EQM, CFD

PET Activities: CWO, EQM projects

CWO project tasks for FY03 include:

CWO-03-002: Infrastructure Development for Regional Coupled Modeling

Environments (PI: John Michalakes, National Center for Atmospheric Research

[NCAR])

CWO-03-008: Enhancing the Capabilities of a Three-Dimensional Nearshore Ocean

Circulation Model System (PI: Chandrasekher Narayanan, University of Southern

Mississippi [USM])

Closely related project tasks include:

CE-03-001: Application Portability

CE-03-006: Data Management and I/O for DoD Applications

CE-03-004: PAPI Deployment, Evaluation, and Extensions

CE-03-002: Consistent Well-documented Computational Environment

ET-03-001: EnVis and EnVisU — Distributed High Performance Batchmode

Visualization

ET-03-002: Geometry-Grid Tool Kit for Multidisciplinary Applications

7

The strategic roadmap for CWO is as follows:

The first five strategic issues will be covered by on-site support. CWO project and training

class formulation in FY04-06 will be coordinated closely with the CE and ET FAPOCs because

of the significant relationship of strategic issues to those FAs. CWO users are also interested in

coupling models, so CWO project formulation in these topics in FY04-06 will be coordinated

with other relevant FAs, in particular EQM and CFD. CWO users are interested in mesh/grid and

coordinate schemes that increase accuracy, performance, and capability: irregular/unstructured

meshes for coastlines, bays, and inland seas; and vertical coordinate systems for quantifying

tradeoffs and determining when each is most appropriate. CWO project formulation in these

topics in FY04-06 will be coordinated with other relevant FAs, in particular the “grid-based”

FAs. Finally, CWO users are interested in faster data assimilation techniques (probably the most

uniquely CWO of the top 10 strategic issues). CWO project formulation in this area will be

conducted in close collaboration with DoD users requiring better assimilation techniques.

The CWO UAP includes:

Alan Wallcraft (Naval Research Laboratory-Stennis Space Center [NRL-SSC])

Jane Smith (U.S. Army Engineer Research and Development Center [ERDC])

Rich Hodur (NRL-Monterey)

Frank Ruggiero (Air Force Research Laboratory [AFRL])

8

Strategic Issue1. Porting/optimizing CWO

codes to new HPC systems

2. Easier tools and environments

3. Common and consistenttools and environments

4. Enhanced ensemble simulation capabilities

5. More scalable algorithms and tools

6. Enhanced visualization capabilities

7. General code coupling for ocean, atmos, bio/chem…

8. Unstructured and irregular meshes

9. Exploring vertical coordinate systems

10. Faster, load balanced data assimilation

FY02 FY03 FY04 FY05 FY06

CWO007 CWO011

CWO & CE projects

CE projects

CWO & ET projects

CWO/CE/ET Training

CE Training

CE projects

CE & ET projects

CWO & EQM projects

CWO projects

CWO & ET projects

CWO projects

CE Training

CE Training

CWO Training

CWO Training

CE009

CE010

CWO Training

CWO Training

CWO & ET Training

CWO Training

Functional Area: CWO

CE002

CE004

CWO008 CE01/06 CWO & CE Projects

CWO002

ET001

ET002

ET012 ET017

CE019

New ‘top10’ issue in FY 02

3.2 Environmental Quality Modeling (EQM). The top priorities for EQM support include:

Development and implementation of new accurate parallel discretizations with

adaptivity based on a posteriori error estimators and scalable solvers for

environmental quality models.

Code coupling of multiphysics, multiphase, multicomponent flow and reactive

transport. Both loose and tightly coupled models would be considered.

The most pressing EQM issues are:

Accurate transport schemes defined on unstructured meshes.

Coupling of groundwater and surface water flow models with reactive transport.

Accurate flow models for the vadose zone.

A posteriori error estimators for flow and reactive transport

Improved linear and nonlinear solvers

EQM project tasks for FY03 include:

EQM-03-003: Enhancements to ADH and CE-QUAL-ICM (PIs: Mary F. Wheeler

and Clint Dawson, The University of Texas at Austin [UT-Austin])

EQM-03-006: Error Estimators/Indicators for Environmental Quality Modeling (PIs:

Mary F. Wheeler and Clint Dawson, UT-Austin)

The strategic roadmap for EQM is as follows:

EQM project areas for FY04-06 are as follows:

EQM/CWO-ADH: Enhancements to the Adaptive Hydrology Model and CE-QUAL:

Two very important codes developed at ERDC are the hydrodynamics code Adaptive

Hydrology (ADH) Model and the water quality model CE-QUAL-ICM. The planned

enhancements include state-of-the-art transport with adaptivity in both codes as well

as multiphase multi-component flow with biogeochemistry in ADH. Time stepping, 9

F u n c t io n a l A r e a : E Q M

S tr a t e g ic F o c u s e d A r e a s F Y 0 2 F Y 0 3 F Y 0 4 F Y 0 5 F Y 0 6

E Q M 0 0 3R e d u c e d F u n d in g b y 1 4 %

2 . C o d e C o u p l in g a n d E n h a n c e m e n ts

3 . Im p r o v e d A l g o ri th m s fo r F lo w a n d T r a n s p o rt E Q M 0 0 3

E T 0 0 3E T 0 0 4

R e d u c e d F u n d in g b y 7 0 %

5 . C o m p u t e r S c ie n c e T o o ls

1 . A D H E n h a n c e m e n ts a n d C E -Q U A L - IC M E Q M 0 0 1 N O T F U N D E D

N O T F U N D E D

N O T F U N D E D

N O T F U N D E D

4 . E rr o r E s t im a to r s a n d A d a p ti v ity E Q M 0 0 6 N O T F U N D E D

linear and nonlinear solvers, and operator splitting strategies that include treatment of

uncertainty and risk assessment will be major topics considered.

EQM/CWO/CE-MC: Code Coupling and Enhancements: Advanced multiphysics,

multi-resolution and multi-domain simulations are typically composed of multiple

components that require couplings and interactions at different levels. Different

physical processes (e.g. flow, transport, and reactions) may occur within the same

physical domain or in different physical domains (with different meshes and time-

steps), giving rise to varied interaction requirements and constraints. These range

from tight in-core coupling, to loose coupling through mortar elements,

interpolation/projection of quantities from one computational mesh to another, and

disk-based data coupling. Parallel implementation issues must be examined to ensure

that the couplings are efficient. Specific examples of DoD applications in EQM and

CWO giving rise to multi-physics models include the Northern Gulf Littoral

Initiative, modeling of the Florida Everglades, and ongoing research in the coupling

of hydrodynamic and groundwater flow models. In these applications, one needs to be

able to couple, e.g., surface water codes, groundwater codes, and reactive transport

codes. EQM proposes to investigate computational science tools that will help to

create a framework for studying complex physical systems by allowing for robust

interfaces/couplings between DoD codes. Specific codes of interest include TABS,

ADCIRC, CH3D-Z, CE-QUAL-ICM, and ADH, but other codes may also be

impacted as well.

EQM/CWO/ET-FEM: Improved Algorithms for Flow and TransportImproved

discretization schemes for mesh-based computations crosscut a number of functional

areas. In EQM and CWO applications, time-dependent flow and reactive transport

problems arise, and require efficient, accurate discretization strategies. New

algorithms such as Discontinuous Galerkin, and related finite volume methods

provide mechanisms for hand-ling highly advective flow on unstructured and

adaptive meshes, and are locally conservative. Investigation and implementation of

these algorithms for flow and transport applications are ongoing and will continue.

More traditional finite element and finite difference methods are still suitable for

many applications and/or different physical processes. Therefore, multi-algorithmic

10

strategies that couple traditional methods with these emerging discretization strategies

are also being considered.

EQM/CWO/ET-EEAU: Error Estimators and Adaptivity and Uncertainty: In

numerical simulations, errors due to mesh discretization, function approximation,

temporal discretization, and uncertainty in physical parameters, can often lead to very

inaccurate results. While a priori error analysis demonstrates convergence of the

solution with fine enough resolution, this type of analysis does not give an estimate of

the error for a given level of resolution. Nor do these estimates provide guidance in

where to put mesh points or higher order polynomials in order to obtain accurate

results most efficiently. Thus, the reliability of computed quantities is an important

issue in numerical modeling; moreover, it is desirable to compute reliably accurate

solutions with minimal degrees of freedom. Error estimation involves post-processing

the numerical solution to obtain computable bounds on the difference between the

numerical and true solutions. In most cases, these bounds are themselves only

approximate, but still can be useful as error indicators; i.e., indicating where the

primary solution errors are occurring. Through these tools, one can not only estimate

the accuracy of a computed solution, but also use this information to adaptively

change the mesh and/or approximating space. There has been substantial research on

such techniques for linear, steady state problems. Only recently have researchers

begun to consider extending these ideas to nonlinear, time dependent applications,

such as those arising in CFD and EQM. Here error estimators for local and global

error measures and other quantities of interest in flow and transport simulations as

well as implementation and testing of estimators in parallel adaptive codes will be

investigated. Determining acceptable confidence intervals in the presence of uncertain

data will also be addressed.

The EQM UAP includes:

Bill Burnett (Commander Naval Meteorology and Oceanography Command

[CNMOC])

Cheryl Ann Blain (Naval Oceanographic Office [NAVO])

Mark Dortch (ERDC)

Stacy Howington (ERDC)

Fred Tracy (ERDC)

11

Pei-Fang Wang (U.S. Navy Space and Naval Warfare Systems Command

[SPAWAR])

3.3 Computational Environments (CE). The top priorities for CE are:

A computational environment that is consistent, well-documented, and easy-to-use

across the MSRCs/ADCs/DDCs

Debugging and performance analysis tools that are scalable and easy-to-use in the

MSRC/ADC/DDC environment

CE will concentrate on:

Enabling DoD users to determine what performance they are getting and improve that

performance on MSRC/ADC/DDC platforms

Parallelization strategies and programming practices that enhance application

portability across platforms

Tools and strategies for efficient file management and I/O

Use of COTS tools and collaboration with tool vendors on new tool features

Training in new programming tools and methodologies (Core)

CE needs as determined by the UAP are:

Consistent computational environment

Same tools (i.e., debuggers, performance analyzers) across sites and HPC

platforms

Adequate documentation and user support for deployed tools

Performance evaluation

Determine what performance my application is getting

Improve application performance on HPC platforms

Easy to use performance analysis tools

Application portability and scalability

Parallelization strategies that are portable across platforms

Tools and programming language practices that enhance portability

Application scalability to hundreds and even thousands of processors

File management and I/O

Manage input and output files and stage them where my application will run

Use parallel I/O and asynchronous I/O to improve performance

Share files with other users

12

Debugging

Use an interactive debugger in the batch queuing environment

CE needs related to specific FAs are:

EQM need for linear and non-linear solvers, including iterative methods

and preconditioners

SIP (and other FA) need for integration of MATLAB with HPC

environment

CWO (and other FA) need for cluster computing

CE project tasks for FY03 include:

CE-03-001: Application Portability (PI: David Cronk, The University of Tennessee-

Knoxville [UTK])

Guidelines and tools for writing portable MPI and OpenMP programs in Fortran

and C

CE-03-002: Consistent Well-documented Computational Environment (PI: Shirley

Moore, UTK)

CE-03-004: PAPI Deployment, Evaluation, and Extensions (PI: Shirley Moore,

UTK)

CE-03-006: Data Management and I/O (PI: David Cronk, UTK)

Asynchronous and/or parallel I/O for I/O intensive applications

Efficient file staging

The strategic roadmap for CE is as follows:

13

Strategic FocusAreas

1. Consistent environment

2. Performance evaluation

3. Portability and scalability

4. File management and I/O

5. Debugging6. FA-specifc

FY02 FY03 FY04 FY05 FY06

Functional Area: CE

CE009

CE010

CE012

CE010

CE009

CE-ENV

CE019

CE012

CE-ENV

CE002CE004

CE004 CE-PERF

CE-DTC

CE001 CE-PORT

CE002CE-RTI

CE-IOCE006

Future areas of concentration for CE include:

CE-ENV: Consistent Environment: In order to be maximally productive, DoD users

need a consistent well-documented computational environment across the

MSRCs/ADCs/ DDCs. Although individual centers may have special requirements

due to specific architectures or application areas, the general computational

environment should be as consistent as possible. The computational environment

includes compilers, message passing libraries, numerical libraries, and debugging and

performance analysis tools, as well as data management and visualization tools. All

components need to be working properly in the MSRC/ADC/DDC batch queuing

environments and be adequately supported and documented. Cross-platform tools

should be made available wherever possible so that users do not need to learn a

different tool interface for each platform. The environment should be updated on a

regular basis to keep pace with changes in architectures and operating systems. New

and emerging CE and ET technologies should be evaluated for possible adoption.

CE-PERF: Performance Evaluation: Users need to be able to determine what

performance their applications are getting and whether or not and in what ways this

performance can be improved. Easy to use tools are needed to collect and analyze

performance data. Guidelines and tutorials on interpreting performance data are

needed, as are case studies of performance analysis of real DoD applications to serve

as models. Effective collaborative tools are needed to allow both synchronous and

asynchronous consulting between application developers and performance analysis

experts, including collaborative browsing and interpretation of performance data and

its relationship to application source code.

CE-PORT: Application Portability and Scalability: Users need to know about and be

able to use parallelization strategies and programming practices that result in

application programs that are portable across MSRC/ADC/DDC platforms. Tools are

needed that assist users in checking their programs for portability problems. Problems

with compilers and library implementations or with ambiguities in standards should

be reported to the relevant vendor and/or standards body and fixed. Applications need

to be able to scale to hundreds and even thousands of processors. Users need

assistance in designing and implementing algorithms and parallelization strategies

14

that scale well. Analysis tools are needed that use scalable log file formats and

scalable performance analysis displays.

CE-IO: File Management and I/O: Users need efficient means of managing input and

output files and staging them where their application will run. Files may be need to be

stored for future use and may need to be shared with other users at the same or other

sites. Some I/O intensive applications are candidates for using asynchronous and/or

parallel I/O to improve application performance. Asynchronous I/O can keep data in

memory until it needs to be written out to disk. The use of MPI-2 I/O in MPI

applications simplifies I/O programming and produces a portable MPI application

while taking advantage of vendor optimized I/O.

CE-DTC: Desktop to Teraflop Computing: Many DoD users develop and test

applications in their desktop environments using tools such as MATLAB. As these

users scale up their applications, they begin to hit a performance wall with either

increasing runtimes or memory usage or both. Effective means of migrating portions

or all of their applications onto high-performance machines are needed, ideally while

maintaining as much of their familiar desktop environment as possible. Such

migration can take various forms — e.g., client-server metacomputing systems such

as NetSolve, or web portals that handle job submission. Commodity cluster

computing environments may offer the best price/performance ratio for some codes

and may serve as easier development environments from which migration to HPC

systems can occur.

CE-RTI: Real-time and Interactive Capabilities: MSRC/ADC/DDC resources have

for the most part been available for use in batch mode with only minimal support for

interactive use and no provisions for meeting real-time requirements. Interactive use

of HPC resources, combined with runtime control for purposes of debugging,

performance analysis, and data extraction and analysis, can greatly increase user

productivity by shortening the turnaround time for performing mission critical tasks.

Integration of real-time high performance embedded computing with data

manipulation performed by traditional HPC (as described in the SIP-HPdC FY03-06

project area) will place real-time requirements on MSRC/ADC/DDC resources.

Feasibility studies followed by prototype testbed implementations and finally by

15

production HPC systems that support interactive and real-time requirements will help

meet these goals.

The CE UAP includes:

Robert Alter (ERDC MSRC, Computational Science and Engineering Group [CSE])

Joseph Baum (Science Applications International Corporation [SAIC], CFD and

CSM)

Michael Gourlay (Northwest Research Associates [NWRA], CFD)

Daniel Pressel (Army Research Laboratory [ARL] MSRC)

Fred Tracy (EQM)

Chris Yerkes (SPAWAR, SIP)

Alan Wallcraft (CWO)

Richard Walters (ERDC MSRC, SciVis)

Joseph Werne (CFD)

3.4 Signal Image Processing (SIP). The top priorities for SIP are:

Improving techniques for conditioning, transmission, storage, and processing of

sensed signals

Determining and refining methods that support the extraction of information from

sensed signals to support

The “Kill Chain”

Homeland Security

Information Superiority

SIP will concentrate on:

New algorithm designs for conditioning, processing and information

extraction that exploit both signal processing and phenomena

Incorporating life cycle software tools to manage exploration, design,

implementation, and deployment using a common framework

Improved processing speed in the design, simulation, verification and

implementation phases (including real-time requirements)

Improved data transmission and storage, because sensors produce

increasingly overwhelming amounts of data

Increased use of COTS tools

Training in new software tools and methodologies (Core)

16

SIP project tasks for FY03 include:

SIP-03-001: High Productivity Computing for Signal and Image Processing (PI: Stan

Ahalt, The Ohio State University [OSU])

SIP-03-002: Signal Formation, Compression, Access, and Analysis (PI: John

Nehrbass, OSU)

The strategic roadmap for SIP is as follows:

FY04-07 project areas for SIP include:

SIP-ATR: SIP/HPC algorithms for ATR — A long-standing and essential task in

many tactical and strategic situations, including both battlefield and surveillance

missions, is detection (Is there something interesting there?), classification (Is it a

missile?) and recognition (Is it a SCUD missile?) of targets in signals from a variety

of sensors such as radar, sonar, IR, and hyperspectral data. To make Automatic

Target Recognition (ATR) a reality, the DoD must develop and test a range of new

and more powerful signal and image processing algorithms that can recognize targets

of interest in very low signal-to-noise ratio (SNR) signals. Further, in many cases, it

is vital to detect targets even when they are camouflaged or hidden (such as tanks

under trees). The development and deployment of more advanced ATR algorithms

will inevitably require the computational power and memory capacity of high

performance computers (HPC), and some of these algorithms will ultimately be

ported to embedded HPC for field application and real-time response. Further, these

algorithms must exploit sensor physics and incorporate adaptive signal processing

methods.17

Strategic FocusAreas

1. New algorithm design

2. Improved data transmission & storage

3. Improved processing throughput & latency

4. Incorporate life cycle software tools

5. Use of COTS

6. Rapid design/deployment

FY02 FY03 FY04 FY05 FY06 FY07

Functional Area: SIP

SIP003

SIP005

SIP-ATR

SIP006

SIP007

SIP017

SIP-HPdC (SIP03-001)

SIP-FCA (SIP03-002)

SIP003SIP005

SIP006

SIP007

SIP-ATR

SIP-ATR

SIP-HPdC (SIP03-001)

SIP-IS

SIP-IE

SIP-IS

SIP-IS

SIP-FCA (SIP03-002)

SIP-FCA (SIP03-002)

SIP-IE

SIP-IE

SIP-FCA (SIP03-002)

SIP-IE: Information Exploitation — One of the most pressing needs in the SIP

community is addressing the challenge of exploiting all the information that is present

in the myriad of signals harvested from deployed DoD sensors. These signal sources

include data such as acoustic, synthetic aperture radar (SAR), hyperspectral, and

visual imagery collected by the DoD. Extracting and transforming the information

that is extant in these rich data sources is essential if the collected data are to be

exploited by our soldiers, officers, security personnel, and senior civilian decision

makers. Applications to be addressed include long- and short-range surveillance,

individual identification, biometrics, voice modification, and sensor webs. These

applications will require that we develop efficient new techniques in distributed

computing, data mining and understanding, cognitive processing, and embedded

computing.

SIP-FCA: Signal and Image data formation, compression, and access — A common

set of fundamental processing steps constitutes the foundation of virtually every SIP

application. Data are 1) collected and formatted for use, 2) compressed for efficient

and/or robust handling, and then 3) organized for access further downstream in the

processing chain. Recent developments in each of these foundational processing steps

offer the DoD an opportunity to reap significant advantages in applications such as

3D SAR, collection and annotation of video surveillance and Unmanned Air Vehicle

(UAV) ATR data, and general sensor-fusion tasks.

SIP-HPdC: High Productivity Computing — The warfighter’s need to confirm the

identity of any target before attack often requires real-time matching to occur on

onboard computer architectures which are often space, power, and cycle-limited.

However, the new generation of petabyte sized databases — which are required for

identification in many complex search spaces — are more efficiently created via

traditional HPC. This “gap” in the two required processing architectures necessitates

innovations in structuring, accessing, and processing data, and the software that is

used in these applications. Fortunately, recent advances in software techniques (XML

and object-oriented and graphical languages) hold promise for 1) optimized

architecture-specific code, 2) maximum code reuse, and 3) efficient incorporation of

technology advancements. Further, life-cycle software has the potential to greatly

increase productivity, shorten development time, and reduce overall costs. Advancing

18

these goals through development of, for example, parallel MATLAB and Simulink

tools while adapting technologies for reuse (Vector/Signal/Image Processing Library

— VSIPL) offer immediate gains for the DoD. Integrated with these tools, Java and

Java Server Pages (JSP) can be employed to portably manipulate data sets across

departments, agencies, and centers. Finally, portable graphical programming

languages, such as the Visual Toolkit (VTK), can be used to provide custom displays

of filtered and compressed information to high-resolution 3D stereographic devices,

PC terminals, secure web ports, or PDAs located on the battlefield.

SIP-IS: Infrastructure Software for SIP — An emerging trend in the SIP community

is the appearance of middleware and middleware standards. High performance

computing (HPC) and High Performance Embedded Computing (HPEC) DoD

applications will significantly benefit from highly efficient and portable

computational middleware for signal and image processing. Open middleware

standards such as VSIPL, CORBA and SOAP, as well as the emergence of powerful

COTS hardware, offer a unique opportunity for the rapid development of easily

maintained HPEC codes that combine portability and flexibility across a number of

applications. Thus the timely transfer of these technologies to the DoD community is

a critical task, and one that is in its infancy. This middleware infrastructure will

support the rapid development and deployment of portable, efficient SIP-critical

applications of immediate benefit to the warfighter.

The SIP UAP includes:

Andy Sullivan (ARL/Adelphi, Govt CTA Lead)

Keith Bromley (SPAWAR)

Rich Linderman (Rome)

Mike Bryant (Wright-Patterson Air Force Base [WPAFB])

Gary Stolovy (ARL/Adelphi)

3.5 Integrated Modeling and Testing (IMT). The top priorities for IMT are:

Application of HPC software tools and techniques with live tests and hardware-

in-the-loop simulations for test and evaluation (T&E)

Tools for collecting, organizing, and combining diverse information sets to

optimally guide T&E decision making

19

Collaboration with CTAs to provide needed discipline-specific expertise to the

T&E community

IMT will concentrate on:

High-fidelity component and process models

Real-time model execution

Integration of information from disparate sources

Test data repositories

Optimal Test Strategy Planning

Building IMT Communities of practice

Training in new software tools and methodologies (Core)

The IMT FY03 project tasks are:

IMT-03-001: Data Repository Infrastructure Evaluation (PI: Tilt Thompkins,

National Center for Supercomputing Applications [NCSA]/University of Illinois

Urbana-Champaign [UIUC])

IMT-03-002: Real-Time Interoperability Protocol Study (PI: Tilt Thompkins, NCSA/

UIUC)

The strategic roadmap for IMT is as follows:

20

Strategic FocusAreas

1. High-fidelity component and process models

2. Real-time model execution

3. Integrating information from disparate sources

3. Test data repositories

4. Optimal Test Strategy Planning

5. Building IMT Communities of practice

FY02 FY03 FY04 FY05 FY06

Functional Area: IMT

IMT002

IMT-IST

IMT-RTVE

IMT-CBTPE

IMT002

IMT-TDMRF

IMT-MBTV

IMT006

IMT-MBTV

IMT-TDMRF

IMT-TDMRF

IMT-RTVE

IMT-CBTPE

IMT-CBTPE

IMT-CBTPE

IMT-CBTPE

IMT-TDMRFIMT001

IMT002

IMT001

FMS001

Future areas of concentration for IMT include:

IMT — RTVE: Real-Time Virtual ExperimentationDeveloping and integrating high-

performance computing techniques and software tools with live tests and hardware-

in-the-loop simulations for test and evaluation of DoD weapons, components, and

subsystems. Achieving high-fidelity representations in virtual and virtual-real

environments is a key to achieving faster, more cost-effect development and

acquisition and will require DoD to develop variable fidelity sensor/scene/target

models, component models, and unique test configurations and facilities. These

requirements will drive the needs for linking and integrate disparate models and

simulations, embedded system-HPC interfaces, and real-time scene rendering.

IMT — MBTV: Model-Based Test Validation: Using verified models to monitor the

performance of new concepts under test by comparing predicted trends of key

parameters with test data. Variances are used to interrupt testing for safety of the test

article and/or facility and to maintain integrity of the test data. Models can be used for

diagnostic analyses to determine anomalies in either the article or instrumentation that

can produce the undesirable variance. Reaching these capabilities will require

development of high capacity processing and data network resources, high-fidelity

component models that can be executed in real-time, and reliable model-data

comparison tools.

IMT — TDMRF: Test Data Management, Retrieval, and Fusion: Providing uniform

integrated access, visualization, and fusion of diverse test and evaluation data from

multiple sources, (computational models, ground tests, and flight tests), to T&E

communities. To provide these capabilities DoD must develop techniques for

automated data archiving, retrieval, and long-term storage; complex data query

processing; links to performance data, documents, system specifications, and test

incident reports; and seamless availability to developers, testers and evaluators.

Achieving these goals will require advanced approaches in networking technologies,

storage area networks, meta-data management and indexing, and distributed data

mining.

IMT — IST: Interoperability and System of Systems Testing: System of Systems

operation is becoming increasingly important as DoD strives to gain revolutionary

new operational capabilities with components (sensors, platforms, weapons) of

21

maturing development potential. To support the development of SoS capabilities the

T&E community must quickly and cost-effectively enable Interoperability among test

ranges, facilities, and simulations; develop tools for verification and validation of

Interoperability; and provide simulation support for distributed real, virtual, and

constitutive simulations around ranges and across geographic areas. Key to providing

these capabilities will be advances in high-speed mobile and fixed networks, and

Interoperability at the M&S level (e.g., HLA).

IMT — CBTPE: Computational-Based Test Planning and Execution: Using verified

model and simulation to: separate critical from non-critical conditions, select

configurations and parameters for test, support a logical buildup approach for

choosing ground or flight-test conditions, and support optimally modifying test plans

as test results accumulate. Achieving these capabilities will require DoD to develop

tools to: define and validate decision metrics; rigorously define the logical and causal

structure of weapon systems and test requirements; and integrate information

resources from theoretical models, simulation results, test experience, and on-going

test results. Providing the computational capacity to satisfactorily define distributions

of model performance and retrieve test experience will severely stress HPC resources.

The IMT UAP includes:

Jere Matty (Arnold Engineering Development Center [AEDC], Govt CTA Lead)

Jeff Highland (ARL)

Guy Williams (Range Commanders Council Modeling and Simulation Sub-Group

[Tri-Service])

Reid Johnson (Range Commanders Council Data and Computer Architecture Sub-

Group [Tri-Service])

3.6 Forces Modeling and Simulation (FMS). The top priorities for FMS are:

To leverage the experience and techniques developed by the HPC community to

improve FMS support to the operational warfighter

Conduct militarily relevant research and transition the technologies to operational

programs

FMS will concentrate on:

Addressing critical user needs

Portability, scalability, responsiveness

22

Strong operational user focus and involvement

Demonstration based results

Cross-functional area projects.

The FMS FY03 project tasks are:

FMS-03-001: Live / Model Data Reduction As A Means Of Populating Constructive

Simulation Datasets (PI: David R. Pratt, SAIC)

FMS-03-002: Joint Warfare System (JWARS) Performance (PI: Ron Painter, CACI)

FMS-03-005: An Analysis Of The Use Of Agent-Based Simulations And Their

Application To Scalable Systems (PI: David R. Pratt, SAIC)

The strategic roadmap for FMS is as follows:

FY04-07 areas of concentration for FMS include:

FMS-UO: User Outreach — FMS is a non-traditional HPC area. For this reason, a

concerted effort needs to be made to reach out to the FMS community to “create the

market” for HPC services and resources and to assist in the transformation of the

community's perceptions.

FMS-TT: Technology Transfer — The HPC community has developed a large

number of algorithms and lessons learned concerning the optimization of parallel

codes. This project area has been set up to aid the transition of experience in the HPC

community to the FMS community via pilot projects that demonstrate the proof of

principle and applicability of the technology.

23

Strategic FocusAreas

1. User Outreach

2. Technology Transfer

3. Technology Exploration

4. Portability

5. Scalability

6. Rapid response / deployment

FY03 FY04 FY05 FY06 FY07

FMS-TE

Functional Area: FMS

FMS Onsite

FMS-UO

FMS-TT

FMS-P

FMS-S

FMS Onsite

FMS-RRD

FMS Onsite

FMS-UO

FMS-TT

FMS-P

FMS-S

FMS Onsite

FMS-RRD

FMS-TE: Technology Exploration — Investigates use of new / novel technologies to

the FMS domain. This effort will be discontinued in favor of more focused

technology area exploration.

FMS-P: Portability — Due to the operational nature of the FMS community, codes

will need to run on commodity deployable hardware in addition to the HPC resources.

This functional area looks into techniques and the application of tools to make such

configuration as seamless and automated as possible.

FMS-S: Scalability — In conjunction with portability, the scalability of codes is a

major concern. The number of entities that can be simulated while maintaining a

fixed time ratio notionally defines scalability. As the systems scale up (or don’t in

many cases) the bottlenecks in the code are exposed. The purpose of this thrust to

leverage the lessons learned with HPC codes to aid in improvements in reducing or

eliminating such bottlenecks.

FMS-RRD: Rapid Response / Deployment — The Air Force’s AETF and Army’s

FCS programs are prime examples of the shift of emphasis from the cold war force

structure to more responsive and “lighter” forces. As a key component of the training

and planning cycle, FMS needs to become more responsive and able to deploy

quickly. In this thrust we examine the ability of leveraging the horsepower of the

HPC resources to provide a rapid “first cut” capability that can later be refined by the

user to suit the operational commitment.

The FMS UAP includes:

Larry Peterson (SPAWAR-Systems Center, Govt CTA Lead)

Steve Gordon (Air Force Agency for Modeling and Simulation [AFAMS])

Mike Macedonia (U.S. Army Simulation, Training, and Instrumentation Command

[STRICOM])

Dave Hoffman (U.S. Army Training and Doctrine Command [TRADOC] Analysis

Center [TRAC]-White Sands Missile Range [WSMR])

Niki Deliman (ERDC / TRAC-Monterey [MTRY])

FMS is in the process of re-examining the membership of the UAP based on the responsiveness

and input from the current members.

3.7 Enabling Technologies (ET). The top priorities for ET are:

Visual Data Mining Research and Applications

24

Development of CTA-specific Visualization Systems

Development of CTA-specific Problem Solving Environments

Automation of Mesh Generation Tools

ET will concentrate on:

Visualization systems that exploit heterogeneous distributed computational resources

and can ingest enormous (10TB) datasets

One or two problem solving environments

Automation in geometry processing for mesh generation

Training in visualization, mesh generation, and data mining tools

The ET FY03 project tasks are:

ET-03-001: EnVis and EnVisU – Distributed High Performance Batchmode

Visualization (PI: Robert Moorhead, Mississippi State University [MSU])

ET-03-002: GGTK (Geometry-Grid Tool Kit) (PI: Bharat Soni, University of

Alabama-Birmingham [UAB])

ET-03-008: Development of an Integrated Simulation Environment (PI: Ralph Noack,

MSU)

ET-03-011: Building Interoperable Portals with Web Services (PI: Mary Thomas,

Texas Advanced Computing Center [TACC], UT-Austin)

The strategic roadmap for ET is as follows:

FY04-07 areas of concentration for ET include:

ET-DV: Distributed Visualization — Once a solution is computed to a CTA problem,

the data is usually remote from those that need to see and analyze the results. To 25

Strategic FocusAreas

1. Distributed Vis

2. Large Scale Vis

3. Visual Data Mining

4. Problem Solving Environments

5. Grid Generation

FY03 FY04 FY05 FY06 FY07

Functional Area: ET

ET001

ET011

ET001

ET008

ET002

ET-DV

ET-LSV

ET-FV

ET-DM

ET-PSE

ET-GG

effectively analyze data, the computational task is often divided into parts so that one

part is carried out at the remote MSRC/ADC/DDC, and the other done on resources

local to the analyst. For example, feature detection and extraction is often best done in

the MSRC/ADC/DDC, while actually mapping to 2D/3D graphical primitives is often

best done on the user's local workstation. The issues in solving this problem start with

balancing the computation demands tempered by the communication bandwidth. The

issues then become where is it best to do what computation, where to put the long-

haul transmission in the visualization pipeline that ingests 4D data and outputs

imagery (after reading the data, after every time step, after constructing 3D graphical

objects, after projecting to 2D graphical objects, after rendering to images, etc.), how

much speed to give up to provide more flexibility to the analyst, etc. Performance is

probably a more critical issue than portability in the end, but due to the continuously

changing collection of resources, portability is probably a significant issue initially.

(Generally resources at MSRCs/ADCs/DDCs are known, but resources at user’s

desktop are not.) Projects that address ingesting, coding, transferring, decoding, and

visualizing enormous (10TB) raw datasets need to be undertaken. Existing

frameworks (e.g., vtk etc.) should be leveraged where possible. Security issues

(Kerberos) must be addressed.

ET-LSV: Large-scale Visualization — Although visualization of high-order entities,

like features and objects, is the ultimate goal, visualization of raw data is often first

required. Analysts are legitimately skeptical of too much automation too soon. To

provide analysts with a sufficient comfort level, they often need to be able to browse

and explore large datasets. They need to be able to do data mining and knowledge

discovery in a labor-intensive way before trusting an automated tool, if for no other

reason than to generate some test cases. ET-LSV is not necessarily interactive

visualization and in fact often is batch-mode visualization. These projects should

produce algorithms, tools, and knowledge bases that allow rapid efficient deployment

of analysis systems. Issues that need to be addressed under ET-LSV to improve the

working environments for DoD users include application-specific

coding/compression schemes that compress the data more or run faster, automated

sizing metrics (to help determine appropriate chunking of data), and visualization

algorithms and user interfaces that help an analyst to better explore and understand

26

massive datasets. User interfaces is an often-neglected area in CTA programs that

could benefit from some of the research done by cognitive scientists and human-

computer interface specialists. User interfaces include issues like how to best

organize data accesses and computations, as well as contextual display issues like text

annotation, time line indicators, and latitude/longitude lines.

ET-FV: Feature-based Visualization — Ultimately it is the features, not the raw data

that the analyst seeks. Where are the (shock, strong, peak) waves, how strong are

they, where are the eddies, how big, how strong, how fast, where is the information

content, where is the target, where are the hot spots, where are the narrow junctions,

etc. Much research has been done on feature detection, feature extraction, and feature

classification. This work needs to be captured in a body of knowledge or library so

that it is useful across CTAs. This will require multidisciplinary projects involving

personnel knowledgeable in the various CTAs. The issues to be addressed in this area

are how to best visualize those features. Developing these visualization methods will

require multidisciplinary projects involving personnel knowledgeable in the various

CTAs and in ET technology. Projects addressing this area that produce software

modules within a standard framework like vtk etc. would be highly desirable.

Exploitation of scripting languages, e.g., tcl/tk, Python, Perl, etc. would help

portability of developed tools.

ET-DM: Data Mining — ET-FV and ET-DM are highly supportive of each other, but

have distinct goals. ET-DM projects will focus on the process of data mining and the

computational techniques and methods for the extraction of useful information from

data. To improve working environments for DoD users in this area, more data mining

techniques need to be developed and the existing techniques need to be tested on

more datasets. Different DM techniques (clustering, trees, neural networks, etc.) work

better on different data sets. A better body of knowledge needs to be developed in

when and how to apply each technique and libraries of techniques need to be

developed. To maximize return on investment, it would be reasonable for some ET-

DM projects to be done in collaboration with some SIP-IE projects, especially those

based on image analysis. To be able to mine data, some idea of what is being sought

is useful. A greater body of knowledge as to what needs to be mined and/or what

knowledge needs to be discovered would be useful. To improve the working

27

environments for DoD users, projects in this area need to involve a domain expert,

not just a mining expert. This area is viewed as a high-risk area, as many projects

have promised much and delivered little.

ET-PSE: Problem Solving Environments — Initial projects in this area should

develop PSEs for some of the most heavily used codes at the MSRCs/ADCs/DDCs.

In particular initial development should be on code-specific portals instead of more

abstract PSEs. Once several application code specific PSEs have been developed (2-3

years), then significant infrastructure development should accelerate, probably as the

computational grid infrastructure becomes available to the DoD user. The technology

underpinnings of portal-based PSEs continue to rapidly evolve. However, in

anticipation of the need to exploit the computational grid infrastructure, DoD

involvement in the Global Grid Forum would be appropriate. Involvement in this

body would ensure the DoD will remain in contact with the larger computing portals

and grids communities and will be able to influence the standards and activities of

this group. Projects in this subarea could be viewed as developing/tracking enabling

technology for enabling technology!

ET-GG: Grid Generation — An important element, and current user need, is the

accurate and rapid processing of geometric models being designed and analyzed. Grid

(or mesh) generation (GG) technology bridges the gap between digital geometry

models and computational simulations for engineering analyses. Current GG

techniques are being developed for incorporation in large production, or commercial,

software systems. Such an approach limits the use of emerging algorithms from

outside developers. The work in GG needs to focus on the use of components and

standard interfaces to be used among larger software systems. The underlying

geometry is moving from CAD/CAM based to solid models. Adaptive and moving

meshes need to be developed, with the goal being dynamic meshes. The mesh

technology needs to support multiple disciplines (multi-disciplinary optimization).

The underlying coding needs to move to object-oriented technology. The modules or

components need to be thread-safe and operate in parallel and distributed computing

environments. The creation of meshes needs to become more automatic and exploit

some intelligence. In particular,

28

Dynamic solution-adaptive unstructured mesh capability for viscous dominated

flows

Temporally/spatially deforming geometry/ mesh capability for multidisciplinary

analysis

Parametric, automatic, and intelligent complex structured, unstructured, and

hybrid mesh generation for multidisciplinary applications

Parallel unstructured/generalized mesh generation with dynamic load balancing

Conservative interpolation techniques for multidisciplinary interactions

Robust mesh generation methods for multidisciplinary applications, including

geo-spatial processes which may not include man-made models

The ET UAP includes:

Alan Walcraft (NRL/SSC)

John E. West (ERDC MSRC)

Jerry Clarke (ARL MSRC)

Frank Witzeman (AFRL)

Dan Kedziorek (Army Tank Command [TACOM])

Aram Kevorkian (SPAWAR Systems Center San Diego [SSCSD])

Pete Gruzinskas (NAVO MSRC)

3.8 Computational Fluid Dynamics (CFD). The top priorities for CFD are:

Improve accuracy, robustness, confidence, throughput, and cost of performing

complex CFD simulations associated with DoD missions

Enhance validation and verification of CFD tools to reduce cycle time and improve

design performance

In Year Two, CFD will concentrate on:

Continuous capture and transition of best practices via training, software, hardware

and process improvements (Core)

Collaborate with MSRC/ADC/DDC users, CHSSI developers, Challenge Project

executors and national CFD community (Core)

Development/enhancement of tools and technology to address parametric geometry

preparation, dynamic, adaptive and parallel mesh generation, and automated feature

detection with improved data standards, interoperability and Problem Solving

Environments

29

Methods for multi-physics coupling and highly complex physics

Algorithms for simulation of time varying geometries

The CFD FY03 project tasks are:

CFD-03-001: Validation of Hybrid RANS/LES Turbulence Models for Unsteady

Flows (PI: Robert Nichols, MSU)

CFD-03-002: 6DOF Library and Standard Interface (PI: Nathan Prewitt, MSU)

The strategic roadmap for CFD is as follows:

FY04-07 areas of concentration for CFD include:

Multi-Physics Analysis Tool Kit (FAs impacted: CFD, CSM, CWO, CEA) — ET

GGTK: Geometry-Grid Tool Kit (FAs impacted: ET, CFD, CSM, CWO, CEA, EQM,

CCM) Enhancements and Mesh adaptation — ET

V&V Standards and Methodologies — CFD

Highly Complex Turbulent and Reacting Flow Techniques — CFD

Integrated Simulation Environment — ET

Simulation Based Acquisition and Design — CFD

The CFD UAP includes:

Robert Meakin (Army, Govt CTA Lead)

Frank Witzeman (AFRL/VAAC [AFRL CFD Research Branch])

Tim Madden (AFRL/DEC [AFRL/Kirtland Directed Energy Center])

Greg Power (AEDC)

Charlie Berger (ERDC)

William Sandberg (NRL)30

Strategic FocusAreas

1. Core Activities

2. Time Varying Geometries & Multi-Physics

3. Knowledge Centers

4. WEB-Based Training, Technology Tools & PSEs

5. Verification & Validation

6. Solution Adaptive Meshing

FY02 FY03 FY04 FY05 FY06

Functional Area: CFD

Overset Technology Center

Mesh Redistribution & Refinement

Training & Workshops

Technology Transfer & Outreach

CFD Core

V&V Strategies & Standards

6 DoF Library

CFD Learning Tool & WEB-based Courses

Simulation Based Acquisition & Design Methods

Integrated Simulation Environment

Automated Feature Detection & Sensors

GGTK : Geometry-Grid Tool Kit

Collaboration & Partnerships: Challenge & CHSSI

Generalized/Unstructured Technology Center

CFD003

ET022

CFD Core Multi-Physics Coupling

Complex Physics: LES, Chemistry, Multi-phase,…

CFD002

ET008

ET019 ET002

CFD001

Susan Polsky (Naval Air Warfare Center [NAVAIR])

Jubraj Sahu (ARL)

3.9 Computational Structural Mechanics (CSM). The top priorities for CSM are:

Increase reliability and efficiency of CSM simulations by incorporating improved

technologies into legacy codes

Extract useful information from those simulations

In Year Two, CSM will concentrate on:

Development of technologies for level-2 code coupling for multi-physics applications

Improved parallel partitioning strategies for CSM applications

Development of formalism and algorithms to enhance verification and validation of

computer simulations

Development of enhanced algorithms and codes for parallel adaptive mesh

refinements and error control

New high-fidelity algorithms

Implementation of damage and failure models for multi-component and non-isotropic

materials

Improvements in contact-impact algorithms for highly dynamic environments

Improved mesh generation strategies for CSM applications

The CSM FY03 project tasks are:

CSM-03-001: Code Coupling Technologies for Multi-Physics Applications (PI:

Graham F. Carey, TICAM, UT-Austin)

CSM-03-003: New tools for Verification of Computational Models for DoD CSM

Applications (PI: J. Tinsley Oden, TICAM, UT-Austin)

31

The strategic roadmap for CSM is as follows:

FY04-07 areas of concentration for CSM include:

CSM-CC: Code Coupling Technologies for Multi-Physics Applications — Over the

last two decades, significant investments have been made in developing effective

simulation codes for important DoD applications. The majority of these codes have

been application-specific, focusing on simulations of particular physical phenomena

of interest. In a growing list of critical applications, there is a need for modeling

multiple types of physical events. A suite of algorithms and codes needs to be

developed that will permit code coupling in a parallel computing environment for two

representative CSM simulation tools. Initial work will focus on coupling CTH with

DYNA. These simulators should allow the modeling of events such as the evolution

of dynamic pressures and blast loads impinging on a deformable structure undergoing

very large deformations. Mesh interfacing, mesh and time scaling, projection methods

to map results obtained in one model on data files of another, load-balancing, and

other issues will be investigated. Special algorithms and solvers that may provide

better management of multiphysics and multiscale effects than direct use of solvers

currently existing in the particular codes will be identified. General principles for

code coupling will also be established.

CSM-PGM: Parallel Grid Partitioning and Management — A central problem to

parallel CSM applications and to many other DoD large scale applications areas is

partitioning of an initial mesh across processors and dynamic load balancing of the

partition as the solution and mesh evolve. This problem is particularly important 32

1. Code Coupling for Multi-Physics1,2

2. Parallel Grid Management3

3. New Tools for Model Verification1,2

4. Adaptive Mesh Technology3

5. High-Fidelity Algorithms3

6. Damage models for Composites2

7. Contact-Impact Modeling3

8. Grid Generation3

FY03 FY04 FY05 FY06

Functional Area: CSM

CSM001/FY03

CSM004/FY03

CSM003/FY03

CSM005/FY03

CSM-PGM

CSM-CC

CSM002/FY03

CSM-HFMP

CSM-AMR

CSM-VVT

CSM-DCI

CSM006/FY03 CSM-IDM

CSM007/FY03

CSM-GGCSM008/FY03

1=funded for FY032=proposal solicited for FY033=white paper solicited for FY03

TrackingNumber

Strategic FocusArea

when re-meshing and adaptive mesh refinement (AMR) are involved. Our proposed

work will address some of the underlying issues related to partitioning for

computation and communication load balancing. In particular, we will develop, test

and deploy new metrics for balancing the partition. We will extend the algorithms in

existing partitioning software and use them to carry out tests of the metrics for

unstructured meshes and dynamic AMR meshes. These ideas will also be

demonstrated and performance studies made for representative DoD CSM test cases

and for meshes arising from other applications such as CFD and CEN.

CSM-VVT: New Tools for Verification of Computational Models for DoD CSM

Applications — The reliability of computer simulations in nonlinear solid mechanics

(indeed, in computational science in general) depends upon how accurately the model

used to depict the physical events of interest has been solved. In modern literature,

this subject is referred to as model verification; the discipline is concerned with the

question: has the model used as a basis for the computation been solved correctly?

The solution is thus to determine reliable, computable error bounds on arbitrary

physical variables, to determine estimates of local approximation error, and, if

needed, to derive and implement corresponding adaptive strategies to reduce or

otherwise control the error. One approach that has proved to be effective that was

developed by the proposers is the so-called GOALS method (Goal Oriented Adaptive

Local Solver) in which special influence functions are computed which can be shown

to characterize errors in local quantities such as pointwise stress, displacement,

average stress, pressures, pointwise temperatures, etc. The objective of the proposed

effort is to extend these ideas to general nonlinear problems in CSM of interest to

DoD users of CSM codes so as to provide them with a useful set of tools for model

verification.

CSM-AMR: Adaptive Mesh Technology for Impact and Penetration Analysis —

Realistic and accurate simulation of events involving impact and penetration is

critical to a wide rage of DoD applications. Adaptive mesh technology provides a

means for control and reduction of numerical error; as such, it is an important method

for implementation into this class of simulation tools, including Eulerian, Lagrangian

and arbitrary Eulerian-Lagrangian (ALE) impact codes. Example application codes in

this area include CTH, DYNA, PRONTO, EPIC, ALE3D and ALEGRA. Successful

33

use of adaptive mesh technology requires the development of estimators for

numerical error that are reliable and computationally efficient, as well as adaptive

strategies that are flexible and can easily be integrated into existing legacy codes.

This project deals with the development, implementation, and testing of sophisticated

error estimates for CSM applications and the parallel adaptive strategies for their

implementation in adaptive meshing.

CSM-HFMP: High Fidelity Algorithms for Coupled Multi-Physics CSM

Applications — In simulations of convection, reaction, and diffusion in multi-physics

CSM applications, a multitude of numerical problems are encountered when

conventional solvers are used. Among these are: non-conservative local

approximations, which result in pollution of the solution accuracy in cases in which

coefficients in one physics model depend upon numerical solutions of another model,

low accuracy, non-physical dissipations, poor scalability, and data structures too

inflexible to readily admit adaptive meshing. The goal of this project is to develop,

analyze, and implement algorithms that are high-order accurate, element-wise

conservative, readily parallelizable, allow easy coupling of multi-physics, and are

built on simple data structures and to demonstrate their effectiveness on significant

CSM applications.

CSM-IDM: Impact Damage Models in Composite Structures — The need to improve

the mobility and survivability of vehicles and structures of interest to the DoD has led

to an expansive use of composite materials in many applications, including aircraft

components, ship and submarine hulls, and land vehicles such as tanks and personnel

carriers. The response of these materials to loads that are highly evolving and large in

magnitude is still an area of intense research, particularly in the area of modeling

damage and failure of these materials. To date, many state-of-the-art simulation tools

for transient structural dynamics use damage and failure descriptions for composites

that are decades old and highly heuristic in nature (for example, the Tsai-Hill model

for failure). These models do not capture the basic phenomenology of the failure

process in composite structures, such as fiber cracking and splintering, matrix void

growth and coalescence, and delamination. As a result, simulations performed using

these models are not considered to be very reliable in the DoD research community.

34

CSM-DCI: Advances in Methodologies for Treatment of Dynamic Contact-Impact —

Accurate and efficient modeling of contact-impact conditions is crucial to many

applications of interest to the DoD, including fluid/structure and blast/structure

interaction, damage to vehicles and structures, impact and penetration of armor, and

hydrodynamic ram. While there have been notable advances made in the last five

years in search and return algorithms, automatic contact, and parallel implementation,

to a large extent the physics and algorithms responsible for contact enforcement have

remained unchanged; there have been very few advances in this area. In Eulerian

CSM, for example, contact-impact enforcement is still achieved using an ad-hoc

combination of mixed-cell thermodynamic and constitutive model algorithms, which

may have little or nothing to do with the physics of contacting bodies.

CSM-GG: Resolving Mesh Generation Bottlenecks and Issues — Mesh generation

remains a major obstacle to DoD simulations. This proposed effort is directed

towards resolving some of the key problems and thereby materially improving the

DoD capability. Despite significant progress in this area, the continuing increase in

geometrical complexity, the need for more accurate modeling of DoD problems and

systems with many multimaterial components of different shape and size, and the

need to carry meshing into the parallel arena limit and frustrate the DoD user. It is

currently the case that meshing a complex weapons problem or similar DoD

application is not fully automated. It frequently takes several person-months to

construct a mesh. Often the subsequent analysis step is a matter of a few days of

computing, at which time the mesh has to be assessed and modified. A secondary

issue is the quality of the meshes that are generated in this process. In CSM,

hexahedral 3D elements are the proven technology, but completing a mesh with these

elements is problematic: the mesh may have “voids” that the generator cannot close,

and the cells may be ill-shaped to the point that the reliability of the simulation is

compromised. In Lagrangian CSM calculations, as the mesh deforms, the elements

become ill-shaped, necessitating local re-meshings. These are the key issues that we

will address. They are targeted to complement the major mesh generation work

ongoing at the DoD and DOE labs and at other universities.

The CSM UAP includes:

Photios Papados (ARL)

35

James Baylot (ERDC)

Alexandra Landsberg (Naval Surface Warfare Center [NSWC])

Anthony Palagatto (Air Force Institute of Technology [AFIT])

Stephen Schraml (ARL)

Mike Giltrud (Defense Threat Reduction Agency [DTRA], Govt CTA lead)

3.10 On-Line Knowledge Center (OKC). The top functionalities need in the OKC for DoD

users are:

Provide a unified way to store and access knowledge for PET and DoD HPCMP users

Administrative and Research Portal for training (registration, assessment and

curricula material), publications, announcements, CTA specific information, HPCC

resources, PET Projects including informational material and rich suite of user

generated input

The critical features of the OKC are:

Allow distributed knowledge update with efficient centralized approval

Provide robust security mechanisms and high-performance servers

Allow common look and feel with component model for all material

Information stored persistently in Oracle databases

Synergy with Grid Forum, HPCMO, community and commercial activities

Support e-mail or form (wizard)-based input and dynamic XML specified schema for

all data and knowledge

Support multi-media, HTML, XML and CGI/Web service based data

The chart below describes OKC efforts by Component. The intent is for there to be a separate

portal for each of the four Components that addresses the FAs managed by that Component. That

portal could reside on the web space at the center, or it could all be centralized at ERDC. Each

FAPOC or a designated set of on-site personnel will maintain the information content for the

respective FAs. HPCMO will supply information on general outreach and the PET program. The

operation of the full OKC will be accomplished by ERDC and by the MOS technologists

resident there, assisted by technologists at the other three Component MSRCs. Technology

development will be accomplished by and at Indiana University. Advice and consent from the

PET management teams (both Government and contractor) and any outside advisory groups will

also be considered.

36

There were no OKC project tasks in FY03.

The strategic roadmap for OKC is as follows:

FY04-07 areas of concentration for OKC include:

OKC-Tech: Ongoing deployment of key OKC Technologies:

We expect all of these to be extensions and customizations of technology

developed and supported for large-scale applications outside PET

Current OKC built on Apache Jakarta technology linked to commercial database

and message service systems

Core technology including security and performance

Component-based Distributed Authoring and Update

37

The OKC Component Efforts

TechnologyIU

OperationsERDC

FA@Comp3 FA@

Comp 1

FA@Comp

2FA@Comp 4

CFDCSMOKCEOTC------EQMCWOCE

FMS/C4IIMTSIPET------CCMCEACENCDLT

AdvisoryCommittee

HPCMP/PETManagement

1. Technologies

2. Update Content

3. CTA Specific OKC

4. Other OKC Applications

5. Integration with Computational Science Portals

6. Data-mining and Semantic Web like capability for a “brilliant” OKC

7. Outreach

FY02 FY03 FY04 FY05 FY06

Functional Area: OKC

OKC001

OKC002

OKC002

OKC001

OKC001

OKC002

OKC-Add Edit Pages

OKC-Tech EnhancementOKC- Initial Tech

OKC-Tech EnhancementOKC- Initial Tech

OKC-initially for CE and TrainingOKC for CTA

OKC-initially for CE and TrainingOKC for CTA

OKC -DSL

OKC-Outreach

OKC-SW

OKC- CompOKC- Comp

Activity Areas

Content Management with rich domain specific standards such as IMS/ADL for

training

Workflow for OKC knowledge updates including reporting and approval

Multi-media support

Search of structured (XML) and unstructured data

Support for Web Service and older (CGI, Servlet) dynamic resources

OKC-Pages: Ongoing update of site content

OKC-FA: Customization of FA Specific Knowledge Centers (e.g. support RIB, ADL

Training and each CTA …)

OKC-DSL: Support of and identification of other applications of the OKC approach

such as online scientific libraries

OKC-Comp: Integration with PET computational portals and scientific data

repositories

OKC-SW: “The Semantic Web for OKC” or support of data-mining, Intelligent

Agents, and “digital brilliance” from enhanced meta-data

OKC-Outreach: Training on use of OKC and Requirement gathering

The OKC UAP includes:

Formal Advisory Panel:

Bob Athow as Government PET lead for ERDC Component

The CPOC and 1 PET Technologist for each component (8 members total)

Representatives of CDLT and EOTC

Other representatives to be added (HPCMO, DCs …)

Current Advice Structure

Close working relation with ERDC

Iterating prototypes with FAPOCs and PET technologists

3.11 Education, Outreach and Training Coordination (EOTC). The top priorities for EOTC

are to:

Provide training across DoD in PET areas, when and where needed

Impact the future researchers in DoD through educational partnerships

Integrate participants from MSIs into mainstream research

In Year Two, EOTC will concentrate on:

Streamlining training coordination through OKC

38

Setting up persistent analysis of user needs through Requirements Survey results,

conference BOFs, user meetings, and special activities

Increasing graduate level student PET participation

Implementing an effective and streamlined Summer Intern Program

Implementing a Visiting Faculty Program for MSIs

Laying the foundation where necessary for full MSI participation in PET

Encouraging MSI partnerships with DoD users and other academic partners

Gaining visibility at the K-12 level

The strategic roadmap for EOTC is as follows:

The EOTC UAP includes:

Margo Frommeyer, NRL/Stennis

Jeanie Osburn, NRL/DC

Matt Grismer, AFRL/WPAFB

Laura Bobo, AEDC

Dean Hampton, ERDC

3.12 Multidisciplinary and Cross-Component Roadmaps. MOS believes there are several

key elements in developing plans for multidisciplinary activities. First, the members of the

academic teams need to consider the Common Software Support Initiative (CHSSI) portfolios,

which are the HPCMP multidisciplinary software projects. There are eight such portfolios. They

are listed here along with a preliminary set of possible cross-FA relationships:

Hyperspectral Image Exploitation (HIE) — SIP, ET, FMS, IMT

Electronic Battlefield Environment (EBE) — CEA, SIP, FMS39

Strategic Focus Areas

1. Training coordination

2. User Needs Analysis

3. MSI Outreach and Participation

4. Graduate level participation

5. Summer Intern Program

FY02 FY03 FY04 FY05 FY06

Functional Area: EOTC

Central, automated registration in OKC

Faculty Visits for 2 MSIs

JSU Summer Institute

User Requirements Data Analysis

Expand Program as Funds allowCentralized Program for MOS Sites

CAU project

Summer Institutes at all participating MSIs

Implement Faculty Visits for all participating MSIs

All MSIs participate in project activities

Persistent Feedback from meetings, conferences

Central Coordination

Include graduate students into

Intern ProgramIntegrate graduate students into various PET activities

JSU and CSUSummer Institutes

Core to projecttransition for MSIs

Centralized HPCMP Intern

Program

Searchable database of program widetraining related informatio

High-Fidelity Simulation of Littoral Environments (SLE) — CWO, CFD, EQM, CE

Weapon-Target Interactions (WTI) — CSM, CFD, CCM

Materials By Design (MBD) — CCM, CSM, CEN

System of Systems Simulation (SOS) — FMS, CEA, SIP, ET

Sensor/Scene Processing and Generation (SPG) — SIP, IMT, CCM

Chemical and Biological Defense (CBD) — CCM, SIP, CFD

Second, HPCMO is sponsoring a total of 39 Challenge Projects in FY2003, 28 of which are

carryover multi-year projects. Almost all of these are highly multidisciplinary in nature. So one

important starting point for PET to consider is the combination of CHSSI and Challenge projects

already being sponsored by DoD.

Third, MOS plans to take the opportunity at the February 2003 Technical Review meeting to

incorporate the ideas from CHSSI, Challenge projects and FAPOCs to develop one or more

white papers for proposed FY04 projects. We intend to ask the PET COR, who is also

responsible for CHSSI, to make a short presentation on both the current CHSSI projects as well

as the plans for FY04. This will generate additional ideas for cross-FA efforts.

4.0. Summary. This is the Year Two version of the TMP, in which MOS has captured the

visions from multiple parties for the future of PET by FA. As we plan for Year Three and

beyond, the potential for PET to make significant new technical contributions to the DoD is high.

40