www.vacet.org

Gunther H. WeberHank Childs, Kathleen Bonnell, Jeremy Meredith,

Mark Miller, Brad Whitlock, E. Wes Bethel

Gunther H. WeberHank Childs, Kathleen Bonnell, Jeremy Meredith,

Mark Miller, Brad Whitlock, E. Wes Bethel

Production Visualization Tools for Adaptive Mesh Refinement DataProduction Visualization Tools for Adaptive Mesh Refinement Data

Mathematics

• PI: Phil Colella (LBNL), SciDAC Applied Partial Differential Equations Center

• Accomplishment(s)– Software engineering to “bring product to

market”. Performance improvements, interface enhancements, file readers, visual data exploration techniques.

• Science Impact– Direct cost savings: APDEC no longer uses

its own resources (e.g., FTEs) to develop, maintain, and support AMR visualization software.

– Ability to perform AMR visualization on large, time-varying data, and using parallel platforms.

– Benefits propagate to all APDEC stakeholders.

• PI: Phil Colella (LBNL), SciDAC Applied Partial Differential Equations Center

• Accomplishment(s)– Software engineering to “bring product to

market”. Performance improvements, interface enhancements, file readers, visual data exploration techniques.

• Science Impact– Direct cost savings: APDEC no longer uses

its own resources (e.g., FTEs) to develop, maintain, and support AMR visualization software.

– Ability to perform AMR visualization on large, time-varying data, and using parallel platforms.

– Benefits propagate to all APDEC stakeholders.

Eliminate Need for Custom AMR Vis Tools

3

• Increasing use of adaptive mesh refinement (AMR) simulations

• Various DOE groups develop and use AMR codes – Differential Equations Center for Technologies SciDAC– LBNL Center for Computational Sciences and Engineering

• Mainstream visualization tools lack AMR support➔AMR code development team invest significant effort

in development of their own tools➔ “Undesired” effort that consumes/diverts resources➔VACET Goal: Provide replacement for custom tools

• Increasing use of adaptive mesh refinement (AMR) simulations

• Various DOE groups develop and use AMR codes – Differential Equations Center for Technologies SciDAC– LBNL Center for Computational Sciences and Engineering

• Mainstream visualization tools lack AMR support➔AMR code development team invest significant effort

in development of their own tools➔ “Undesired” effort that consumes/diverts resources➔VACET Goal: Provide replacement for custom tools

4

Basis of Our Efforts: VisIt

• Richly featured visualization and analysis tool for large data sets

• Data-parallel client server model, distribution on per patch-basis

• Use of meta-data / contracts to reduce amount of processed data

• Built for five use cases:– Data exploration– Visual debugging– Quantitative analysis– Presentation graphics– Comparative analysis

Ideal basis for specialized AMR visualization tool replacement

• Richly featured visualization and analysis tool for large data sets

• Data-parallel client server model, distribution on per patch-basis

• Use of meta-data / contracts to reduce amount of processed data

• Built for five use cases:– Data exploration– Visual debugging– Quantitative analysis– Presentation graphics– Comparative analysis

Ideal basis for specialized AMR visualization tool replacement

[Hydrogen flame, LBNL CCSE]

[Radial velocity in supernova simulation, LBNL CCSE/Stony Brook University]

5

VisIt Already Handles AMR Data

• Supported as “first-class” data type• Handled via “ghost-cells”: Coarse cells that are

refined are marked “ghost” in the lower level• Work on rectilinear grids and skip ghost cells or

“remove” results produced in ghost cells later on

• Supported as “first-class” data type• Handled via “ghost-cells”: Coarse cells that are

refined are marked “ghost” in the lower level• Work on rectilinear grids and skip ghost cells or

“remove” results produced in ghost cells later on

Transitioning ANAG/APDEC SciDAC to VisIt

• Customer project manager serves as “interface” between APDEC and VACET

• Interact constantly with group members to assess changing needs

• Implement missing capabilities in VisIt (e.g., visual spreadsheets)

• Ease transition to new tool (e.g., custom macros)• Help in using new capabilities not available in

ChomboVis (e.g., expressions, animations)• Implement additional new capabilities required

(e.g., mapped grids, crack-free isosurfaces)

• Customer project manager serves as “interface” between APDEC and VACET

• Interact constantly with group members to assess changing needs

• Implement missing capabilities in VisIt (e.g., visual spreadsheets)

• Ease transition to new tool (e.g., custom macros)• Help in using new capabilities not available in

ChomboVis (e.g., expressions, animations)• Implement additional new capabilities required

(e.g., mapped grids, crack-free isosurfaces)www.vacet.org

7

Visual Spreadsheets

• Significance:– Essential debugging tool for code development teams– Support inspecting values in individual cells– Available in ChomboVis (ANAG) and AMRVis (CCSE)

• Development:– Implemented ChomboVis version of visual spreadsheets– Linked it to VisIt’s “pick” capability– Mark picked points in 2D/3D visualization view (existing

VisIt capability) and spreadsheet to indicate correlation

• Significance:– Essential debugging tool for code development teams– Support inspecting values in individual cells– Available in ChomboVis (ANAG) and AMRVis (CCSE)

• Development:– Implemented ChomboVis version of visual spreadsheets– Linked it to VisIt’s “pick” capability– Mark picked points in 2D/3D visualization view (existing

VisIt capability) and spreadsheet to indicate correlation

8

Visual Spreadsheets

9

Mapped Grids • Important, e.g., in climate simulations

• Implementation simplified by VisIt capabilities– conn_cmfe– Macros (added for APDEC)

• No existing file format -> Workaround with two files, since Chombo files either node or cell centered

• VisIt macros tying it all together (e.g., AddPlot callback)

• Important, e.g., in climate simulations

• Implementation simplified by VisIt capabilities– conn_cmfe– Macros (added for APDEC)

• No existing file format -> Workaround with two files, since Chombo files either node or cell centered

• VisIt macros tying it all together (e.g., AddPlot callback)

10

Mapped Grids – Additional Images

First Success: VisIt Replaces ChomboVis!

www.vacet.org

VisIt At CCSE

• Some AMRVis capabilites more difficult to implement in VisIt (such as their “flavor” of Spreadsheets)

• Due to limited resources: VisIt not a complete AMRVis substitute, yet

• BUT:– Facilitate VisIt adoption via postdocs in group and

collaborators– VisIt established as important tool in group, e.g., many

presentation/poster/publication images were generated with VisIt

• Some AMRVis capabilites more difficult to implement in VisIt (such as their “flavor” of Spreadsheets)

• Due to limited resources: VisIt not a complete AMRVis substitute, yet

• BUT:– Facilitate VisIt adoption via postdocs in group and

collaborators– VisIt established as important tool in group, e.g., many

presentation/poster/publication images were generated with VisIt

www.vacet.org

13

Lots of “grunt” work “behind the scenes”

• Macros to ease transition of Applied Numerical Algorithms Group to VisIt (one click for common operations)

• Show only root level, show only explicit materials• Invoke VisIt from gdb• Performance enhancements, specifically for rendering patch

boundaries• Ability to select data using logical indices• Basic support for embedded boundaries• Deployment of VisIt development version to ANAG

machines• Mouse wheel support• Automatic expressions in Chombo files• Bind clip and slice operator via plane tool• Countless bug fixes• … much more basic infrastructure to support these functions

• Macros to ease transition of Applied Numerical Algorithms Group to VisIt (one click for common operations)

• Show only root level, show only explicit materials• Invoke VisIt from gdb• Performance enhancements, specifically for rendering patch

boundaries• Ability to select data using logical indices• Basic support for embedded boundaries• Deployment of VisIt development version to ANAG

machines• Mouse wheel support• Automatic expressions in Chombo files• Bind clip and slice operator via plane tool• Countless bug fixes• … much more basic infrastructure to support these functions

14

Ongoing Research: AMR-aware Streamlines

Stay in Level 0 “Descend” into AMR hierarchy

Recent Research: Crack-free Isosurfaces

• Multiple requests from science stakeholders for crack-free isosurfaces

• Needed for analysis• Started integration in

VisIt based on prior work• Research focus:

– Efficient parallel implementation

• Multiple requests from science stakeholders for crack-free isosurfaces

• Needed for analysis• Started integration in

VisIt based on prior work• Research focus:

– Efficient parallel implementation

www.vacet.org

Science Impact

• Astrophysics/cosmology– ETH Zürich (Chombo/CHARM)– Stony Brook (BoxLib/Maestro)

• Reactive Flow/Combustion– CCSE (BoxLib)

• Particle acclerators– Berkeley Lab Laser Accelerator (Chombo)

• Porous flow/carbon sequestration– Energy Frontier Research Center for Nanoscale Control of Geologic CO2 (Chombo)– CCSE (BoxLib)

• Climate– BISICLES (Chombo)

• Fusion Science– Edge Simulation Laboratory (Chombo)– Princeton Plasma Physics Laboratory (Chombo)

• Aerospace Engineering– NASA (Chombo)

• Land Water Simulations– River, Estuary and Land Model (Chombo)

• Astrophysics/cosmology– ETH Zürich (Chombo/CHARM)– Stony Brook (BoxLib/Maestro)

• Reactive Flow/Combustion– CCSE (BoxLib)

• Particle acclerators– Berkeley Lab Laser Accelerator (Chombo)

• Porous flow/carbon sequestration– Energy Frontier Research Center for Nanoscale Control of Geologic CO2 (Chombo)– CCSE (BoxLib)

• Climate– BISICLES (Chombo)

• Fusion Science– Edge Simulation Laboratory (Chombo)– Princeton Plasma Physics Laboratory (Chombo)

• Aerospace Engineering– NASA (Chombo)

• Land Water Simulations– River, Estuary and Land Model (Chombo)

www.vacet.org

17

Future Work (1/2)

• Will continue to adapt VisIt to ANAG/APDEC’s and CCSE needs

• Further user interface changes– Important: Users need to make efficient use of their time– Substantial time investment necessary to get things right

• Functionality improvements– Improved ghost data support / generation– Native Precision

• Maintenance of software infrastructure

• Will continue to adapt VisIt to ANAG/APDEC’s and CCSE needs

• Further user interface changes– Important: Users need to make efficient use of their time– Substantial time investment necessary to get things right

• Functionality improvements– Improved ghost data support / generation– Native Precision

• Maintenance of software infrastructure

18

Future Work (2/2)

• Invest in critical, missing visualization algorithms– Improved streamline generation

– Crack-free isosurfaces

– Improved embedded boundary support, …

• Current list of desired functions likely to fully utilize available VACET resources

• Anticipate further function requests as VisIt becomes the main tool for APDEC and its collaborators

• CCSE requirements likely to add significant additional burden

• Invest in critical, missing visualization algorithms– Improved streamline generation

– Crack-free isosurfaces

– Improved embedded boundary support, …

• Current list of desired functions likely to fully utilize available VACET resources

• Anticipate further function requests as VisIt becomes the main tool for APDEC and its collaborators

• CCSE requirements likely to add significant additional burden