flash workshop hamburger sternwarte , university of hamburg, feb 15 – feb 16, 2012

44
FLASH Workshop Hamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012 A Solution Accurate, Efficient and Stable Unsplit Staggered Mesh MHD Solver in FLASH Dongwook Lee University of Chicago The Flash Center for Computational Science

Upload: amos

Post on 25-Feb-2016

32 views

Category:

Documents


2 download

DESCRIPTION

The Flash Center for Computational Science. A Solution Accurate, Efficient and Stable Unsplit Staggered Mesh MHD Solver in FLASH. Dongwook Lee University of Chicago. FLASH Workshop Hamburger Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012. Outline. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

A Solution Accurate, Efficient and Stable Unsplit Staggered Mesh MHD Solver in FLASH

Dongwook Lee

University of Chicago

The Flash Center for Computational Science

Page 2: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Outline

Split vs. unsplit formulations

Unsplit solvers in FLASH (UHD & USM) CFL stability (reduced or full?)

Reduced/Full corner-transport-upwind (CTU) for 3D

Divergence-free magnetic fields for USM-MHD constrained-transport (CT)

Verifications, convergence, performance Runtime parameters

Summary

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 3: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 1

Dimensionally Split vs. Unsplit???

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 4: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 1

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Single-mode Rayleigh-Taylor Instability Top figures:

Dimensionally split using PLM, PPM+old limiter, PPM+new limiter

high-wavenumber instabilities grow Bottom figures:

Dimensionally unsplit using PLM, PPM+old limiter, PPM+new limiter

high-wavenumber instabilities suppressed the split solvers experience high compressions and expansions in subsequent directional sweeps where there is a local high strain rate Almgren et al, ApJ, 715, 2010

Page 5: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 1

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Weakly magnetized 2D field loop Gardiner and Stone 2005 (JCP); Lee and Deane 2009 (JCP)

Page 6: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 1

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

8-wave split MHD scheme (Powell et al. 1999) at t=2.0

Unsplit staggered mesh MHD scheme (Lee and Deane, 2009) at t=2.0

Page 7: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 1

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

What is wrong with the split formulation for MHD? In the split formulation, you cannot correctly include terms proportional to

Gardiner and Stone (2005) Dynamics of in-plane magnetic fields in x and y directions are ruined from erroneous growth of magnetic field in z direction:

Page 8: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 2

Unsplit Hydro/MHD Solvers & Algorithms

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 9: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Hydro Unit in FLASH

Hydro_Unsplit

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 10: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Unsplit Staggered Mesh (USM) MHD Solver

Shock-capturing high-order Godunov Riemann solver (Lee & Deane, JCP, 2009; Lee 2012, to be submitted)

Finite volume method New data reconstruction-evolution algorithm for high-order accuracy Adaptive mesh refinement, uniform grid 1st order Godunov, 2nd order MUSCL-Hancock, 3rd order PPM, 5th Order WENO Approximate Riemann solvers: Roe, HLL, HLLC, HLLD, Marquina, modified

Marquina, Local Lax-Friedrichs Monotonicity preserving upwind PPM slope limiter for MHD (Lee, 2010,

Astronum) Divergence of magnetic fields is numerically controlled on a staggered grid,

using a constrained transport (CT) method (Evans & Hawley, 1998) Wide ranges of plasma flows Full Courant stability limit (CFL ~ 1 for 3D)

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 11: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Unsplit Formulations

Take a deep breath!

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 12: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

MHD Governing Equations

MHD system of equations:

This can be written in a simple matrix form:

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 13: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

MHD Governing Equations

Conservative variables and fluxes:

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 14: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

A primitive form:

where the coefficient matrix is

Linearized System

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 15: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Corner Transport Upwind (CTU)

Linear systemin 3D

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 16: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Corner Transport Upwind (CTU)

Linear systemin 3D

Normal predictor Transverse corrector

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 17: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Corner Transport Upwind (CTU)

Linear systemin 3D

Normal predictor Transverse corrector

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Traditional approach (Colella 1990; Saltzman 1994)

Characteristic tracing for the normal predictor

Subsequent calls to Riemann solvers for transverse corrector

Page 18: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Corner Transport Upwind (CTU)

Linear systemin 3D

Normal predictor Transverse corrector

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Traditional approach (Colella 1990; Saltzman 1994)

Characteristic tracing for the normal predictor

Subsequent calls to Riemann solvers for transverse corrector

New approach (Lee and Deane 2009):

Characteristic tracing for BOTH normal predictor and transverse corrector!

Page 19: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

A primitive form:

where the coefficient matrix is

First consider the evolution in the x-normal direction and treat the normal magnetic field separately from the other variables:

Linearized System, cont’d

Normal predictor

MHD source termFLASH Workshop

Hamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 20: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Single-step data Reconstruction-evolution in USM

Normal Predictor

Characteristic Tracing

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 21: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Characteristic tracing for Transverse corrector

A jump relationship:

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 22: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Reduced 3D CTU in USM

Characteristic Tracing for

Normal Predictor

Transverse Corrector

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 23: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Full 3D CTU in USM

Characteristic Tracing for

Normal Predictor

Transverse Corrector

F u l l C T U d i a g o n a l c o u p l i n g FLASH Workshop

Hamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 24: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Summary of Part 1

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

New approach of using characteristic tracing for BOTH normal predictor and transverse corrector

Reduced 3D CTU A direct extension of 2D CTU to 3D Requires 3 Riemann solves for 3D (6-ctu needs 6 Riemann solves) Only including second cross derivatives CFL limit ~ 0.5

Full 3D CTU Full considerations of accounting for third cross derivatives Requires 3 Riemann solves for 3D (12-ctu needs 12 Riemann solves) CFL limit ~ 1.0 20% relative performance gain compared to reduced 3D CTU

Page 25: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 2

Divergence-Free fields:Constrained Transport (CT) MHD

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 26: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 2

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

CT scheme by Balsara and Spicer, 1998:

Page 27: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 2: recall…

Conservative variables and fluxes:

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 28: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 2

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

New upwind biased modified electric field construction(upwind-MEC), Lee 2012:

Page 29: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 2

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Small angle advection of the 2D field loop:

Page 30: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 2

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Small angle advection of the 3D field loop:

Page 31: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Summary of Part 2

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Three CT schemes were discussed: Standard CT scheme by Balsara and Spicer, 1998:

Takes a simple arithmetic averaging Lacks numerical diffusion for magnetic fields advection

Modified electric field construction (MEC) scheme by Lee and Deane, 2009: 3rd order accurate in space Not enough numerical diffusion for field advection

Upwind biased MEC (upwind-MEC) scheme by Lee, 2012 (to be submitted) Upwind scheme of MEC Added numerical diffusion to stabilize field advection

Page 32: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 3

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Verification, convergence, and performance

Page 33: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 3

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 34: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 3

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 35: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 3

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 36: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 3

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 37: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 3

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 38: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Part 3

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 39: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Summary of Part 3

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Verification tests for the reduced/full 3D CTU schemes:

CFL=0.95 for all 3D simulations using the full CTU scheme

CFL=0.475 for the reduced CTU scheme

They both converge in 2nd order

20% performance gain in using the full CTU scheme:

Various choices in runtime parameters

Page 40: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Conclusion

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Directionally split vs. unsplit formulations for hydro and MHD

Unsplit hydro/MHD solvers in FLASH4 (also FLASH3 in part) The reduced and full 3D CTU algorithms Upwind-MEC scheme for MHD Stable solutions with 2nd order convergence with CFL=0.95 20% performance gain in the full CTU scheme over the reduced CTU

scheme

Work in progress: Fully implicit Jacobian-Free Newton-Krylov implicit solver for the unsplit

solvers More HEDP capabilities for the USM solver

Page 41: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Thank You

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Questions?

Page 42: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

New Upwind PPM for Slowly Moving Shock

Upwind PPM 5th order WENO

Standard PPM Standard PPM with increasing By

larger By

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 43: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

New Upwind PPM for Slowly Moving Shock

Upwind PPM 5th order WENO

Standard PPM Standard PPM with increasing By

Lee, 2010, 5th Astronum Proceeding;

Lee, 2011, in preparation

larger By

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012

Page 44: FLASH Workshop Hamburger  Sternwarte , University of Hamburg, Feb 15 – Feb 16, 2012

Block and Mesh Packages

Uniform Grid AMR with variable patch size - CHOMBO

q Mesh package can be selected at configuration time

q The basic abstraction is a block of interior cells surrounded by guard cells

q Grid unit makes sure that blocks are self contained before being given to the solvers

Oct tree based AMR - PARAMESH

FLASH WorkshopHamburger Sternwarte, University of Hamburg, Feb 15 – Feb 16, 2012