1
Douglass E. PostChair, DOE Fusion Simulation Project
Steering Committee(D. Batchelor—ORNL, R. Bramley—U. of Indiana,
J. Cary—U. of Colorado, R. Cohen—LLNL, P. Colella—LBL, S. Jardin—PPPL, D .Post—LANL)
June 4, 2004PASCI meeting, Princeton, NJ
What are the scientific issues for The Fusion Simulation Project?
PRINCETON PLASMA PHYSICS LABORATORY
PPPL
October 14, 2003 2
Fusion Simulation capability has three levels of maturity.
• 3 major task categories—an R&D challenge– “New research”
• Things that we don’t know how to do• In conjunction with existing theory/computational program and SCIDAC,
launch R&D initiatives to begin the process of developing practical modules, will likely take several stages to mature
• Strongest research focus
– “Focused Integration Initiatives”• Things we have a good idea of how to approach, but need some R&D• Fund them to get experience, develop algorithms, deliver capability to
program in short to medium term• Great opportunity for incremental delivery
– “Whole Device modelling”• Things we really know how to do• Fund them to deliver capability in short term, learn and gain experience,
develop prototypes for final simulation code• Good prototypes already exist• EU effort beginning with this emphasis
3
The PSACI asked us to identify the scientific issues the FSP will address.
• FSP will address the overarching question in the DOE Office of Science Strategic Plan: “How can we create and stably control a high-performance fusion plasma?”
• In particular many of the major outstanding scientific questions of vital importance to the future of the program:1. What are the conditions under which a sequence of nonlinear
magnetohydrodynamic (MHD) events lead to a catastrophic disruption of the fusion plasma?
2. What is the physics of the plasma edge whose structure critically affects global confinement of heat and particles?
3. What is the physics of fine scale turbulence on global transport length and time scales?
4. How do we best use heat, particle, and current sources to control plasmas to achieve high levels of fusion power?
4
Planning path
UltimateCapability
GlobalSimulationFor burningPlasmasDischarge controlTheory&Model testingScenarioOptimization
Development path2005 202020152010
PhysicsMathComputer ScienceData analysisActuator controls
2005 202020152010
Ne
w re
sea
rch
ele
me
nts
Extended MHD 1Extended MHD 2Extended MHD 3
Well defined elements:Neutral beam heating/CDEdge neutrals
Well defined FIIs: RF heating CD+MHD stabilityEdge plasma transport +other edge physics
Make useful contributions Gain experienceEstablish credibility
Wh
ole
de
vice
pro
totyp
es
Address unsolved problemsDevelop concepts for solutionsDevelop algorithms/methods
FIIs w
ith sh
ort
Te
rm a
nd
in
term
ed
iate
term
b
en
efit
Well defined FIIs
Core Transport 1Core Transport 2Core Transport 3 Core Transport
Extended MHD
5
Multiple times are a challenge.
6
FSP has always recognized the central role of R&D.
7
In 2002, US Fusion Community proposed a “Fusion Simulation Project”
• In 2002, FESAC and DOE proposed a “Fusion Simulation Project”
• Ramp up to $20 M per year in 3 to 4 years, begin with $4M in FY05
Fusion Simulation Project, Integrated Simulation and Optimization of Fusion Systems
Jill Dahlburg, General Atomics (Chair)James Corones, Krell Institute, (Vice-Chair)Donald Batchelor, Oak Ridge National LaboratoryRandall Bramley, Indiana UniversityMartin Greenwald, Massachusetts Institute of TechnologyStephen Jardin, Princeton Plasma Physics LaboratorySergei Krasheninnikov, University of California - San DiegoAlan Laub, University of California - DavisJean-Noel Leboeuf, University of California - Los AngelesJohn Lindl, Lawrence Livermore National LaboratoryWilliam Lokke, Lawrence Livermore National LaboratoryMarshall Rosenbluth, University of California - San DiegoDavid Ross, University of Texas - AustinDalton Schnack, Science Applications International Corporation
Office of Science
U.S. Department of Energy
On the Status of MHD Modeling
S.C. JardinPPPL
meeting of the FSP planning steering committeeMay 18-20, 2004 Boulder, Colorado
Office of Science
U.S. Department of Energy
Outstanding Problems in MHD
Full nonlinear sawtooth oscillation modeling in fusion-grade plasmas
Tearing mode and neoclassical tearing mode excitation in high-beta plasmas
Nonlinear evolution and control of resistive wall modes, including toroidal flows.
Effects of Fast Ions
Edge MHD-type instabilities
Disruptions and Vertical Displacement Events
Pellet Injection fueling of a Burning Plasma
For each of these, the goal is to have both a detailed 3D complete model that can be invoked, and a simplified approximate model for the WDM code
Office of Science
U.S. Department of Energy Full nonlinear sawtooth oscillation modeling in fusion-grade plasmas
What will be the period and inversion radius of the sawtooth oscillation in an ITER-class burning plasma, as a function of plasma current and pressure?
What physics underlies complete and incomplete reconnection during the sawtooth?
Under what conditions will the sawtooth instability trigger the onset of a metastable island (neoclassical tearing mode) or lead to a disruption?
Can RF be used effectively to stimulate small sawteeth in ITER?
• Requires Extended MHD model (more terms than in resistive MHD) to get correct onset criteria, crash time
• Requires Hybrid description including energetic alpha particles in ITER
• Multiple space-scales due to small reconnection regions
• Free boundary with vacuum region required.
• Extensive validation program required.
• Coupling to RF modeling desired.
Cohen FSP_edge 5/18-20/2004 -11-
EDGE SIMULATION ISSUES FOR THE FSP
Ronald H. Cohen
Presentation to
FSP Workshop, Boulder
May 18-20, 2004
Acknowledgments: W. Nevins, T. Rognlien, M. Umansky, X. Xu
Cohen FSP_edge 5/18-20/2004 -12-
Integrated edge simulation vital for ITER/BP
• Edge Pedestal establishes T, n at core-edge interface– Dominant control parameters for core confinement– Important transient phenomena (see next slide)
• Transport through edge region strongly impacts ability to handle power & He exhaust and particle fueling
Projections of fusion gain for ITER
Cohen FSP_edge 5/18-20/2004 -13-
What is needed for a predictive capability
• Need to model a range of phenomena/processes:– Microturbulence– Intermittant meso-scale transport (“blobs”)– Formation of “transport barriers” (“L-H transition”)– Edge-Localized Modes (ELMs)– Bootstrap current– Neoclassical (drift-induced) transport– Dust generation/transport– Neutral-particle and impurity transport– Wall erosion processes
Cohen FSP_edge 5/18-20/2004 -14-
What is needed for a predictive capability(cont)
• Essential simulation ingredients– 3-D multi-species plasma model; kinetic– Electromagnetic (including MHD)– Field-line-aligned grid with X-point geometry– Neutrals– Impurities– Plasma-wall interactions– Spatial domain spanning from material wall in through the
pedestal– Temporal evolution spanning scales from turbulence
(µsec) through multiple ELM cycles, macroscopic evolution in presence of recycling (100’s of ms)
• Of necessity, this is integrated simulation!
Cohen FSP_edge 5/18-20/2004 -15-
Roadmap to Predictive Edge Simulation
Where we’ve been: mostly isolated pieces, limited parallelism
UEDGE, B2 (2-D transport)
BOUT (3-D 2-fluid turbulence)
WBC, Redep (near-surface transp/ chemistry)
Filling in the gaps
Coupled fluid turbulence/transport (LLNL-LDRD)
Comprehensive fluid turbulence (community SciDAC proposal)
Kinetic edge code (turbulence timescales) (LLNL, NYU, Helsinki)
Fusion Simulation Project:
EDGE FII
Where it would come together
DIVIMP, MCI (impurity transport)
DEGAS, NUT (kinetic neutral transport)
MD-based models of plasma-wall interaction (Argonne, U. Ill, LLNL)ELITE (Edge MHD stability)
etc
Fusion Simulation Project:
Global model
Cohen FSP_edge 5/18-20/2004 -16-
We would bring it all together in the FSP
• Edge is a logical choice for an FII– Bring together:
• Multiple-timescale techniques (to extend to discharge timescales)• Full edge plasma physics
– Initially fluid (from SciDAC)– Add kinetic option when available
• Physics-based (ultimately, MD-based) plasma-wall interactions • Monte-Carlo neutrals• Evolving MHD edge equilibrium
Cohen FSP_edge 5/18-20/2004 -17-
An edge FII must address computational challenges
• Here are a few:– Multiple timescales– Robust solution algorithms– Efficient utilization of many CPU’s for heterogeneous
physics– Open-closed flux surface topology change
• Singularity at x point• Different b.c.’s
– Spatial scales -- broad range but much overlap• High-order discretizations• Preconditioners• Multi-dimensional parallelization• AMR
Cohen FSP_edge 5/18-20/2004 -18-
Plausible stages of computational development-1
• Baseline approach (SciDAC?)– Multi-fluid 3-D plasma with:
• Static field-line-aligned grid• Fully implicit advance (Krylov/PVODE)• Spatial-discretization-independent representation of physics • Finite difference as initial spatial discretization, other options
as they are available
– Fluid neutrals– Fluid impurities– Phenomenological plasma-wall interaction models– Component framework (e.g. CCA)– Science applications: Dynamics for single ELM crash;
turbulence and neoclassical transport between crashes
Cohen FSP_edge 5/18-20/2004 -19-
Plausible stages of computational development-2
• Extension to long timescales– Coupling of turbulence and transport timescales via
relaxed iteration– 3-D turbulence; 2-D transport equations– Improved preconditioning– AMR, if warranted– Science application: pedestal evolution on transport
timescale
Cohen FSP_edge 5/18-20/2004 -20-
Plausible stages of computational development-3
• Extension to kinetic models and physics-based plasma-wall interactions:– Complete development of 3-D edge gyrokinetic turbulence
model (begun under other funding). Nominally: 5-D continuum (alternate: PIC)
– Substitute gyrokinetic turbulence model and its 2-D reduction for transport for fluid-based turbulence and transport models
– Replace fluid neutrals model by Monte-Carlo neutrals, or hybrid– Include radiation transport, especially for hydrogen lines– Extend gyrokinetic plasma model to include kinetic impurities– Retain fluid model option, with added closure terms to model
kinetic effects (fast-running option)– Replace phenomenological plasma-wall models by models
based on physics computations (e.g. molecular dynamics and near-sheath plasma) of wall interactions
– Science applications: previous problems at higher temperature and/or lower density
21
These issues are driven by the requirement to simulate all aspects of a burning plasma.
• Solve conservation equations for density, momentum, and energy on a set of grids– Calculate source and sink terms– Calculate coefficients
• Individual elements– MHD– Transport– Edge– RF, CD, fuelling, impurities, NBI, plasma control,
diagnostics, physical data
• Integrate them
22
We are asking for $2M to get started. Why?
• We are asking for $2M to get started. What will we spend it on?– It’s $1M from OFES, $1M from OSACR– An FTE at a lab costs roughly $330k fully burdened. Universities are a little
less. $1M = 3 FTEs
• 3 FTE’s ~ 6 half-time staff– OFES: Project leader, MHD, transport, edge, whole device modelling,
sources, all at 0.5– OSACR: 0.5 Project leader, 1 computational mathematicians, 0.5
parallelization, 0.5 computational infrastructure, 0.5 collaboration infrastructure
• Essential to leverage base program• $2M may seem like a huge sum. It’s actually $1M from OFES.
OASCR will fund “computer science” activities, not physics. – It’s actually small for planning a $300M project. The usual rule of
thumb is that the design phase of a project costs about 10% of the total project.
– One reason that $1M is a substantial sum on the SciDac scale is that SciDac is highly leveraged. I don’t have the exact numbers for the MHD SciDac effort, but if 20 people are working on it and the funding is about $500k, that’s an average of $25k per person, about 8% of what that person actually costs.
23
Strawman WBS for pre-conceptual design
• OFES ( all at about 1/2 FTE of senior staff from many different institutions): – Project leader—overall coordination, ensure that vision is developed, interface with
DOE, ITER, community, ….– MHD—assess state of art, develop plans and requirements for FII’s and R&D, gather
requirements from experimentalists, ITER, etc., coordinate with comp math community on new opportunities,
– Transport, Edge, sources - similar– Whole device modelling—assess state of art, data bases, code libraries, investigate
opportunities for improvement, gather requirements from experimentalists
• OSACR—OSACR community needs to involved from the beginning: – 0.5 Project leader—same as above – 1 FTE of computational mathematicians—work with fusion community to identify
opportunities for improved algorithms, liaison with applications and comp math communities
– 0.5 parallelization and computer infrastructure—assess state of parallelization, advantages and opportunities, develop appropriate strategies, interface with platform issues
– 0.5 computational and software infrastructure—assess state of the art, identify issues and opportunities for fusion community, develop plan, begin prototyping
– 0.5 collaboration infrastructure—assess state of the art, identify issues and opportunities, begin prototyping using pre-conceptual design team,
24
Senior people are interested in getting involved
• Most of the present FSP committee is interested in a half time involvement– E.g. Phil Colella has expressed a strong interest, other comp
math people are interested, liaison groups like D. Schissel’s group are interested, but project must have a good chance of becoming real!!!
• Many others in the OFES and OASCR communities have expressed interest
• FSP is a major new activity, and a path for the future• EU and Japanese groups have expressed a strong
interest in collaboration• ITER physics groups have also expressed a strong
interest
October 14, 2003 25
ASCI had a large R&D effort in their large Program ~ $747 M/year (proposed FY02)
Program element Budget($M)
Applications—Development of applications and algorithms 147MPM—Materials and Physics Models 68PSE-Problem Solving Environment 54DisCom—High Speed Classified Network 29Views—visualization 80Ongoing Computing—Operation of present computers 97Platforms—New computers 153Institutes—Computer Science and Math Research 16Path Forward—Enabling Technologies 24Verification and Validation—V&V 42ASCI Alliances and University Programs 25 + 12
October 14, 2003 26
Summary Requirements for ICF/HEDP Simulation Capabilities and Features, From LA-UR-03-0960, D. Post, May 26, 2002 Challenging but easier than MFE
27
An appropriate balance between a strong R&D effort and delivering a product must be maintained!
• Tension between: – Sufficient freedom to innovate and do research to develop methods to
solve new problems we can’t yet solve– Sufficient structure and accountability to ensure that a useful product is
delivered on time and on budget
• In particular:– Sufficient local authority within the project elements to innovate and
develop the new methods and physics understanding essential for successAnd – Sufficient central authority to make binding decisions on technical merit
rather then political considerations– Sufficient structure and authority to ensure adequate coordination among
the various collocated and non-collocated project elements. – Sufficient structure that the project sponsors (DOE and the fusion
community) can hold the project and the project team members accountable
28
The FSP can use the “lessons learned” from other large scale computing projects.
• Lessons learned are available from other large scale scientific computing projects—we can be more efficient by building on them– ASCI, NNSA– ASCI Alliances– DARPA HPCS– ESMF, climate modelling, environmental sciences,– DOD, NOAA, industry, astrophysics, atomic physics,
hydrology and oil reservoir, CFD, Combustion and fire modelling, Geophysics, engineering, chemistry, solar and planetary physics, …
• DARPA HPCS
29
FSP is an opportunity, perhaps our best opportunity, to get the additional resources we need to make progress.
• The reality of the present budgeting process at DOE is that we need to have a “new, exciting, innovative initiative” to get new funding.
• Otherwise, we will be looking at constant funding or worse for the foreseeable future
• The FSP offers us a chance to reverse that trend. Let’s go for it. • It will not only give us additional plasma physics resources, but
will give us leverage with the comp sci community• OASCR is really enthusiastic about the FSP. Not only do they
want to help us with funding, but are giving us priority for computer time and access to their best people. Let’s let them help us.
• All of us involved in the FSP understand the prime importance of R&D