FRG: Multiscale Simulation of Atomistic Processes in Nanostructured Materials
Rajiv K. Kalia, Aiichiro Nakano & Priya VashishtaConcurrent Computing Laboratory for Materials Simulations
Dept. of Physics, Dept. of Computer Science, Louisiana State Univ.
Dept. of Materials Science, Dept. of Physics, Dept. of Computer Science, Dept. of Biomedical Engineering
Univ. of Southern California (after September 2002)Email: {kalia, nakano, priyav}@bit.csc.lsu.edu
URL: www.cclms.lsu.edu
NSF Division of Materials ResearchComputational Materials Theory Program Review
Program Managers: Dr. Bruce Taggart & Dr. Daryl HessOrganizers: Dr. Duane Johnson & Dr. Jeongnim Kim
June 20, 2002, Urbana, ILCCLMSCCLMSCCLMS
Outline
1. Multiscale finite-element/molecular-dynamics /quantum-mechanical simulation on a computational Grid
2. Immersive & interactive visualization of billion-atom systems
3. Multimillion atom MD simulations of fracture & nanoindentation in crystalline, amorphous & nanophase materials & ceramic nanocomposites
4. New educational and outreach programs
Multiscale FE/MD/QM Simulation
10-6 - 10-10 m> 10-6 m
Multiscale simulationto seamlessly couple:• Finite-element (FE)
calculation• Atomistic molecular-
dynamics (MD) simulation
• Quantum-mechanical(QM) calculationbased on the densityfunctional theory(DFT)
• Newton’s equations of motion
• Interatomic potential
• N-body problemlong-range electrostatic interaction—O(N2)
• Efficient O(N) solution—space-time multiresolution algorithm1. Fast Multipole Method (FMM) (Greengard & Rokhlin, 87)
2. Symplectic Multiple Time-Scale (MTS) method (Tuckerman et al., 92)
Molecular Dynamics Simulation
mi
d2r r i
dt2= −
∂Vr r N( )
∂r r i(i = 1,...,N )
V = uij rij( )
i< j∑ + v jik
r r ij ,r r ik( )
i, j<k∑
Evaluate at x = xj (j = 1,...,N)Ves(x) =qi
| x − xi |i=1
N∑
Density functional theory (DFT)
O(CN ) O(N3 )Constrained minimization problem:• Minimize E[{ψn}] with orthonormal constraints,
Efficient parallelization: real-space approaches• Finite difference (Chelikowsky, Troullier, Saad, 94) & multigrid acceleration (Bernholc, 96)
O(N) algorithm (Kim, Mauri & Galli, 95)
• Asymptotic decay of density matrix:• Localized functions:
• Unconstrained minimization—Lagrange multiplier
Quantum Mechanical Calculation
ψ (r1,r2 ,K,rNel) ψn (r) | n = 1,K,N el{ }
dr∫ ψm* (r)ψ n (r) =δmn
ρ(r, ′ r ) ≡ ψn* (r )ψ n ( ′ r )
n=1
Nel
∑ ∝ exp −C | r − ′ r |( )
φm (r) = ψ n (r)Unmn∑
ψn(r) φm(r)
Rcut
On 1,024 IBM SP3 processors:• 6.44-billion-atom MD of SiO2• 444,000-electron DFT of GaAs
Scalable Scientific Algorithm Suite1,024 IBM SP3 & Cray T3E processors at NAVO
IEEE/ACM Supercomputing 2001Best Paper Award
Hybrid MD/QM Algorithm
QM
MD
Handshakeatoms
Additive hybridizationReuse of existing MD & QM codes(Morokuma et al., ‘96)
Scaled-position link atomsSeamless coupling ofMD & QM systems
MD simulation embeds a QM cluster described by a real-space multigrid-based density functional theory
E ≅ EMDsystem + EQM
cluster − EMDcluster
QM
MD≅ + −MD QM MD
Hybrid FE/MD Algorithm• FE nodes & MD atoms coincide in the handshake region• Additive hybridization
MD
FE
[0 1 1]
[1 1 1]_
HS
_[1 1 1]
[2 1 1]
Si/Si3N4 nanopixel
FE/MD/QM Simulation: Oxidation on Si Surface
Dissociation energy of O2 on a Si (111) surface dissipated seamlessly from the QM cluster through the MD regionto the FE region
MD FE
Stress Corrosion in Si
Yellow: QM-HRed: QM-OGreen: QM-SiBlue: HS-SiGray: MD-Si
Reaction of H2O molecules at a Si(110) crack tip
237 QM atoms
Stress Corrosion in SiSignificant effects of stress intensity factor on the reaction
Yellow: HRed: OGreen: Si
Distributed Cluster Computing
• Globus/MPICH-G2 — Ian Foster (ANL)• S. Ogata (Yamaguchi), F. Shimojo (Hiroshima),
K. Tsuruta (Okayama), H. Iyetomi (Niigata)
MD
QM1
QM3
QM2
Additive hybridization; multiple QM clustering
Multiscale Simulation on a Grid
• Scaled speedup, P = 1 + 8n (n = number of clusters)• Efficiency = 94.0% on 25 processors in the US & Japan
Immersive & Interactive Visualization
• Octree-based fast view-frustum culling• Parallel/distributed processing
Billion-atom walkthrough
PC cluster
WAND User
positionGraphics
server
ImmersaDeskthrough DURIP
Reduced data
Parallel & Distributed Visualization
Nearly real-time walkthrough for 1 billion atoms on an SGI Onyx2 (2 MIPS R10K, 4GB RAM) connected to a PC cluster (4 800MHz P3)
Outline
1. Multiscale finite-element/molecular-dynamics /quantum-mechanical simulation on a computational Grid
2. Immersive & interactive visualization of billion-atom systems
3. Multimillion atom MD simulations of fracture and nanoindentation in crystalline, amorphous & nanophase materials and ceramic nanocomposites
4. New educational and outreach programs
Fracture in Glasses, NanophaseCeramics & Nanocomposites
Systems– Nanophase Si3N4, SiC, Al2O3
– SiO2 glass – Nanocomposite - SiC fibers in a Si3N4 matrix
Issues– Atomistics of crack propagation– Mechanisms of energy dissipation– Scaling properties of fracture surfaces
Nanophase ceramics
Si3N4
Amorphous SiO2
0
0.5
1
1.5
2
0 5 10 15 20
Expt.
MDS
N(q
)
q (Å -1)
Validation of Interatomic Potential
MD results for elastic moduli also agree well with experiments
Fracture in Amorphous Silica: 15 Million Atom MD Simulations
Fracture ToughnessKIC =1 MPa.m 1/2 (MD)
0.8 MPa.m 1/2 < KIC < 1.2 MPa.m 1/2 (Experiment)
Experiment by Bouchaud et al.
Cavity formation & coalescence with crack in a glass
(Top) AFM study in a glass:(a) nanometric cavities before the crack advances; (b) growth of cavities; (c) crack propagation via the coalescence of cavities.(Bottom) MD results showing the same phenomenon in a-SiO2
Fracture Simulation & Experiment
Dynamic Fracture in Nanophase Si3N4
Pore CoalescenceCrack Deflection
Toughening Mechanism: Crack deflection, branching & coalescence with nanopores
Nanophase Si3N4 is much tougher than Si3N4 crystal
Morphology & Scaling Behavior of Fracture Surfaces
Fracture surfaces are anisotropic & statistically invariant under an affine transformation
),,(),,( bzbyxbzyx ζ→
z z+r
∆h(r)
x
ζ is called the roughness exponent
( ) 2/12)()()(z
zxrzxrh −+=∆
ζrrh ∝∆ )(
P. Daguier, B. Nghiem, E. Bouchaud & F. Creuzet (1997)
Scaling Behavior of Cracks
Two regimes with roughness exponents 0.5 & 0.8
0
0.5
1.0
1.5
2.0
2.5
2 3 4 5 6ln(z)
ln∆h
(x)
ζ = 0.58 ± 0.14
ζ = 0.84 ± 0.12
MD results for roughness exponents agree with experiments
MD results reveal that the smaller roughness exponent is due to intrapore correlations and the larger one due to coalescence of pores and crack
NanophaseSi3N4
Color code: Si3N4; SiC; SiO2
Fracture in a Nanocomposite
Silicon nitride matrix-silicon carbide fiber nanocomposite
1.5-billion-atom MD on 1,024 IBM SP3 processors
0.3 mm
Nanoindentation in Si3N4
Multi-million atom MD simulations
Nanohardness yields important informationabout elastic and plastic deformation
Indentation Fracture & Amorphization
<1210> Indentation fractureat indenter diagonals
Amorphous pile-upat indenter edges
Anisotropicfracture toughness
<1010>
<0001>
Hardness of Silicon Nitride
Load-displacement curve
α-crystal (0001)50GPa
Amorphous32GPa
Microindentation experiments on α-crystal (0001): 31-48GPa
Summary
• Parallel multiscale MD/QM simulation methodology
• Interactive visualization of billion atoms
• Multimillion atom MD simulations> fracture in silica glass, nanophase
ceramics & nanocomposite> nanoindentation in crystalline &
amorphous Si3N4
Graduate Education
Ph.D. in the physical sciences & M.S. in computer science
Dual-Degree Program
International course on computational physics
• Synthesis
• Characterization
• Property measurements
Experimental training
• LSU/USC
• TU Delft, The Netherlands
• Niigata Univ., Japan
Undergraduate Outreach Activities
Computational Science Workshop for Underrepresented Groups
• 19 participants from 11 institutions — Hampton, Clark-Atlanta, Morehouse, Jackson State, Mississippi State, Texas Southern, Univ. of Texas –– Pan American, Xavier, Grambling, Southern & Univ. of Louisiana in Monroe
• Activities: Construction of a PC cluster from off-the-shelf components & using this parallel machine for algorithmic and simulation exercises.