high-performance computing in examples - ictp · high-performance computing in examples dr. axel...
TRANSCRIPT
High-Performance ComputingHigh-Performance Computingin Examplesin Examples
Dr. Axel Kohlmeyer
Scientific Computing Expert
Information and Telecommunication SectionThe Abdus Salam International Centre
for Theoretical Physics
http://sites.google.com/site/akohlmey/
2
My Background
● Undergraduate training as physical chemist, PhD in Theoretical Chemistry, University Ulm
● Postdoctoral Research Associate,Theoretical Chemistry, Ruhr-University Bochum
● Senior IT-Support Staff, Associate DirectorCenter for Molecular Modeling,University of Pennsylvania, Philadelphia
● Associate Vice-Dean for Scientific Computation,Assistant VP for High Performance Computing,Temple University, Philadelphia
8
Cluster Deployment and Usage
CPU cores online (incl. storage, admin)
CPU cores online (incl. storage, admin)
Running processesRunning processesLoad averageLoad average
Nodes onlineNodes online
9
Batch System Usage / Service Units
April May June July August September October November December January0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
800,000
900,000
1,000,000
normalmanycorehighmemgpudevelall
Se
rvic
e U
nits
10
Batch System Usage / Utilization
0 %
10 %
20 %
30 %
40 %
50 %
60 %
70 %
80 %
90 %
100 %
develgpuhighmemmanycorenormal
Experiments on ssDNA-CNT
Staii, C. et al, Nano Letters, 5, (2005)
Air flowDNT flow
■ Bare Nanotube
■ Nanotube + ssDNA
Detected Chemicals
Methanol
Dinitrotoluene Propionic Acid
Dimethyl methylphosphonate
Trimethylamine
DNA Sequence Specific Response
Detected Chemicals
Methanol
Dinitrotoluene Propionic Acid
Dimethyl methylphosphonate
Trimethylamine
■ Sequence A
■ Sequence B
Air flowMethanol flow
Self-Assembly of ssDNA-CNT
Right-handed Helix
Left-handed Helix
Kinked Structure
G
A
T
C
0.7 eV
0.64 eV
0.59 eV
0.53 eV
System equilibrates within ~30 ns
Many possible conformations
Adsorption driven by stacking Captures 67% of adsorption energy
Purines exhibit largest stacking energy G > A > T > C
Kinked structure stabilized by intra-DNA stacking interactions
15
Ergodicity Problem with MDP
oten
tial
E
ner
g y
ssDNA Conformation
300 K
System Temperature
System trapped in local minimum
700 K
Pot
enti
al
En
erg y
Full ssDNA configuration space sampled
16
Replica-Exchange Technical Details
Gromacs 4.x classical MD package, with parallel replica support
Individual MD scales out at 32CPUs
64 Replicas with temperatures ranging from 290 K to 715 K
Exchange attempted every 0.6 ps
Temperatures chosen to maintain exchange probability of 20 – 30%
Run on 2048 CPUs of IBM Blue Gene/L at SDSC in 2007
More than 3.8×106 ssDNA conformations sampled (each RE-attempt)
17
Simulation Results
R. R. Johnson, A. Kohlmeyer, A. T. C. Johnson and M. L. KleinNano Letters, 2009, 9 (2), pp 537–541
PEG Surfactant Example
= HOCH2-
= -CH2OCH2-
= -CH2CH2CH2-
= -CH2CH3
Building Blocks of PEG surfactant
C12E2
Shinoda et. al. Mol. Sim., 33, 17 (2007)
Parameterized from the ground up!
One CG particle per ~10 atoms.
Fit bulk systems of alkanes, alcohols, glycols and ethers to reproduce surface tension and density using LJ 9-6.
Application: Phase diagram / C12E6
Jonsson et al., ‘Surfactants and Polymers in Aqueous Solution’
X X X
PEG C12E6 Phase Transition
807,360 CG beads61696 PEG molecules
Start 50 wt% PEG(Hexagonal phase)
80 wt% PEG(Lamellar)
Dehydrate
80 ns
PEG C12E6 Formation of Hexagonal Phase
1,237,760 CG beads64,000 PEG molecules
50 wt% PEGStart random configuration
67 ns
Self-Assembly of Cyclic D,L-α-Peptides with CG-MD
● Antimicrobial Agents● Self assemble into tubes-like monomers● Induce membrane leakage● Leu-Trp repeat
Ekta Khurana, R. H. DeVane, A. Kohlmeyer and M. L. KleinNano Lett., 2008, 8 (11), pp 3626–3630
System Setup
● 220 Peptide Rings in Nonane/Water● ~ 29000 Coarse Grain Beads● 5 fs time step, Velocity Verlet● LAMMPS MD code● > 2µs trajectory
27
Optimization and Parallelization
●
●
●
●
●
●
● 30M CPU hour INCITE grant on Cray XT5● Plain LAMMPS doesn't scale well enough
● Vesicle fusion study:impact of lipid ratio in binary mixture
● LAMMPS MD software● Experimental size:
4M particles for 1 vesicle and solvent