high performance computing in computational chemistry - scicomp

65
High Performance Computing in COMPUTATIONAL CHEMISTRY March 23, 2004

Upload: others

Post on 03-Feb-2022

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

High Performance Computing in COMPUTATIONAL CHEMISTRY

March 23, 2004

Page 2: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Tutorial outlineIntroduction to the tutorial

Overview of applications performanceComputational Chemistry at CINECA

Electronic structure applicationsGaussian, Gamess - Angelo RossiNWchem - Sigismondo Boschi

Molecular Dynamics applicationsCHARMM, GROMACS, AMBER - Angelo Rossi NAMD - Joachim Hein

Car-Parinello applicatinosPWscf - Carlo Cavazzoni

Page 3: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

High Performance Computing in COMPUTATIONAL CHEMISTRY

INTRODUCTIONSigismondo Boschi, CINECA

[email protected]

Page 4: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

IntroductionOverview of the applications that commonly runs on our High Performance systems in the applicative areas of:

Physical ChemistryChemical Physics

Page 5: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

ChemistryLong tradition in coding electronic structure codes (from 60's), started with Gaussian by Pople and his collaboratorsAb-initio codes have been distributed from the very beginning (from 70's)Most of Computational Chemists are used to use them as black-boxesMany as state-of-the-art engines on which they base codes peculiar of their group researchSome still develop their own codesLess develop large distribution codes

Page 6: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

PhysicsStarted using large distribution codes more recentlyIn 'original' (QCD, continuum theories) disciplines codes were proprietaryA lot of work for ab-initio codesToday they share a lot of research areas with Chemistry

Page 7: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

SharesNew discoveries in science are driving the application areas of the scientists coming from the two disciplines:

BiochemistryMatter PhysicsBioinformaticsGenomics, Proteomics...

Page 8: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Electronic structure applicationsLocalized orbitals, ab-initio codesBorn for single molecules, applied to periodic systems too

Hartree-Fock theory, SCF methodDensity Functional theory, SCF methodPost-SCF methods:

Configuration Iteration methods (Full CI, CIS, CISD,...)Coupled Cluster methodsTime-Dependent HF/DFT methodsMoller-Plesset perturbation theory

Plane waves: Car-Parinello codesBorn for periodic systems, applied to single molecules too

Page 9: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Molecular Mechanics applicationsThe atomic/molecular interactions are simplified, in order to simulate much larger systems

Molecular Dynamics methodMonte Carlo methodDocking techniquesEnergy minimization techniques

Page 10: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

'others'Genome analysis and pattern matchingMolecular docking techniquesProtein activity prediction, ProteomicsMixed methods (QM/MM, Oniom, ...)Chemical activity prediction, Cheminformatics

Page 11: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Problem sizesAb-initio: 1-500 atomsMolecular mechanics: 1000-1000000 atoms

Obviously, the kind of information investigated and obtained is different in the 2 cases

Page 12: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Comparison of Classical and Ab Initio Molecular Dynamics

Classical MDPhenomenological energy surface (typically two-body force-fields, obtained from...)

Difficult to describe bond making/breaking

Electronic properties not available

Can do millions of particles: ensemble and thermodynamics properties

First principles MDPotential energy surface calculated directly from Schrodinger equation - many body terms automatically included

Describe bond making/breaking

Electronic spectra included in the calculation

Limited to some hundredths of atoms with "significant" dynamics

Page 13: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Computational chemistry needsFrom now on I am referring to ab-initio, electronic structure, localized basis functions application, and in particular to:

Gaussian (g98 on CINECA SP4)NWchem (version 4.5 on CINECA SP4 and CLX)

What do they typically need? If you ask to a researcher, used to Gaussian:

MemoryDiskA lot of time (no possibilities of software/OS checkpoint or restart, in most of post-scf methods; no MPP for Gaussian and most of other applications).

Page 14: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

SCF-HF exampleThe complexity of a computational chemistry system is determined by the number of basis functions: for the simplest engine you do need to evaluate:N4/8 integrals: O(N4) each of them will be used more times to build the Fock matrix.Apply the SCF procedure to an NxN matrix: O(N2.0-3.0)In general (not only for SCF) you can distinguish three approaches:

Page 15: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Integral evaluation approachesIN CORE: all the integrals are evaluated once and put in memory. Then the matrix is build from them;DIRECT: any time you need an integral it is evaluated, but never stored;SEMI-DIRECT: some of the integrals are stored in memory or on disk, the others are evaluated when needed;

How do I choose? It depends on the characteristics of your computer. On today architectures semi-direct is the most common choice.

Page 16: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Gaussian98 on SP4Distributed in all the world for usage on workstations of research departmentsVery few copies for HPC where:

memorydisknumber of CPUs per nodenumber of nodes

are "extraordinary"

Are there any problems in the "scaling" of Gaussian to larger platforms?

YES

Page 17: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Using the available resourcesMemory:

On SP4: comment out -bmaxdata:0x40000000, in rs6k.make and link.make with 64 bits compilations.On other systems (Irix, Linux) you need to enlarge "SHMMAX" (System V shared memory).

Disk: after a first period, when everything was fine, the applications doing large post-scf computation (MP2, CCSD, CASSCF run with Gaussian, Molpro, Molcas) where dying for I/O problems (e.g. write -1 instead of 1879183176) - with RWF of ~30GBs.

Page 18: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

AIX 5.1.0.?-32bit undocumented feature

Even if with _LARGE_FILES support, it was not possibile to write more than 0x70000000-1 (1879048191 bytes) with one single write call.Now the limit is grown to 0x80000000-1 (2147483647 bytes): probably thanks to one of the many applied patches.Is it a bug? POSIX does not cover 64bit extensions of I/O primitives... but it was documented nowhere.

� ��� �� �� �� � � �� �� � � �� �� � �

in mdutil.c and everything was fine again. Originally it was 2000MB, so now it should be working also unmodified!

Page 19: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Using essl

g98 came with ATLAS for Power3 included. Linking g98 with ESSL (3.2, 3.3) gave 10%-15% increase in performance.Standard version available in case of doubts in results.

Page 20: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Parallel GaussianForkWhile some of the post-SCF methods are parallelized on this platform (e.g. qci and coupled cluster when using certain algorithms), for the most part they do not use multiple processors very efficiently. All scf energy, gradient and frequency calculations are well-parallelized (i.e. HF, DFT, CIS) as well as TD-DFT.All the tasks use the same shared memory.

LindaSome bad experience with T3E. SCF and MP2 parallelized, but with replicated memory on all the nodes, plus LINDA memory! no scalability, no capability.Still to be tried on IBM platforms.

OpenMPGaussian03

Page 21: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

I/O subsystems for parallel systems

200 MB/s on CLX

30 MB/s on CLX

scalablededicateddistributed cachedifficult to export data

high speedshared in bandwidthimmediate to uselarge latency

Page 22: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Comparing HF-SCF with NWChem

Timing of the same run (cc-pvtz amoxycillin, 449 basis functions, 109 integrals, 10GBs workarea) with different choices of integral storage

cc-pvtz amoxycillin, 44 atoms, 449 basis functions, C1 symmetry.109 integrals (10GBs workarea)Integral storage choices:

local disks: semidirect, with minimal usage of memorygpfs: semidirect, with minimal usage of memoryno disk: direct if integrals do not fit into memory; in-core otherwiseno disk + mem: tell NWchem to use up to 100MWlocal disk + mem: use local disks and 100MW buffer on every CPU.

Page 23: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Timing

� � � � �� � � � � � � � �� ���� � �

�� � ��� � � �

�� � � � �

���� � � !" #"$% &"

' � !" #' � !" #)( *+ *���� � � !" #" ( *+ *

,.- /0 132

456 7

897:

;<=9 >

Page 24: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Parallel efficiency

? @ A B ?C D @ C A ? @ B @E CFFHG @

FHG AFHG C

FHG B?

? G @? G A

? G C? G B

@

I�JKL IMNO POQR SO

T J MN O PT J MN O P)U VW VI�JKL IMNO PO U V W V

X.Y Z[ \^]

_`a `b

b cbcdde

fe cg f

h

Page 25: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

CPU/elapsed time ratio

i j k l im n j m k i j l jo mpprq i

prq jprq n

prq kprq o

prq mprq s

prq lprq t

i

u�vwx uyz{ |{}~ �{

� v yz{ |� v yz{ |)� �� �u�vwx uyz{ |{ � � � �

�.� �� �^�

��� ��

� ���

����

� �

Page 26: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Computational chemistry needs

With this new view, we have another answer to the question: What do they typically need?

A lot of CPUs!

Page 27: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

High Performance Computing in COMPUTATIONAL CHEMISTRY

Using NWChem 4.5Sigismondo Boschi, CINECA

[email protected]

Page 28: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

� ��� �� ��� � � ��   � � � ¡£¢ ¤ � �� � � � ¥� ��� �� ��� � � ��   � � � ¡£¢ ¤ � �� � � � ¥

Page 29: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Why NWChem Was Developed

Developed as part of the construction of the Environmental Molecular Sciences Laboratory (EMSL)Envisioned to be used as an integrated component in solving DOE's Grand Challenge environmental restoration problemsDesigned and developed to be a highly efficient and portable Massively Parallel computational chemistry packageProvides computational chemistry solutions that are scalable with respect to chemical system size as well as MPP hardware size

Page 30: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

How do you get NWChem?

http://www.emsl.pnl.gov/pub/docs/nwchem => RegisterWebsite with lots of other NWChem informationPrint, fill-out, and sign site agreement form and faxback to PNNL, where Form will be signed by PNNL official anddownload information will be sent via [email protected] for HELP!Mailing lists:

[email protected]

[email protected]

Page 31: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

NWChem ArchitectureR

un-t

ime

data

base DFT energy, gradient, …

MD, NMR, Solvation, …

Optimize, Dynamics, …

SCF energy, gradient, …

Inte

gral

AP

I

Geo

met

ry O

bjec

t

Bas

is S

et O

bjec

t

...PeI

GS

...

Global Arrays

Memory Allocator

Parallel IO

MolecularModeling

Toolkit

MolecularCalculation

Modules

MolecularSoftware

DevelopmentToolkit

GenericTasksEnergy, structure, …

Object-oriented design• abstraction, data hiding,

APIsParallel programming model

• non-uniform memory access, global arrays, MPI

Infrastructure• GA, Parallel I/O, RTDB,

MA, ...Program modules

• communication only through the database

• persistence for easy restart

Page 32: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

NWChem Molecular Electronic Structure - I

The following quantum mechanical methods are available to calculate energies, and analytic first derivatives with respect to atomic coordinates. Second derivatives are computed by finite difference of the first derivatives.

Self Consistent Field (SCF) or Hartree Fock (RHF, UHF, high-spin ROHF). Code to compute analytic second derivatives is under development. Gaussian orbital based Density Functional Theory (DFT), using many local and non-local exchange-correlation potentials spin restricted and spin unrestricted with formal N3 and N4 scaling. Analytic second derivatives for closed shell. Time-Dependent DFT. MP2 including semi-direct using frozen core and RHF or UHF reference. Complete active space SCF (CASSCF).

Page 33: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

NWChem Molecular Electronic Structure - II

The following methods are available to compute energies only. First and second derivatives are computed by finite difference of the energies.CCSD(T), with RHF reference. Selected-CI with second-order perturbation correction. MP2 fully-direct with RHF reference. Resolution of the identity integral approximation MP2 (RI-MP2), with RHF and UHF reference.TCE: Tensor Contraction Engine module, that can generate unrestricted CISD, CISDT, CISDTQ, LCCD, CCD, LCCSD, CCSD, QCISD, CCSDT, CCSDTQ, MBPT(2), MBPT(3), MBPT(4) wavefunctions

For all methods, the following operations may be performed:Single point energy (including the use of ECPs and high angular momentum segmented or generally contracted basis sets in sphericals or Cartesians)Geometry optimization (minimization and transition state)

Page 34: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

NWChem Molecular Electronic Structure - III

Molecular dynamics on the fully ab initio potential energy surface Numerical first and second derivatives automatically computed if analytic derivatives are not available Normal mode vibrational analysis in cartesian coordinates. Generation of an electron density file for graphical display. Evaluation of static, one-electron properties. Electrostatic potential fit of atomic partial charges (CHELPG method with optional RESP restraints or charge constraints)

In addition, automatic interfaces are provided to:The natural bond orbital (NBO) package Python

Page 35: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

NWPW Modules

Three modules are available to compute the energy, optimize the geometry, numerical second derivatives, and perform ab initio molecular dynamics using pseudopotential plane-wave DFT.

PSPW - (Pseudopotential plane-wave) A gamma point code for calculating molecules, liquids, crystals, and surfaces.

Band - A band structure code for calculating crystals and surfaces with small band gaps (e.g. semi-conductors and metals)

PAW - a prototype (gamma point) projector augmented plane-wave code for calculating molecules, crystals, and surfaces

Page 36: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

NWPW Capabilities

Conjugate gradient and limited memory BFGS minimization Car-Parrinello (extended Lagrangian dynamics) Constant energy and constant temperature Car-Parrinello Fixed atoms in cartesian and SHAKE constraints Pseudopotential libraries Hamann and Troullier-Martins norm-conserving pseudopotentials with optional semicore corrections Automated wavefunction initial guess, now with LCAO Vosko and PBE96 exchange-correlation potentials (spin-restricted and unrestricted) Orthorhombic simulation cells with periodic and free space boundary conditions. Modules to convert between small and large plane-wave expansions Interface to DRIVER, STEPPER, and VIB modules Polarization through the use of point charges Mulliken, Blöchl point charge, Wannier, ELF, DPLOT (wavefunction, density and electrostatic potential plotting) analysis

Page 37: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

NWChem Pseudopotential Plane-wave Electronic Structure

The following modules are available to compute the energy, minimize the geometry and perform ab initio molecular dynamics using pseudopotential plane-wave DFT with local exchange-correlation potentials:

Fixed step length steepest descent,Car-Parrinello (extended Lagrangian dynamics),

with,LDA and LSDA exchange-correlation potentials (Vosko et al),( point) Periodic orthorhombic simulation cells,�Hamann and Troullier-Martins norm-conserving pseudopotentials, andModules to convert between small and large plane-wave expansions.

Page 38: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

NWChem Molecular Dynamics (MD)

The following classical molecular simulation functionality is available:

Single configuration energy evaluation Energy minimization Molecular dynamics simulation Free energy simulation (multistep thermodynamic perturbation (MSTP) or multiconfiguration thermodynamic integration (MCTI) methods with options of single and/or dual topologies, double wide sampling, and separation-shifted scaling)

Page 39: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

NWChem MD and Combined Classical and Quantum

The classical force field capabilities includes:Effective pair potentials (functional form used in AMBER, GROMOS, CHARMM, etc.) First order polarization Self consistent polarization Smooth particle mesh Ewald (SPME) Twin range energy and force evaluation Periodic boundary conditions SHAKE constraints Consistent temperature and/or pressure ensembles

NWChem also has the capability to combine classical and quantum descriptions in order to perform:

Mixed quantum-mechanics and molecular-mechanics (QM/MM) energy minimization and molecular dynamics simulation Quantum molecular dynamics simulation by using any of the quantum mechanical methods capable of returning gradients.

Page 40: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Minimal Input Example

Minimal input (all defaults)

¦§ ¨© § ª¬« ­® ¯ § ° ° ° ® § ¯ ±

²´³ µ ¶ µ ® ¯ § · ¶ ²« ³ « ­ ¸ ¸º¹ »¼ ±¾½ ® § ¯ ±

ª ³ µ ¿ µ ¸ À

Performs a closed-shell SCF on the neon atom

Page 41: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Files and RestartingFiles for Á à ĬŠÁ Ä

in the permanent directoryÆ ÇºÈ ÉËÊ ÌÍÏÎ ÐÑ Ò ÓÔ Õ×Ö ØÙ Ú Û Ú Ü ÚÝÞ ßËà áâÏã äåæ

Job-1 - files will be called

ç�è éê ë é çíì ê çíî ê ï ð

ñ òóô òõ ö

÷ öô øóõ öõ òúù ûüô ýËþ ñ öô ñ ý ø ö

ÿ ö� ø ö òô �� õ ö � � � � öõ û

�ó ñ ü ñ � õ ö � ü �ô óô � � ��� ÷ û� � öõ û

ò ó ñ � ñ �

Job-2 - converge the SCF tighter

� �� �� � � � �

� � �� � � � � ��� �� � ��� � � �� �� �

� � ��� � � � � ! �#" $ � � � �

� � � % � � �

Page 42: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Memory If the program terminates with error codes asking for more memory, add this line:

&' &( )* +, - - & . /10 2 3 4

&' &( )* , - - & . /15 687 4

For the amoxycillin runs (SCF):9: 9; <= > ; >@? A BC C D A ; E? A FC C 9 E

and also, for the case "memory-no-disk":GH IGJ K LM L1N J H O KJ K G L8P J QR R R R R R R R I LS J G L P J R

J TM

Attention: these are # of integrals: 800MBs

Page 43: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Geometry Input

UV WX V YZ [ \ ] ^ _ Y` aZ V _ ^ a ^ U` YZ W X `

b c c c

d c cfe gh h i j ce k l m g

d c n cfe gh h i ce k l m g

V ^ opq

rs tu s vw x y z { v| } y ~ ~ ~ � � } z rs y z { v|� � � �

� � ��� �� � � �� � � �

� � � ��� �� � �� � � �

s z �rs tu s vw x y z { v| } y ~ ~ ~ { z� y v y| { z r � u } vw {��

� u } vw {���� � � � � �� � �

� � � � � �� � � � � �� � � � �� �

s z �

s z �

Page 44: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Geometry Input: symmetry�� �� � �@� � � � � �@� � �       � �¡ � � �� � � � � �� � � � � �

¢ £ £ £

¤ £ ¥§¦ ¨© ¨ ª £ ¦ ¨ ¥ ¨

� �� � � � � � « ¬®­

¯ � °¢© £ ± �¡ � �

�� �� � �@� �   ² � � °� ³ ¥ ¦ ´ ´ ´µ � � ° ¥§¦ ¶· ´ µ

¸ � �� �@� � �

� �� � � �@� � ± ¹

« ª ¥ ¦ ¬ ¬ ¨º © µ ¥ £¦ £ ¶ ¦ ¶ ¥ ´ ¶ ¥ ¬ ¥

� � °

Page 45: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Turning off autosym, autoz and center

»¼ ½¾ ¼ ¿À Á  ½Ã Ä ¿ ½Å  ½ Ã Ä ¿ ½Æ Á¾  ½Ç ¼  ¿ ¼ À

È É É É

Ê É ÉÌË ÍÎ Î Ï Ð É Ë Ñ Ò Ó Í ÔÕ Ö× Ø Ù@Ú ÛÜ Ø

Ý É Þ ÉÌË ÍÎ Î Ï É Ë Ñ Ò Ó Í

ß Ö à

Page 46: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Geometry Input: ZCOORD -- Forcing internal coordinates

áâ ãä â åæ ç

è é êìë ê ê ê êí î ê ê êë ê ê ê êí î ê ê êìë ê ê ê êí î ê ê

ï ð êìë ñò óô í î ê ê ð êë õ õ êö í î ê ê êìë ö ó÷ ñí î ê ê

ï êìë ö ó ö ó í î ê ê ð êë õ÷ ö ñ í î ê ê ð êìë ñ ñ óø í î ê ê

ï ð êìë ö õ óù í î ê ê êë ÷ ê ó õí î ê÷ ð êìë ö ÷ ó ö í î ê ê

è é êìë ÷ ô ñ ñ í î ê÷ êë ÷ ù ù ô í î ê÷ êìë ÷ ô ñù í î ê÷

ï êìë ö ö ÷ ù í î ê ê êë ø ó ö ö í î ê÷ êìë ø ù ÷ ÷ í î ê÷

ï êìë ø ù ò ò í î ê÷ êë ô õ êù í î ê ê êìë ø ù ó ñí î ê÷

ï êìë ø ù ÷ ò í î ê÷ êë ø ó õ÷ í î ê÷ êìë ö ö ÷ ó í î ê ê

â ú ûü ü ü ä ý þ â è é ð è é û éÿ å ý ú � â ò ë ê ý ú áÿ åæ ãä ÿ ü ü ü

áâ ãä â åæ ç ý û� �ÿ å ü é ú é å é ý � ÿ å ý å â� � ã ãæ û� ã ú û ÷ ò ò ë ê ê æ � ã ú ÿ å ý ú å

â ú û

â ú û

Page 47: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Geometry Input: System fractional coordinates for periodic systems

�� � �� � � � �� � � �� � � � � � � � � � � � � � � � � � � � �� � � �� � � � �� � � � �� � ��� � ��� � �� � ! � � � � �� � ��� " ��� � �� � � � ��� � ��� � �� � � � � # � $ � � " � � � $ � � � � � $ � � � � �% & � � � & � � � & � � � % � � � � & � � � % � � & � � � � � % & � � � � � � � % & � � � � & � � � � & � �� � % � � � � � � � � & � �� � % � � � � & � � � � � �� � % & � � � � � � � � � �� � � � �

Page 48: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Hartree-Fock

FunctionalityInputWavefunctionsInitial MO vectorsDirect and semidirect algorithmsConvergence, files, and restarting

Page 49: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Hartree-Fock Functionality

Energies and gradients' ( )+* ,- .0/ , 1 - ) )2 34 5 6

7 8:9 ;=< > ?@ A B ? ;=C B@ DFE G ;=H G > A 9 ;=< I 9 @ < > A G @ J JK L MN O P

Q R:S T=U V W U XY Z [ X T=\ [ Y ] ^S Y U V Z _ Y ` `a bc d e

Analytic second derivatives (RHF and UHF)Finite point groupsResolution of the identity (energy)1500 functions are routinef g h+i j k l lm m n j op k q i orts um m v ki wr x vy z h z z o{ j o

Page 50: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Input

SCF input block, e.g.,|} ~��� ��� ��� ��� � � ~

� � �

Defaults� ��� � �� ��� �� ��� �� ��� � �� � � � � � � ��¡ �

� ¢� � � � �� £ � � � � � ¤¥ � �¡ � � ¡ � � �¥ ¡ � � £ ¦ ¡ ¥ � � � ¥ � § � ¡ ¦ � � � £¡ � � � ¦ �©¨ � � � ¡ �

� ª £ ¦ ¦ � � � £ � � � � � �� � � �� � « � § � ¡ ¦ � �� £

Page 51: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Simple Example

3B1 CH2 ROHF and UHF optimizations

¬­® ¯ ­ °±² ³ ´ µ °¶ · ³¸ ¹ ¹ ¹º ¹ »½¼ ¾¿ ¾ÁÀ ¹¼ ¾ » ¾¶² ¯ ¯­ °±²  ÃÅÄ­ ´ ÆÇ·¶ µ¶º È µ DZ ·±² ÉÀ à » ¬Ê ¸ È µ DZ ·±² ÉÀ à » ¬­ ´ Æ

¶  ËÌÊ °± µÎÍ È ­ ° Ê ­ ´ Æ° ·¶ Ï ¶  ˮ Í ° µ ¯ µÑÐ ­

¶  ËÌÊ ³ Ò ËÊ ­ ´ Æ° ·¶ Ï ¶  ˮ Í ° µ ¯ µÑÐ ­

Page 52: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Density-Functional Theory

FunctionalityInputXC functionalsGrid & Convergence options

Page 53: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

DFT Functionality-I

Gaussian function-based DFT • energies, • gradients and• second derivatives

finite symmetryLocal (LDA) and Gradient-Corrected (GCA) functionals for

• Closed-Shell systems and Open-Shell systems (2nd derivatives not ready for OS)

Page 54: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

DFT Functionality-II

Coulomb potential evaluated with 4-center, 2-electron integrals or charge density fitting (Dunlap's fit, 3-center integrals).

Exchange-correlation potential evaluated by numerical quadratures with possible use of an auxiliary fitting basis (not recommended)

• Exchange functionals: LDA, GCA and HF• Correlation functionals: LDA and GCA• Exchange-correlation combinations: any possible

combinations of what listed above (including Hybrid functionals).

Page 55: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Input

DFT input block, e.g.,Ó ÔÖÕ×Ø Ù Õ Ú

ÛÜ Ó

Defaults (similar to Hartree-Fock)Ý Þ�ß àá âã äå æ çè�é á ê êë ß ì ç�í á è ç ß å î ï ðñ ò

ó ô�õ õ ö ÷ø õ ù ú ö ûü ø ýþ�ÿ � � ÷ � � ��� �þ �� � ù � �þÿ õ ö þ�ÿ �ÿ � �ÿ ü ÷ ù

�� ü û � û� ø ü û � �

ó ù � �ÿ ü ÷ ù ø ú �ÿ � û �ÿ � û � ü �ÿ �ÿ � �ÿ ü ÷ ù

Page 56: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Minimal Input Example

Minimal input (all defaults)

� �� � ��� �� � � � � � � � � �

��� � � � � � � � � �� � � � � �! "# �%$ � � � �

� � � & � ' �

Performs a closed-shell N4 DFT calculation using the local density approximation on the neon atom (no fitting)

Keywords: ( )* +�, -. / 0 1 / * , 2 ( ) -3 ) 2* ) / 3 - 45 /

+ , 6 ) -7 2* ).

Page 57: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

XC functionals

BLYP calculation8 9;:< = >@? = A? B B C�D E

? F 8

B3LYP calculationG H;I JK L MN O�P Q J RS G

"Hybrid Functional

T

calculationU V;W XY Z [ V]\ Y Z [ ^`_ a b \ Z c@\ d d ^_ a e f@g W \ h ^ _ a

f�i j X \ k U

Page 58: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

XC functionals

lmmmmn op q

mm

mrs ot u v

wmmx n yz

wmmmop{ |p }~� �� �

wmmmop{ |p }~��� ��� �

lmmmo� �s n

lmmop{ |p �� � v �

wmmm�{ y �� � � q� �� ~� � q~ � n � �

lmm{ n op }�

lmmnp t v p� } �

lmm�s n

lmmnp t v p� � �

lmm� u � � }�

lmmz n op }�

lmmz np t v p� } �

lmmop{ |p � �

lmn� } � � v�

lmnp t v p� � �

lm�� ��� ���

lm�� ��� �� t n�

lm� �� yp t

� � v �������p s � xt v

Page 59: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

SCF/DFT Input Differences

Important differences between DFT and SCF

� ��� ��� �¡  �!¢ £!¤ ¥ � ¥ ¥¢ � ¥ ¦§ ¨ª© © « ¬ ­ ®

¯° ±² ° ³´ µ¶ ·° ¸ ¸ ¸ ¶ ° · ¹

º�» ¼ ½ ¼ ¾ » ± º» ¼ ½ ¼ ¿

·° À ½ º´ » ´ µ ¾Á ÂÃ Ä Å Á ÆÇ ±´ º ½ ³ » ÀÈ ¾ ¶ ° · ¹

º�» ¼ ½ ¼ ¾ÊÉ ¹ º» ¼ ½ ¼ ¾

·° À ½ º´ » ´ µ ¾Á ËÌ» Í ¼ ¼ Î Ï Á Æ Ç Ð ± Í À ± ² º Æ ½ ³ ³ ½ · ¯ ¾ ¶ ° · ¹

º�» ¼ ½ ¼ ¾¡Ñ É º» ¼ ½ ¼ ¾

·° À ½ º´ » ´ µ ¾Á ËÌ» Í ¼ ¼ Î Ï Á Æ Ç ÒÑ É Ó» · ¯° Æ ½ ³ ³ ½ · ¯ ¾ ¶ ° · ¹

³ » ¼ Ô ¹ Õ ³

Ö ×�Ø ÙÚ Û Ü¡Ý Þ ß Ü¡à áÚ â Û Þ á Ü¡ã à ä åæ çè é êë ì ê íïî ð

ñò óô ò õö ÷ø ùò ú ú ú ø ò ù û

ü�ý þ ÿ þ ø ùò � ÿ üö ý ö ÷ � � � �� û�� ø ò ù û

û � õ ø ñö ÿ û � ÿ ù ò ø ò ù û

õ ý þ � û � õ

Page 60: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Grid Options

Numerical integration keywords and targets using Mura-Knowles radial and Lebedev angular quadratures: � � � � � �� � � � � � �� �� ��� � �� �

� ! " #$ % � &' � $ ( ) " )* � +, -�. / 01 2

- 34 5 67 8 - 9 : - 81 9 5 :; - <= >�? @ AB C >ED F AB GH I

> FH C JK L > F LNM D C D M > OP Q�R S TU V

Q WX Y Z[ \ Q ] W \N^ _ Y _ ^ Q `a b�c d ef g

Addition quadrature choices, e.g.,h ikj l mn o h pq rs t r p h oq r l pu h

h ikj l mn o h v p w p h px yz { { l p u h

( = G98 fine)

Page 61: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Modifying Accuracy

Controlling accuracy

|~} ��� �� ��� �

��� �� � ��

�� ��� � � �E� �� �� �� � � � �

�~� � � ¡ �¢

£�¤ ¥§¦ ¨© ª« ¦ ¬ £�¤ ¥­ ¨ ®¤ ¯±° ²´³ ¯ µ

¶ ·¹¸ ºN» ¼½¾ ¿ ¸ ½À À Á  Áà  ¿  ÁÄ Å ÆÀ Ç È Å ½ ÇÀ Á ¿ ÂÉÊ Ë Â ÁÉÀ à ½ ¼ Ì

Í ÎÏ�Ð ¼ ¸ ¸ Ñ ÅÒ ÌÔÓ ÕÖ Ö ×ÙØ Ú Û

ÜÞÝ ßNà á âã ä åæ

ç è é ê éë

ì�í î§ï ðñ òó ï ô ñ ó ó õí ö î ÷ ø

When to change it?

ù ú û ü üþý ÿ� ��� ÿ û ÿ � ü� �� � � � � � ý � � ÿ

� � �� �� � �� ��� �� � � � � � � � � �� � � � ��� � � � � ���� �� � ! � �" � �� � � �"�#

$ %'& ( )* +�, -. / * * , 021 3 - /4 5 / . - 0 6 4 3 37 -4 38 * , , 9

Page 62: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Wavefunctions

We are optimizing density defined by a Kohn-Sham determinant (Kohn-Sham orbitals)

Default is closed-shell LDA/GCA

:; <>= ?

Must change multiplicity if Open Shell calculations are desired

@ ACB D EF G B H D IJ @

We do not have "RO-DFT"

Page 63: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Simple Example

E.g., 3B1 CH2 LSDA optimization

KL MN L OQP R S T U OQV W S

X Y Y Y

Z Y []\ ^_ ^ ` Y \ ^ [ ^

V RN N L O P R a bdc

L T ef WV UV

Z g U fP WP R h ` b [ Ki X g U fP WP R h ` b [ K

L T ee j O i N S g O h i L T e

O WV k e j O Ml O UN Unm L

Page 64: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

Semidirect and Direct

Semidirect DFT is the default. Available disk is used to cache integrals:

2-center 2-electron "cd basis" integrals

2-center overlap "xc basis" integrals

3-center 2-electron integrals

To turn off disk access: op q p

To force fully direct:

rs tu v w

You can check the DFT-SCF integral caching with the same SCF block than for HF. NWChem is not very good in guessing available resources!

Page 65: High Performance Computing in COMPUTATIONAL CHEMISTRY - ScicomP

ECCE 3.1Is another good piece of software developed at EMSLA very powerful free builderA very powerful basis function selectorA nice job management systemA powerful tool for information sharing

Let's see it LIVE!