and something else…

Mikkel Strange (but presented by Karsten W. Jacobsen)

European Centre of Excellence (CoE)

Overview of current projects• Creating a big database for:


DFT + Excited states + Molecular dynamics 
(writing parsers)

• Error bars in DFT calculations (finite basis set) 
Useful for comparing calculations, or not?

• Self-consistent 
hybrid functionals

Outline

• Brief overview of the NOMAD project.

• “Parsers” that converts output from electronic structure codes to a common format

• “Error bars” from basis and k-point sampling

• Self-consistent hybrid functionals in GPAW

NOMAD overview

• Virtual reality (fx: exciton (r,r’)) • Remote visualisation 


(structures, wave-functions, etc)

• Stores your data for at least 10 years

• Advanced analysis tools 
(machine learning etc)

3,027,865 calculations today

Nomad: Producing dataRepository:

output files from all codes are

welcome (file.gpw)

Archive:common data

format (contains

everything that is in the output files)

50 most cited codes (GPAW, VASP, Gaussian,


ASAP…)

Conversion

http://nomad-repository.eu/cms/

Big data analytics Materials Encyclopedia

…

Nomad: Producing dataParsers done at DTU for: • GPAW | works • MOPAC | works • ASAP | works • ATK | not started • GROMACS | created

All data should be parsed: Structures Energies

Electron density Wave functions

SCF iteration time Force field information

Constraints …Start uploading your

favourite GPAW gpw filescome June 15th!

Error bars project

Next step: something about the error due to:• finite basis set • k-point sampling • other numerical settings?

Should be useful for judging the quality of data in the
NOMAD database

RESEARCH ARTICLE SUMMARY◥

DFT METHODS

Reproducibility in density functionaltheory calculations of solidsKurt Lejaeghere,* Gustav Bihlmayer, Torbjörn Björkman, Peter Blaha, Stefan Blügel,Volker Blum, Damien Caliste, Ivano E. Castelli, Stewart J. Clark, Andrea Dal Corso,Stefano de Gironcoli, Thierry Deutsch, John Kay Dewhurst, Igor Di Marco, Claudia Draxl,Marcin Dułak, Olle Eriksson, José A. Flores-Livas, Kevin F. Garrity, Luigi Genovese,Paolo Giannozzi, Matteo Giantomassi, Stefan Goedecker, Xavier Gonze, Oscar Grånäs,E. K. U. Gross, Andris Gulans, François Gygi, D. R. Hamann, Phil J. Hasnip,N. A. W. Holzwarth, Diana Iuşan, Dominik B. Jochym, François Jollet, Daniel Jones,Georg Kresse, Klaus Koepernik, Emine Küçükbenli, Yaroslav O. Kvashnin,Inka L. M. Locht, Sven Lubeck, Martijn Marsman, Nicola Marzari, Ulrike Nitzsche,Lars Nordström, Taisuke Ozaki, Lorenzo Paulatto, Chris J. Pickard, Ward Poelmans,Matt I. J. Probert, Keith Refson, Manuel Richter, Gian-Marco Rignanese, Santanu Saha,Matthias Scheffler, Martin Schlipf, Karlheinz Schwarz, Sangeeta Sharma,Francesca Tavazza, Patrik Thunström, Alexandre Tkatchenko, Marc Torrent,David Vanderbilt, Michiel J. van Setten, Veronique Van Speybroeck, John M.Wills,Jonathan R. Yates, Guo-Xu Zhang, Stefaan Cottenier*

INTRODUCTION:The reproducibility of resultsis one of the underlying principles of science. Anobservation canonly be accepted by the scientificcommunity when it can be confirmed by inde-pendent studies. However, reproducibility doesnot come easily. Recent works have painfullyexposed cases where previous conclusionswerenot upheld. The scrutiny of the scientific com-munity has also turned to research involvingcomputer programs, finding that reproducibil-ity depends more strongly on implementationthan commonly thought. These problems areespecially relevant for property predictions ofcrystals and molecules, which hinge on precisecomputer implementations of the governingequation of quantum physics.

RATIONALE:Thiswork focuses ondensity func-tional theory (DFT), a particularly popular quan-

tum method for both academic and industrialapplications. More than 15,000 DFT papers arepublished each year, and DFT is now increas-ingly used in an automated fashion to buildlarge databases or applymultiscale techniqueswith limited human supervision. Therefore, thereproducibility of DFT results underlies thescientific credibility of a substantial fraction ofcurrent work in the natural and engineeringsciences. A plethora of DFT computer codesare available, many of them differing consid-erably in their details of implementation, andeach yielding a certain “precision” relative toother codes. How is one to decide formore thana few simple cases which code predicts the cor-rect result, and which does not? We devised aprocedure to assess the precision of DFT meth-ods and used this to demonstrate reproduci-bility among many of the most widely used

DFT codes. The essential part of this assessmentis a pairwise comparison of a wide range ofmethodswith respect to their predictions of theequations of state of the elemental crystals. Thiseffort required the combined expertise of a largegroup of code developers and expert users.

RESULTS:We calculated equation-of-state datafor four classes of DFT implementations, total-ing 40 methods. Most codes agree very well,with pairwise differences that are comparableto those between different high-precision exper-

iments. Even in the case ofpseudization approaches,which largely depend ontheatomic potentials used,a similar precision can beobtainedaswhenusing thefull potential. The remain-

ing deviations are due to subtle effects, such asspecific numerical implementations or the treat-ment of relativistic terms.

CONCLUSION: Our work demonstrates thatthe precision of DFT implementations can bedetermined, even in the absence of one absolutereference code. Although this was not the case 5to 10 years ago,most of the commonlyused codesand methods are now found to predict essen-tially identical results. The established precisionof DFT codes not only ensures the reproducibilityof DFT predictions but also puts several past andfuture developments on a firmer footing. Anynewly developedmethodology can nowbe testedagainst the benchmark to verify whether itreaches the same level of precision. NewDFT ap-plications can be shown to have used a suffi-ciently precise method.Moreover, high-precisionDFT calculations are essential for developing im-provements to DFTmethodology, such as newdensity functionals, whichmay further increasethe predictive power of the simulations.▪

RESEARCH

SCIENCE sciencemag.org 25 MARCH 2016 • VOL 351 ISSUE 6280 1415

The list of author affiliations is available in the full article online.*Corresponding author. E-mail: kurt.lejaeghere@ugent.be (K.L.);stefaan.cottenier@ugent.be (S.C.)Cite this article as K. Lejaeghere et al., Science 351, aad3000(2016). DOI: 10.1126/science.aad3000

Recent DFTmethods yield reproducible results.Whereas older DFT implementations predict different values (red darts), codes have now evolved tomutual agreement (green darts).The scoreboard illustrates the good pairwise agreement of four classes of DFT implementations (horizontal direction)with all-electron results (vertical direction). Each number reflects the average difference between the equations of state for a given pair of methods,withthe green-to-red color scheme showing the range from the best to the poorest agreement.

ON OUR WEB SITE◥

Read the full articleat http://dx.doi.org/10.1126/science.aad3000..................................................

on

June

1, 2

016

http

://sc

ienc

e.sc

ienc

emag

.org

/D

ownl

oade

d fr

om

Delta DFT:DFT codes can be converged to good agreement

Error bars projectSome simple questions

• Is convergence with k-points and 
basis set independent?

• Can we give error estimates?

Error bars project

1 225

H

LiH

3 225

Li

LiF

11 225

Na

Nacl

19 225

K

KCl

37 225

Rb

RbCl

55 225

Cs

CsCl

87

Fr

4 216

Be

BeS

12 225

Mg

MgO

20 225

Ca

CaO

38 225

Sr

SrO

56 225

Ba

BaO

88 225

Ra

RaF

21 225

Sc

ScS

39 225

Y

YN

57-71

La-Lu

Lanthanide

89-103

Ac-Lr

Actinide

22 225

Ti

TiN

40 225

Zr

ZrC

72 225

Hf

HfC

104

Rf

23 225

V

VC

41 225

Nb

NbC

73 225

Ta

TaC

105

Db

24 225

Cr

CrN

42 187

Mo

MoC

74 187

W

WC

106

Sg

25 225

Mn

MnS

43 14

Tc

TcO2

75 221

Re

ReO

107

Bh

26 225

Fe

FeO

44 136

Ru

RuO

76 187

Os

BOs

108

Hs

27 225

Co

CoO

45 136

Rh

RhO

77 136

Ir

IrO

109

Mt

28 225

Ni

NiO

46 131

Pd

PdO

78 131

Pt

PtO

110

Ds

29 216

Cu

CuBr

47 225

Ag

AgCl

79 224

Au

Au2S

111

Rg

30 216

Zn

ZnO

48 225

Cd

CdO

80 225

Hg

HgF

112

Uub

31 216

Ga

GaP

13 216

Al

AlP

5 194

B

BN

49 216

In

InP

81 225

Tl

TlCl

113

Uut

6 225

C

TiC

14 216

Si

SiC

32 160

Ge

TeGe

50 62

Sn

SnS

82 225

Pb

PbS

114

Uuq

7 225

N

VN

15 216

P

InP

33 216

As

GaAS

51 216

Sb

InSb

83 215

Bi

BiF5

115

Uup

8 225

O

CdO

16 216

S

ZnS

34 186

Se

CdSe

52 216

Te

ZnTe

84 225

Po

PoO

116

Uuh

9 225

F

NaF

17 216

Cl

CuCl

35 225

Br

KBr

53 225

I

LiI

85

At

117

Uus

10

Ne

2

He

18

Ar

36

Kr

54

Xe

86

Rn

118

Uuo

1

2

3

4

5

6

7

1 IA

2 IIA

3 IIIA 4 IVB 5 VB 6 VIB 7 VIIB 8 VIIIB 9 VIIIB 10 VIIIB 11 IB 12 IIB

13 IIIA 14 IVA 15 VA 16 VIA 17 VIIA

18 VIIIA

57 176

La

LaCl

58 225

Ce

CeN

59 225

Pr

PrN

60 225

Nd

NdN

61 164

Pm

PmO2

62 225

Sm

SmN

63 225

Eu

EuN

64 225

Gd

GdN

65 225

Tb

TbN

66 225

Dy

DyN

67 225

Ho

HoN

68 225

Er

ErN

69 225

Tm

TmN

70 225

Yb

YbN

71 225

Lu

Lu

89 176

Ac

AcCl

90 225

Th

ThC

91

Pa

92

U

93

Np

94

Pu

95

Am

96

Cm

97

Bk

98

Cf

99

Es

100

Fm

101

Md

102

No

103

LrZ Spacegroup

Symbol

Binary solid

notused

Periodic Table of Chemical Elements for Binary solids

Codes: GPAW | PW | DTU

VASP | PW | TU Graz Exciting | LAPW | HU Berlin

AIMS | NAO | FHI Berlin

Properties:Structures Energies

Band gaps

Functional: PBE and LDA 90 Binary solids 73 Elemental (from ∆DFT) -> ~5000 calculations

(+ random systems from the NOMAD database)

Error bars projectTotal energies (binaries): light vs really tight basis

AIMS

-12 -10 -8 -6 -4 -2 0Etot (Ec=400eV) (eV)

-12

-10

-8

-6

-4

-2

0

E tot

(Ec=

1600

eV) (

eV)

kdens=4, error: 7.08%kdens=6, error: 7.07%kdens=8, error: 7.07%

GPAW

Error from k-point sampling looks “small” 
compared to basis set

Error bars projectTotal energies (binaries)

Things really improve at Ecut = 600eV!

MAE: Mean absolute error MAXAE: Max absolute error MAPE: Mean absolute percentage error

Error bars projectCohesive energies (binaries)

Energy differences benefit from error cancelation

MAE: Mean absolute error MAXAE: Max absolute error MAPE: Mean absolute percentage error

Error bars project

N

O

F

Elemental solids

N, O and F are the bad guys!

Error bars projectBinaries without


N, O and F

N, O and F are the bad guys!

Binaries

Error bars project

400 800 1200Ecut (eV)

1

10

100

1000

E tot

abs

erro

r (m

eV)

Mean | binariesMax | binariesMean | monomersMax | monomers Max abs error for binaries

and elemental solids similar

Simple error 
estimate suggestion:simply use the error for the worst element.

Error bars projectIs convergence with basis set and k-points independent?

E(kpts, basis) = F1(kpts) + F2(basis)

Å Å

400 eV 1600 eV

4 5 6 7 8kdens

-0.15

-0.1

-0.05

0

0.05

∆E t

ot(1

600e

V) - ∆

E tot

(400

eV) (

eV)

�Etot

(⇢k

) = Etot

(⇢k

= 8)� Etot

(⇢k

) Independent or not?

Error bars projectAlso need to look at: - band gaps - structures…

Self-consistent hybrid functionals (HSE, PBE0, …)

Challenges:• Slow (compared to GGA) • Convergence is 


supposed to be difficult

Currently in GPAW• Non-SC with PW and k-points • SC with FD Gamma point only

VASP tells you to use: 
Damped “molecular dynamics”

Self-consistent hybrid functionals (HSE, PBE0, …)

Challenges:• Convergence is difficult

hg|ĥ|g0iHamiltonian in PW basis

We are trying with damped molecular dynamics but also brute force:

ĥHF = t̂+ v̂H + v̂F + ...

Should be stable, but limited in supercell size

Thank you for the attention!