Exploring Algorithm SpaceVariations on the Exchange Theme

Daniel M. Zuckerman

Department of Computational Biology

School of Medicine

University of Pittsburgh

Goal

• More efficient atomistic sampling, consistent with statistical mechanics

• Take care with the meaning of “efficiency”

Outline

• Protein fluctuations in biology

• Replica exchange simulation -- a second look

• Resolution exchange simulation– Initial results– How to approach larger systems?

• Exchange Variants

• Assessing Sampling

Transport Proteins Fluctuate - I

Transport Proteins Fluctuate - II

Motor Proteins Fluctuate

Signalling Proteins Fluctuate

Conformational Change Requires Fluctuation

• Either ligand leaves free-like bound structure or ligand binds bound-like free structure (or nearly so)

free

ligand

boundbound

free

ligand

Biology Take-Home Message

• Fluctuations are ubiquitous and essential– They are not a sideshow; they are the show!

• Experimental structures are only snapshots -- just the beginning of the story

Key for medicinal chemists especially

• Drug design via “docking” is a key practical use of molecular modeling– Typically, drug candidate molecules are fitted into

static protein structures– Common lament: need to know protein fluctuations

• Necessary for free energy calculations– e.g., binding affinity

Questioning low RMSD in MD

• Is 1.3 Å right? What is nature’s avg RMSD???

RM

SD

time

1 - 1.5 Å

A Physical View of Fluctuations

• Rough, high-dimensional energy landscape

x

U

Simplest Physical Picture: Bistable system

• Most phenomena can be understood from a toy picture

x

U x

t

x

p

Defining the Problem

• We want a good sample of p(x)– “Equilibrium distribution”– “Complete canonical ensemble”– Probability density function– x is a vector in configuration space -- i.e., vector of

all coordinates: (x1,y1,z1, x2,y2,z2, …)

• In English: We want a set of structures distributed according their probability of occurrence at the specified temperature

• Hard because we access p(x) only indirectly– Blind person feeling elephant

It’s NOT optimization/search/minimization!

• However, undiscovered sampling algorithms may be similar to search algorithms!

The Problem with the Problem

• It’s too hard!!

• Present methods, implemented on standard computers, are inadequate by orders of magnitude -- think timescales– Simulations access nsec - sec timescales– Proteins fluctuate on nsec - sec timescales– 3-9 orders of magnitude short!

• Today: taking steps toward the solution

Theoretical/Computational Basics

• Boltzmann factor

• “Forcefield” (potential energy function)– Configuration vector to real number

– Terms not shown: sterics, electrostatics, four-body (e.g., dihedral)

€

p(x)∝ exp −U(x) kBT[ ]

€

U(x) = 12 k1 l1 − l10( )

2+ 1

2 k1 l2 − l20( )2

+ ...

+ 12

ˆ k 1 θ1 −θ10( )2

+ 12

ˆ k 1 θ2 −θ20( )2

+ ...

+ ...

l1

l2

l3

1

2

l1

U

l10

Exchange Schemes

• Original idea: use higher temperature to facilitate barrier crossing [Swendsen, 1986]– Barriers are the real problem

• Arrhenius law: – rate ~ barrier’s Boltz. fac.

x

U

€

k ∝ e−ΔU / kBT

x

U

Ufwd

Exchange Ladder• High-temperature hops percolate down via configuration swaps (

temperature swaps)– Independent sim’s with occasional exchange attempts

t

hot

300K

Exchange attempts

How does replica exchange work?

• It’s just Monte Carlo

• Physics view of Metropolis– Accept trial move: xold xtry with min[1,exp(-U/kT)]

– U=U(xtry) - U(xold)

• Probability view:– Accept with min[1, prob(try)/prob(old)]

Exchange as simple Monte Carlo

• Exchanges are only attempted in pairs

• Two independent simulations– Probability for combined

system is simple product: p = p1*p2

– Metropolis criterion: min[1, ptry / pold]

hot

300K

T1

T2

€

pold = p x1;T1( ) × p x2;T2( )

€

ptry = p x2;T1( ) × p x1;T2( )

time

Does replica exchange really help?• For a given investment of CPU time, is better fixed-T

sampling achieved?– Compared to equal time direct simulation -- e.g., for a 20-

level ladder, a simulation 20 times as long

• To my knowledge, no convincing evidence yet• Key: Sampling limited by top level• Worry 1: High T does not help with entropic barriers

– Hard-to-find low energy pathways

• Worry 2: High T not so helpful for low barriers– Simulations and experiments suggests barriers are low– Even for 600K simulation, only moderate speedup

• 2kT 2.7 speedup• 4kT 7.4 speedup• 6kT 20.1 speedup

€

exp −ΔU /kB 600K( )[ ]

exp −ΔU /kB 300K( )[ ]

Summary of Concerns re Replica Exchange

• Efficiency limited by top level (highest T)

• Highest T may not be fast enough for biomolecules– High T does not affect entropic barriers– Energy barriers may be low

• Should work for sufficiently high energy barriers

Can replica exchange be fixed?

• Yes

• Two improvements today

• Plus a sketch of other variants

Improvement (1): Pseudo-exchanges

• Key: Need complete sampling top level (highest T)

• Work from top down …if we can “pseudo exchange”

hot

300K

hot

300K

x

U

time

Top level can be generated with multiple simulations

Anatomy of a Pseudo-Exchange• Point 1: Normal exchanges need not be performed at

identical intervals– Not required in derivation of Metropolis criterion– Imagine one fast CPU & one slow CPU

• Point 2: Imagine top-level CPU is extremely fast– Long intervals no correlations equil. dist.– Alternatively, view top level as “perfect” Monte Carlo equil.

dist.

• Conclusion: no need to continue top-level sim. from exchanged configuration can pull randomly each time from top level

fast

slow

Two Ways to Use Pseudo Exchange

• Same ladder• More widely spaced ladder

– Lower acceptance OK since trials are cheap (serial)– No need for frequent attempts in parallel since few high T hops

• Essentially guaranteed to be more efficient than standard parallel replica exchange.

hot

300K

hotter!

300Ktime

Top-down test: Di-leucine Peptide

• Two amino-acid peptide with two main conformations• 50 atoms (144 degrees of freedom)• Langevin dynamics; GBSA continuum solvent model

– ALL SIMULATIONS

Example: Di-leucine via two-level ladder

• Di-leucine, a 50-atom peptide: two levels only

T=500K, shuffled

T=298K using pseudo-exchanges with shuffled 500K trajectory

T=500K

Not really efficient

• Boost to 500K only modestly increases hop rate– In 300nsec: 488 hops at

500K vs. 300 at 298K– Barriers are too low

• Ordinary trajectories shown (no exchange)

• Still should be better than parallel exchange sim.

T=500K

T=298K

Improvement (2): Resolution Exchange

• Canonical sampling in detailed model

Coarse

Detailed

Dreams of multi-scale modeling

• (At least) since Levitt and Warshel, Nature (1975)

• Warshel -- free energy for detailed model based on coarse-grained reference (1999)

• Brandt and collaborators -- complex multi-level formulation

• Vendrusculo and coworkers -- ad hoc addition of atomic detail onto coarse structures

• Resolution exchange is concrete, simple and general

Improvement (2): Resolution Exchange

• Qualitative picture

COARSE

detailed

Exchange attempts

time

Implementing Resolution Exchange

• Need – Formulate as exchange process– Derive acceptance criterion

• Coarse model will use subset– Detailed (regular) model

x = (l1,l2,l3, …, 1, 2, …, 1,2, …)

– Coarse model is subset, e.g., = (1,2, …)

– Arbitrary potential Ucoarse() -- i.e., pcrs() = exp[- Ucoarse() / kT ]

– Simply exchange common coords.

l1

l2

l3

1

2

2

1

Key Point: Subsets are natural for coarse models

• Examples– Dihedrals only (fixed angles, lengths)– Backbone coordinates only– Side-chains by beta carbons

• Proteins are branched chains

l1

l2

l3

1

2

2

1

Res-Ex Metropolis Criterion • The trial exchange

– From: (la,aa) and b [“old”]

– To: (la,ab) and a [“try”]

• Metropolis: min[1, ptot(try) / ptot(old)]

• Final criterion– min[1,R]

€

R =pdtl la,θa ,φb( ) × pcrs φa( )pdtl la,θa ,φa( ) × pcrs φb( )

detailed

coarse

time

CANONICAL SAMPLING FOR ALL COORDS, ALL LEVELS!!!

Downside of Res-ex: more work!• The ladder needs to be engineered• Analogy to replica exchange: limit on difference

between models– simple solution (later)

• Implicit solvent: still hard and important problem

COARSE

detailed

Exchange attempts

time

You can recycle!

• Top-down approach (pseudo-exchanges) permits old trajectories to be exchanged into new– New temperature– New forcefield

• Same or different numbers of coordinates

• Minimal CPU cost, if original trajectory already crossed barriers

Initial Results

• Still early stages

• Verifying the algorithm

• Efficiency in a 50-atom di-peptide

• [A penta-peptide]

• Reduced models of proteins are reasonable

central dihedral

Algorithm Check: Butane

• Butane is C4H10

Line is from direct sim.

Real Molecular Test: Di-leucine Peptide

• Two amino-acid peptide with two main conformations• Exchange all-atom to united-atom (GBSA “solvent”)

– eliminate non-polar H– 50 atoms to 24 “united atoms”

united atom

Initial Results: Res-ex really works

• CPU Savings: Factor of 15 (including united-atom cost)

Leucine free energy difference via Res-Ex• G measures if correct time spent in each state• Increased precision indicates speedup (first report??)• Cost of united-atom simulation included in graph

From long brute-force sim.

Comments

• Results obtained from a two-level ladder

• Faster sampling should be possible with more levels– Requires forcefield engineering

• Can use higher temperature also– AND/OR softer parameters

Spin Systems Too

• Absolute spins

• … or block spins as coarse variables ()– Relative spins as detailed coordinates (+–)

€

↑,↓,↓,↓( ), ↑,↓,↓,↑( ), ↑,↓,↑,↑( ), ↓,↓,↑,↑( ){ }

€

;−,+,+,+( ), ⇓;−,+,+,−( ), ⇑;+,−,+,+( ), ⇑;−,−,+,+( ){ }

How do we progress from here?

• Need an exchangeable ladder– But we have design criteria

• Top level needs to explore important fluctuations

A Possible Ladder

1. Backbone only (Go interactions)

2. Backbone + beta-carbon “side-chains”

3. United groups (quasi rigid)

4. United atom

5. All atom

• Each level omits specific internal coordinates

• Other levels may be needed

Key Point: Resolution Difference is Tunable

• Can (de)coarsen part of a molecule at a time– e.g., groups of 3 residues

• Initial results: Met-enkephalin– Less overall CPU time for de-coarsening one residue at a

time vs. whole molecule (for a fixed number of “hops”)– Order of magnitdue more efficient than single-step

decoarsening– Poster by Ed Lyman

all coarse

all detailed

Resolution Exchange Variants

• Switching– Coarse sim. as MC trial

• Decorating– Sample coarse and detailed coordinates separately– Re-weight by true Boltzmann factor

• “Algorithm Space” has not been fully sampled!

coarse

detailedt

€

pfull l,θ,φ( ) = pcrs φ( ) × paddl l,θ( )

Annealing based approach: replica exchange variant

• Can be re-weighted for canonical sampling at low T [Neal, 2001]

hot

cold

Equivalent to Jarzysnki (exactly)

=0 =1

So you’ve got a new method …How do we judge sampling quality?

• Without enumerative technique, generally impossible to guarantee full sampling– Can’t know about unseen regions

• Best we can hope for: proper distribution among states visited– Very difficult [new approach under study]

• We can show: lack of convergence, even among visited states

Previous Approaches

• Stare at RMSD vs. time plot

• Principal components– Mostly 2D visual inspection– How to quantify?

• Van Gunsteren and co-workers: cluster counting– Fails to account for relative populations

New Approach: cluster, then classify1. Cluster via (e.g.) RMSD threshold2. Choose reference structure from each cluster3. Re-analyze trajectory, classifying (binning) each

structure with closest reference• Classification is statistically “rigorous”• Simple 1D histogram results• Easy to implement for large proteins

p

Met-enkephalin: the old view

• Is it converged?

Evolution of Distribution

2nsec 4nsec

10nsec 50nsec

198nsec

Self-referential comparison: 1st vs. 2nd half4 nsec 20 nsec

100 nsec 198 nsec

Conclusions

• Sampling matters -- life runs on fluctuations• Parallel replica exchange has key limitations• Resolution exchange (+ top-down) offers hope

– Good results using only two levels, single T– Much work to be done in completing a ladder– BUT: a concrete path to ever-increasing efficiency

• Res-ex applies to molecular and spin systems and …?

• Algorithm space is large -- many variants• Semi-systematic convergence analysis

Acknowledgments

• Edward Lyman

• Marty Ytreberg, Svetlana Aroutiounian

• Ivet Bahar, Robert Swendsen, Hagai Meirovitch, Carlos Camacho, Eva Meirovitch

• Funding– NIH– Depts of Computational Biology, Environmental &

Occupational Health

A more complete picture

• In configuration space

If barriers are low, why are dynamics slow?

• Too many barriers!

x

U

Buildup Schemes

• Stochastic growth of molecule– Not dynamics– Re-weighting using Boltzmann factor and

distribution used for construction

Multiple-histogram view of Replica Exchange

• Temperature increments constructed to minimize overlap– Just enough to permit exchange– WHAM just fills out high-energy tail of coldest distribution

E

p hottestcoldest (target)

Top-down: how much CPU time saved?

• Optimal: time spent at low T tiny– Cost is same as for high T

• Down-side: limited by Arrhenius factor

Top-down vs. Parallel: Rough Comparison• Typical standard replica exchange

– 20 levels tuned to 20% exchange-acceptance ratio– 1 nsec each (106 snapshots/energy calls)– No need to attempt frequent exchanges due to

relatively slow top-level dynamics/hopping

• Compare to top-down + pseudo-exchange– 20 levels; only top level is 1 nsec

• Attempt exchange every 10 steps• 104 steps 200 acceptances (many hops!)

– Also can use higher T ladder (lower acceptance)

Systematically checking convergence

• Ambiguous results– Energy– RMSD vs. starting config

Can we afford to climb down the ladder?

• How many energy calls?– Depends on desired ensemble size:

Say 104

– Assume 100 ladder levels; only 1% exchange acceptance (conservative)

• Assume 108 energy calls– Top level: 9*107 (cheapest!)– 105 calls per lower level– Attempt every 10th step 104

attempts 100 exchanges (hops)– Almost every exchange will yield a

new basin: good sampling!

hot

300K

Some Resolution Exchange Statistics

• Di-leucine (UA to AA; OPLS)– Modified: 0.16% acceptance– Unmodified: 0.14%– Incremental (one residue at a time): ~2.5% (UA to mixed),

~0.25% (mixed to UA)

• Met-enk (UA to AA; OPLS)– Whole molecule (75 atoms to 57): 0.09% acceptance --

modified UA– Incremental (one residue at a time): so far, 10% acceptance

for 3/5 levels -- modified UA -- ongoing

• Comparison: Replica Exchange: 15-20%• Met-enk (top-down temperature exchange)

– ~2% T ladder: 200K, 270, 367, 505, 950, 1305, 1810– Comparison: max 700K with 15% exchange

What’s wrong with NMR “ensembles”?• Determined by search/minimization approaches

• Peak, not tails, of distribution

x

p

x

p

Need proper distribution• 10 equi-prob regions, e.g.

•10 structs: 1 per region•20 structs: 2 per region

Hope at the top of the ladder• Reduced models capture large-scale fluctuations

tendimistat

Closer look at top (most reduced) level

• Inexpensive “smart” models can be built with lookup tables (dihedrals/orientations)– Ramachandran propensities– Peptide-plane sterics– Backbone H-bonding– Beta carbon (hydrophobicity)

• Go interactions can stabilize any model– Canonical sampling preserved by res-ex

criterion

[Dickerson & Geis]

More Go-model fluctuations: ferrodoxin

ferrodoxin

More Go-Model Fluctuations: Protein G

Protein G

Sampling Strategies [to get p(x)]

• “Direct” dynamics

• [Build-up schemes]

• Exchange dynamics– Temperature– Resolution

Direct Dynamics

• Dynamical trajectory x(t) histogram p(x)

• Varieties of dynamics – All embody U(x); f = -dU/dx; Boltzmann dist.– Newtonian (“Molecular Dynamics”)– Langevin/Brownian -- fully stochastic

• TODAY’S DATA

– Monte Carlo -- fully stochastic (dynamical??)

• All lead to Boltzmann distribution

€

p(x)∝ exp −U(x) kBT[ ]

Research

• Free energy calculations (fluctuations)– F, absolute F

• Rare dynamic events / Path sampling (fluctuations)– Theory and molecular applications

• Equilibrium sampling– Today

• Non-traditional coarse-grained model design– Discretization; different “resolution levels”

• Overall Goal: Make biologically relevant desktop computations possible– Stay true to statistical mechanics