genetic algorithms and genetic programming for multiscale modeling
Post on 20-Jan-2015
3.089 Views
Preview:
DESCRIPTION
TRANSCRIPT
This work is supported by the AFOSR F49620-00-0163 and FA9550-06-1-0096, NSF DMR-99-76550 and DMR 03-25939 (MCC), DOE DEFG02-91ER45439 (MRL), and CSE fellowship, UIUC.
Genetic Algorithms and Genetic Programming for Multiscale Modeling: Applications in Materials
Science and Chemistry and Advances in Scalability
Kumara SastryIndustrial and Enterprise Systems Engineering
University of Illinois at Urbana-Champaignksastry@uiuc.edu
Outline
Background and motivation
Objective and proposed approach
Genetic algorithms (GAs) and Genetic programming (GP)
Applications:Multi-timescale modeling of alloy kineticsMultiscaling quantum-chemistry simulation
Advances in scalability:Population sizing for GPScalable GP designScalability limits of multiobjective GAs
Summary & Conclusions
Multiscaling is Ubiquitous in Science & Engineering
Many phenomena are inherently multiscalePhysics, chemistry, biology, materials science, biotech, etc.,
Need accurate and fast modeling methodsTo advance science and expedite synthesisAccommodate different system sizes and time scales
Time (picoseconds-minutes) & Space (Nanometers-Meters)
Existing modeling methodologies effective on single scaleElectronic structure calculations (Angstroms)Finite-element method (micro- to millimeters)
Bridging methods for effective multiscaling is non-trivialCurrent multiscale methods fall short
GAs and GP for Multiscale Materials Modeling
Need accurate and efficient multiscale methodsSparsely sample low-level modelsDevelop custom-made constitutive relationship
Use GAs & GP for bridging higher- and lower-level modelsRobust, competent, and efficientHandle multiple objectivesIndependent of representation
[Sastry et al (2004) IJMCE]
Genetic Programming
f = 0.5
f = 0.9f = 0.95
f = 0.1
f = 0.01
f = 0.11
Applications: Multiscale Modeling in Materials Science and Chemistry
Accuracy of modeling depends on accurate representation of potential energy surface (PES)
Both values and shape matter
Ab initio methods:Accurate, but slow (hours-days)Compute PES from scratch
Faster methods:Fast (seconds-minutes), accuracy depends on PES accuracy Need direct/indirect knowledge of PES
Known and unknown potential function/methodMultiscaling quantum chemistry simulations [Sastry et al (2006) GECCO; Sastry et al (2007) MMP]
Multi-timescaling alloy kinetics [Sastry et al (2006) Phys Rev B]
Molecular dynamics (MD): (~10–9 secs) many realistic processes are inaccessible.
Kinetic Monte Carlo (KMC): (~secs) need all diffusion barriers a priori. (God or compute)
Efficient Coupling of MD and KMCUse MD to get some diffusion barriers.Use KMC to span time.Use GP to regress all barriers from some barrier info.
Span 10–15 seconds to seconds (15 orders of magnitude)
Recap: Multi-timescale Modeling of Alloy Kinetics
Rea
l tim
e
Complexity
TableLookupKMC
On the fly KMC
SymbolicallyRegressed KMC
(sr-KMC)
ΔE calculated: ∼3% (256) configurationsLow-energy events: <0.1% prediction errorOverall events: <1% prediction error
Results: (001) Surface Vacancy-assisted Migration
Total 2nd n.n. Active configurations: 8192
Dramatic scaling over MD (109 at 300 K)
102 decrease in CPU time for calculating barriers
103-107 less CPU timethan on-they-fly KMC
chosen by the AIP editors as focused article of frontier research in Virtual Journal of NanoscaleScience & Technology, 12(9), 2005
Multiscaling Photochemical Reactions
Photochemistry is important for photosynthesis, vision, solar energy, pharmacology,…
Need methods with accurate description of excited states
Dynamics requires many electronic-energy evaluations
Ground stateHalorhodopsin Excited state
Silver “Humies” Medal in Human Competitive Results, Best paper award, Real-world applications track, GECCO-2006 (ACM SIGEVO conference)
Reaction Dynamics Over Multiple Timescales
Accurate but slow (hours-days)Can calculate excited states
Fast (secs.-mins.), accuracy depends on parameters.Calculate integrals from fit parameters.
Ab InitioQuantum Chemistry
Methods
SemiempiricalMethods
TuneSemiempiricalParameters
Accurate excited-state surfaces with semiempirical methodsPermits dynamics of larger systems: proteins, nanotubes, etc.,
Fitting/Tuning semiempirical potentials is non-trivial
Energy & shape of the PES matterEspecially around ground and excited states
Choose few ground- and excited-state configurationsFitness #1: Errors in energy
For each configuration, compute energy difference via ab initio and semiempiricalmethods
Fitness #2: Errors in energy gradientFor each configuration, compute energy gradient via ab initio and SE methods
Fitness: Errors in Energy and Energy Gradient
ab initio SE method
ab initio SE method
*O OO
Current Reparameterization Methods Fall Short; Need Multiobjective Optimization
Current Method: Staged single-objective optimizationFirst minimize error in energiesSubsequently minimize weighted error in energy and gradient
Multiple objectives and highly multimodalDon’t know the weights of different objectivesLocal search gets stuck in low-quality optima
Simultaneously obtain “Pareto-optimal” solutions.
Avoid potentially irrelevant and unphysical pathways.
Multiobjective optimization
Multiobjective Optimization
Unlike single-objective problems, multiobjective problems involve a set of optimal solutions often termed as Pareto-optimal solutions.
Notion of non-domination [Goldberg, 1989]
• A dominates C• A and B are non-dominant• B is more crowded than A
Solution A dominates C if:A is no worse than C in all objectivesA is better than C in at least one objective
Two goals:Converge onto the Pareto-optimal solutions (best non-dominated set )Maintain as diverse a distribution as possible
[Deb, 2002; Coello Coello et al, 2002]
Multi- vs. single-objective optimization
MOGA Finds High Quality SE Parameters
Not even one Pareto-optimal solution obtained with multiple weighted single-objective optimizations
MOGA results have significantly lower errors than current results.
Are all MOGA solutions good from chemists perspective?
MOGA vs. published results
Each point is a set of 11 parameters
MOGA Population Quantifies Parameter Stability
GA population contains useful data that can be minedE.g., 80,000 candidate solutions sampled in each GA run.
Example: stability/sensitivity of parameter setsRMS deviation of the error in energy and energy gradients of solutions within 1% of the Pareto-optimal solutions
Stable parameter set has low RMS deviations.
How to set the threshold?Perturbation of 1% around PM3 set reveals
RMS deviation in error in energy: 0.99RMS deviation in error in energy gradient: 0.023
On-Line Parameter Stability Analysis in MOGA
Analysis of quality of solutions around Pareto-optimal set yields a good measure of the SE parameter stability.
Online analysis is efficient and reliable
MOGA Results Yield Accurate Engergies
MOGA-optimized parameters yield accurate energies for untested, critical configurations.
Parameter sets with lower error in energy are preferable
Pyramidalized
E
D2d twisted
+ E(Pyr-CI S1)∗ E(D2d S1) - E(D2d S0)x E(D2d S1) - E(Pyr-CI S1)
Check potential energy surface with dynamics
Population transfer determined using 50 initial conditions for each parameter setParameter sets with lower error in energy gradient values have lifetimes close to ab initio value
[Quenneville, Ben-Nun & Martinez, 2001]
ab initio value: 180±50 fs
Semiempirical Parameter Interaction Identification
Multiple high-quality parameter sets
Symbolic regression of semiempirical parameters via GP
Interpretable optimized semiempiricial methods
Advances in Scalability: GP Population Sizing, Competent GP, and Scalability of MOGAs
Premium on competent and efficient GAs and GPNeed to understand their scalabilityGA and GP designs that solve hard problems quickly, reliably, and accurately
Population Sizing in GP [Sastry, et al (2003) GPTP; Sastry, O’Reilly, & Goldberg (2004) GPTP]
Building-block supply and decision-making grounds
Competent GP design [Sastry & Goldberg (2003). GPTP]
Automatically identify and exchange key building blocks
Scalability of competent MOGAs [Sastry, Pelikan, & Goldberg (2005) CEC; Pelikan, Sastry & Goldberg (2006) SOPM]
Reliably maintain diverse Pareto-optimal solutions
Recap: How do we size populations?
Tradeoff: Expense = # runs x popsize x # generations
Valuable tool for GP-theorists and practitionersInsight on GP mechanismsPractical guide to set population sizeCompetency: If it’s not a problem with pop size, it must be something elseScalability: How does population size scale with problem size?Efficiency: How to choose the tradeoff?
Limited attention paid in GP theoryCorrect population sizing is critical to GP success
[Sastry, et al (2003) GPTP; Sastry, O’Reilly, & Goldberg (2004) GPTP]
Population Sizing for Building Block Supply in GP
At least one copy of every competing schema in initial population:
Bigger the primitive set, greater the population size
Bigger the trees, smaller the population size
Expression also matters
[Holland, 1975; Goldberg, 1989; Reeves, 1993; Goldberg, Sastry, & Latoza, 2001; Sastry et al, 2004]
Tree size
# competing schemas
Error tolerance
Alphabet cardinalityExpression probability
GP Population Sizing: Decision-Making Model
Make correct decision between competing BBsStatistical in natureFitness is measure of the entire chromosome [Goldberg (1989); Goldberg & Rudnick (1992); Goldberg, Deb & Clark (1992);Harik, Cantú-Paz, Goldberg, & Miller (1997)]
[Goldberg, Deb, and Clark (1992)]
Population Sizing in GP for Good Decision Making
Population size increasesSignal-noise-ratio decreasesError tolerance decreasesTree size decreasesAlphabet size increasesBuilding block size increasesProblem size increasesBloat increases
Probabilistic safety factorSignal-to-noise ratio
Sub-componentcomplexity
Number ofsub-components
[Sastry, O’Reilly, & Goldberg (2004)]
Scalable GP Design
Design a competent GPSolve hard problems quickly, reliably, and accuratelyIdentify and exchange substructures effectivelyPolynomial scale-up on boundedly difficult problem
Combine ideas from GAs & GPExtended compact GA (eCGA) [Harik, 1999]
Probabilistic incremental program evolution (PIPE)[Salustowicz & Schmidhuber, 1997]
Study scale-up of competent GPGP-easy and GP-hard problems
[Sastry & Goldberg (2003). GPTP]
Extended Compact Genetic Algorithm (eCGA)
A Probabilistic model building GA [Harik, 1999]
Builds models of good solutions as linkage groups
Key idea:Good probability distribution ≡ Linkage learning
Key components:Representation: Marginal product model (MPM)
Marginal distribution of a gene partition
Quality: Minimum description length (MDL)Occam’s razor principleAll things being equal, simpler models are better
Search Method: Greedy heuristic search
PIPE: Model Representation & Sampling
Univariate model: Independent nodes [Salustowicz & Schmidhuber, 1997]
Start with root node, depth first, left-to-right traversalChoose either a function or terminal based on model
Extended Compact Genetic Programming: eCGP
Complete n-ary tree (similar to PIPE)Marginal product model (similar to eCGA)
Partition tree-nodes into clustersMarginal probability distribution of each cluster
Scalability of eCGP: GP-easy and GP-hard Problems
GP-Hard problem
eCGP scales as O(λk3)
Advantageous when linkage-learning is critical
Do such problems exist in GP domain?Broader issue of problem difficulty in GPOptimization vs. System identification
GP-Easy problem
[Sastry, & Goldberg, 2003]
Scalability of Multiobjective GAs
Multiobjective estimation of distribution algorithms (EDAs) [Bosman & Thierens, 2002, Khan et al, 2002, Ocenasek, 2002, Ahn, 2005]
Model-building and model-sampling of EDAsSelection and replacement of multiobjective GAs
Outperform MOGAs on boundedly-difficult additively and hierarchically decomposable problems
Limited scalability analysisDemonstrated scalability is a strong point of EDAs
How do MOEDAs scale with problem size on additively separable boundedly-difficult problems?
[Sastry, Pelikan, & Goldberg (2005) CEC; Pelikan, Sastry & Goldberg (2006) SOPM]
Usual Scalability Game
Boundedly-difficult problem where linkage learning is criticalScalability as a function of # building blocks, mUse similar approach for MOEDAs
Overwhelming The Nicher is Easy!
Building blocks are accurately identified and sampled
Test problem has exponential # Pareto-optimal solutions Optimal solutions composed by 0000 and 1111 blocks2m solutions in the Pareto-optimal setm+1 distinct points in the Pareto-optimal set
At least one copy of all 2m solutions
O(2m)
OneMax-ZeroMax
MOEDAs Outperform Simple MOGAs
[Pelikan, Sastry, Goldberg (2005)]
Practical population sizes can’t yield all optimal solutionsCapture representative solutions on the Pareto-optimal frontEDAs outperform simple MOEAs [Pelikan, Sastry, Goldberg (2005)]
Controlled Growth of Competing Building Blocks
Understand the fundamental limit of growth in # Pareto-optimal solutions that can yield polynomial scale-up
Controlledgrowth of competing
building blocks
O(m2)
[Sastry, Pelikan, & Goldberg, 2005]
TrapTrapTrap Trap Trap
Inv TrapTrapTrap Trap Inv Trap
Obj #1
Obj #2
# competing BBs = 2
eCGA pop. size:
Niching pop. size:
Ensuring polynomialscalability:
Summary and Conclusions
Broadly applicable in chemistry and materials scienceAnalogous applicability in modeling multiscaling phenomena.
Facilitates fast and accurate materials modelingAlloys: Kinetics simulations with ab initio accuracy.
102-1015 increase in simulation time.104-107 times faster than current methods
Chemistry: Reaction-dynamics with ab initio accuracy.102-105 increase in simulation time10-103 times faster than current methods.
Lead potentially to new drugs, new materials, fundamental understanding of complex chemical phenomena
Science: Biophysical basis of vision, and photosynthesisSynthesis: Pharmaceuticals, functional materials
top related