1
Fitness Inheritance in BOAFitness Inheritance in BOA
Martin PelikanMartin PelikanDept. of Math and CSDept. of Math and CS
Univ. of Missouri at St. LouisUniv. of Missouri at St. Louis
Kumara Kumara SastrySastryIllinois GA LabIllinois GA Lab
Univ. of Illinois at UrbanaUniv. of Illinois at Urbana--ChampaignChampaign
2
MotivationMotivation
Bayesian optimization algorithm (BOA)Bayesian optimization algorithm (BOA)Scales up on decomposable problemsScales up on decomposable problemsO(O(nn))--O(O(nn22) evaluations until convergence) evaluations until convergence
Expensive evaluationsExpensive evaluationsRealReal--world evaluations can be complexworld evaluations can be complexFEA, simulation, …FEA, simulation, …O(O(nn22) is often not enough) is often not enough
This paperThis paperExtend probabilistic model to include fitness infoExtend probabilistic model to include fitness infoUse model to evaluate part of the populationUse model to evaluate part of the population
3
OutlineOutline
BOA basicsBOA basicsFitness inheritance in BOAFitness inheritance in BOA
Extend Bayesian networks with fitness.Extend Bayesian networks with fitness.Use extended model for evaluation.Use extended model for evaluation.
ExperimentsExperimentsFuture workFuture workSummary and conclusionsSummary and conclusions
4
Bayesian Optimization Bayesian Optimization AlgAlg. (BOA). (BOA)
Pelikan, Goldberg, and CantuPelikan, Goldberg, and Cantu--Paz (1998)Paz (1998)Similar to genetic algorithms (Similar to genetic algorithms (GAsGAs))Replace mutation + crossover byReplace mutation + crossover by
Build Bayesian network to model selected solutions.Build Bayesian network to model selected solutions.Sample Bayesian network to generate new candidate Sample Bayesian network to generate new candidate solutions.solutions.
5
BOABOA
Current population Selection
New population
Bayesian network
Restricted tournament replacement
6
Bayesian Networks (Bayesian Networks (BNsBNs))
2 components2 componentsStructureStructure
directed acyclic graphdirected acyclic graphnodes = variables (string positions)nodes = variables (string positions)Edges = dependencies between variablesEdges = dependencies between variables
ParametersParametersConditional probabilities Conditional probabilities p(X|Pp(X|Pxx), where ), where
X is a variableX is a variablePPxx are parents of X (variables that X depends on)are parents of X (variables that X depends on)
7
BN exampleBN example
A
B C
BB p(Bp(B))00 0.250.2511 0.750.75
AA BB p(A|Bp(A|B))00 00 0.100.1000 11 0.600.6011 00 0.900.9011 11 0.400.40
CC AA p(C|Ap(C|A))00 00 0.800.8000 11 0.550.5511 00 0.200.2011 11 0.450.45
8
Extending Extending BNsBNs with fitness infowith fitness info
Basic ideaBasic ideaDon’t work only with Don’t work only with conditional probabilitiesconditional probabilitiesAdd also fitness info for Add also fitness info for fitness estimationfitness estimation
AA BB p(A|B)p(A|B) f(A|B)f(A|B)00 00 0.100.10 --0.50.500 11 0.600.60 0.50.511 00 0.900.90 0.30.311 11 0.400.40 --0.30.3
Fitness info attached to Fitness info attached to p(X|Pp(X|Pxx) denoted by ) denoted by f(X|Pf(X|Pxx))Contribution of X restricted by Contribution of X restricted by PPxx
avg. fitness of solutions with X=x and avg. fitness of solutions with X=x and PPxx==ppxx
avg. fitness of solutions with avg. fitness of solutions with PPxx
f X = x | Px = px( )= f X = x,Px = px( )− f Px = px( )f X = x,Px = px( )
f Px = px( )
9
Estimating fitnessEstimating fitness
EquationEquation
In wordsIn wordsFitness = avg. fitness + avg. contribution of each bitFitness = avg. fitness + avg. contribution of each bitAvg. contributions taken w.r.t. context from BNAvg. contributions taken w.r.t. context from BN
f X1, X2 ,K , Xn( )= favg + f Xi | PXi( )
i=0
n
∑
10
BNsBNs with decision treeswith decision trees
Local structures in Local structures in BNsBNsMore efficient representation for p ( X | More efficient representation for p ( X | PPxx ))
Example for p ( A | B C )Example for p ( A | B C )
( )0,1| == CBAp
B
C
0 1
10
( )1,1| == CBAp
( )0| =BAp
11
BNsBNs with decision trees + fitnesswith decision trees + fitness
Same ideaSame ideaAttach fitness info to Attach fitness info to each probabilityeach probability
)0|( =BAf
)0,1|( == CBAp
B
C
0 1
10
)1,1|( == CBAp
)0|( =BAp
)0,1|( == CBAf )1,1|( == CBAf
12
Estimating fitness againEstimating fitness again
Same as before…because both Same as before…because both BNsBNs represent the samerepresent the sameEquationEquation
In wordsIn wordsFitness = avg. fitness + avg. contribution of each bitFitness = avg. fitness + avg. contribution of each bitAvg. contributions taken Avg. contributions taken w.r.tw.r.t. context from BN. context from BN
( )∑=
+=n
iXiavgn i
PXffXXXf0
21 |),,,( K
13
Where to learn fitness from?Where to learn fitness from?
Evaluate entire initial populationEvaluate entire initial populationChoose inheritance proportion, pChoose inheritance proportion, pii
After thatAfter thatEvaluate (1Evaluate (1--ppii) proportion of offspring) proportion of offspringUse evaluated parents + evaluated offspring to learnUse evaluated parents + evaluated offspring to learnEstimate fitness of the remaining prop. pEstimate fitness of the remaining prop. pii
Sample for learning: N(1Sample for learning: N(1--ppii) to N+N(1) to N+N(1--ppii))Often, 2N(1Often, 2N(1--ppii))
14
Simple example: Simple example: OnemaxOnemax
OnemaxOnemax
What happens?What happens?Average fitness grows (as predicted by theory)Average fitness grows (as predicted by theory)No context is necessaryNo context is necessaryFitness contributions stay constantFitness contributions stay constant
( ) ∑=
=n
iin XXXXf
121 ,,, K
( )( ) 5.00
5.01−==+==
i
i
XfXf
15
ExperimentsExperiments
ProblemsProblems5050--bit bit onemaxonemax10 traps of order 410 traps of order 410 traps of order 510 traps of order 5
SettingsSettingsInheritance proportion from 0 to 0.999Inheritance proportion from 0 to 0.999Minimum population size for reliable convergenceMinimum population size for reliable convergenceMany runs for each setting (300 runs for each setting)Many runs for each setting (300 runs for each setting)
OutputOutputSpeedSpeed--up (in terms of up (in terms of realreal fitness evaluations)fitness evaluations)
16
OnemaxOnemax
0 0.2 0.4 0.6 0.8 10
5
10
15
20
25
30
35
Proportion inherited
Spe
ed-u
p (w
.r.t.
no in
herit
ance
)
17
TrapTrap--44
0 0.2 0.4 0.6 0.8 10
5
10
15
20
25
30
35
Proportion inherited
Spe
ed-u
p (w
.r.t.
no in
herit
ance
)
18
TrapTrap--55
0 0.2 0.4 0.6 0.8 10
10
20
30
40
50
60
Proportion inherited
Spe
ed-u
p (w
.r.t.
no in
herit
ance
)
19
DiscussionDiscussion
Inheritance proportionInheritance proportionHigh proportions of inheritance work great.High proportions of inheritance work great.
SpeedSpeed--upupOptimal speedOptimal speed--up of 30up of 30--5353High speedHigh speed--up for almost any settingup for almost any settingThe tougher the problem, the better the speedThe tougher the problem, the better the speed--upup
Why so good?Why so good?Learning probabilistic model difficult, so accurate Learning probabilistic model difficult, so accurate fitness info can be added at not much extra cost.fitness info can be added at not much extra cost.
20
ConclusionsConclusions
Fitness inheritance works great in BOAFitness inheritance works great in BOATheory now exists (Theory now exists (SatrySatry et al., 2004) that et al., 2004) that explains these resultsexplains these resultsHigh proportions of inheritance lead to high High proportions of inheritance lead to high speedspeed--upsupsChallenging problems allow much speedChallenging problems allow much speed--upupUseful for practitioners with computationally Useful for practitioners with computationally complex fitness functioncomplex fitness function
21
ContactContact
Martin Martin PelikanPelikanDept. of Math and Computer Science, 320 CCBDept. of Math and Computer Science, 320 CCBUniversity of Missouri at St. LouisUniversity of Missouri at St. Louis8001 Natural Bridge Rd.8001 Natural Bridge Rd.St. Louis, MO 63121St. Louis, MO 63121
EE--mail: mail: [email protected]@cs.umsl.edu
WWW:WWW: http://http://www.cs.umsl.edu/~pelikanwww.cs.umsl.edu/~pelikan//