By Mohammad Reza Karimi Dastjerdi

Supervisor: Dr. Amin Nikanjam

Implementation of Multi-model Approach

for Estimation of Distribution Algorithms (EDA)

to Solve Multi-Structural Problems

1

Summer 2016

Contents

■ Introduction

■ Heuristic strategies

■ Stochastic Heuristic Searches

■ Genetic Algorithm

■ Estimation of Distribution Algorithms

■ EDAs Pseudo code

■ Bayesian Optimization Algorithm

■ BOAs Pseudo code

■ Bayesian Networks

■ Bayesian Dirichlet Metric

■ Find Best Network

■ Generate New Solutions2

■ Selection Operators

■ Parent Selection

■ Truncation Selection

■ Tournament Selection

■ Survivor Selection

■ Worst Replacement

■ Restricted Tournament Replacement

■ Multi-model Approach

■ Multi-structural Problems

■ Conclusion & Suggestions

■ References

Introduction

■ Optimization is a mathematical discipline that concerns the finding of

the extreme of numbers, functions, or systems.

■ All the search strategy types can be classified as:

• Complete strategies : All possible solutions of the search space

• Heuristic strategies : Part of them following a known algorithm

3

Heuristic strategies

■ Deterministic

• Same Conditions → Same Solution

• Main drawback → Risk of getting stuck in local optimum values

■ Non-deterministic

• Even in Same Conditions → Different Solutions

• Escape from these local maxima by means of the randomness

4

Stochastic Heuristic Searches

■ Store one solution: Simulated Annealing

■ Population-based: Evolutionary Computation.

5

Genetic Algorithms

■ One of the examples of evolutionary computation is Genetic Algorithms (GAs).

■ Inspired by the process of natural selection

■ Relied on bio-inspired operators such as mutation, crossover and selection.

■ The behavior of GAs depends on too many parameters like:

• Operators and probabilities of crossing and mutation

• Size of the population

• Rate of generational reproduction

• Number of generations

• and so on.

6

Estimation of Distribution Algorithms

■ All these reasons have motivated the creation of a new type of algorithms classified

under the name of Estimation of Distribution Algorithms (EDAs).

■ EDAs are:

• Population-based

• Probabilistic modelling of promising solutions

• Simulation of the induced models to guide their search

7

EDAs Pseudo code

8

Bayesian Optimization Algorithm

■ By Pelikan, Goldberg, & Cantu-paz [1998]

9

BOAs Pseudo code

10

Bayesian Networks■ A Bayesian network encodes the relationships between the variables contained in

the modeled data.

■ Bayesian networks can be used to:

• Describe the data

• Generate new instances of the variables with similar properties as those of given data

■ Each node → one variable.

■

𝑝 𝑋 =

𝑖=0

𝑛−1

𝑝 𝑋𝑖 𝑋𝑖

11

Bayesian Dirichlet Metric

■ By Heckerman et al [1994]

■ Prior knowledge about the problem + Statistical data from a given data set.

■ The BD metric for a network B given a data set D and the background information 𝜉denoted by 𝑝 𝐷, 𝐵 𝜉 is defined as:

𝒑 𝑫,𝑩 𝝃 = 𝒑 𝑩 𝝃

𝒊=𝟎

𝒏−𝟏

𝝅𝑿𝒊

𝒎′ 𝝅𝑿𝒊 !

𝒎′ 𝝅𝑿𝒊 +𝒎 𝝅𝑿𝒊 !

𝒙𝒊

𝒎′ 𝒙𝒊,𝝅𝑿𝒊 +𝒎 𝒙𝒊,𝝅𝑿𝒊 !

𝒎′ 𝒙𝒊, 𝝅𝑿𝒊 !

𝑚 𝜋𝑋𝑖 =

𝑥𝑖

𝑚 𝑥𝑖 , 𝜋𝑋𝑖

12

Find Best Network

■ K = 0

• An empty network is the best one (and the only one possible).

■ K = 1

• special case of the so-called maximal branching problem

■ K > 1

• NP-complete

• A simple greedy search

• Only operations are allowed that:

o keep the network acyclic

o at most k incoming edges into each node

• The algorithms can start with:

o An empty network

o Best network with one incoming edge for each node

o Randomly generated network. 13

Generate New Solutions

■ Time complexity of generating the value for each variable: 𝑂 𝑘 .

■ Time complexity of generating an instance of all variables: 𝑂 𝑛𝑘 .

14

Multi-model Approach

22

■ Many real-world problems present several global optima.

■ Most EDAs typically bypass the issue of global multi-modality.

■ It is often preferable or even necessary to obtain as many global optima as possible.

■ Used in ECGA by Chuang and Hsu [2010].

Implementation

23

Results

Average

fitness calls

Average model-

building time

Average

algorithm

time

Final Population sizeRTR?#ModelsProblem Size

68220.0190.02813120130

164160.0210.01518120330

159950.0220.0287341330

24

0200

400600800

1000

12001400

160018002000

Traditional Multi-Models Multi-Models with

RTR

Po

pu

lati

on

Siz

e

Different Approaches

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

Traditional Multi-Models Multi-Models with

RTR

Ave

rage

Fit

ne

ss C

alls

Different Approaches

Implementation

25

𝑁𝑚𝑜𝑑𝑒𝑙𝑖 = 𝑓𝑖

𝑗=1𝑀𝑜𝑑𝑒𝑙𝑠 𝑓𝑗

𝑁𝑚𝑜𝑑𝑒𝑙𝑖 =𝑒𝑓𝑖

𝑗=1𝑀𝑜𝑑𝑒𝑙𝑠 𝑒𝑓𝑗

Results

Average

fitness calls

Average

model-

building time

Average

algorithm

time

Final Population

sizeSoftMax?RTR?#ModelsProblem Size

68220.0190.028131200130

1436890.1930.1261525010330

265800.0390.03957811330

26

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

Traditional Multi-Models with

Softmax

Multi-Models with

Softmax and RTR

Po

pu

lati

on

Siz

e

Different Approaches

0

20000

40000

60000

80000

100000

120000

140000

160000

Traditional Multi-Models with

Softmax

Multi-Models with

Softmax and RTR

Ave

rage

Fit

ne

ss C

alls

Different Approaches

Implementation

27

ResultsAverage

fitness calls

Average

model-

building

time

Average

algorithm

time

Final Population sizeParent

Displacement?SoftMax?RTR?#Models

Problem

Size

68220.0190.0281312000130

77310.0090.0381562100330

72040.0120.071921101330

28

0

200

400

600

800

1000

1200

1400

1600

1800

Traditional Multi-Models with

Parent Displacement

Multi-Models with

Parent Displacement

and RTR

Po

pu

lati

on

Siz

e

Different Approaches

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

Traditional Multi-Models with

Parent Displacement

Multi-Models with

Parent Displacement

and RTR

Ave

rage

Fit

ne

ss C

alls

Different Approaches

Implementation

29

ResultsAverage fitness calls

Average model-

building time

Average

algorithm

time

Final

Population

size

Pop.

Type

Parent

Displacement?SoftMax?RTR?#Models

Problem

Size

68220.0190.0281312----000130

77310.0090.0381562Single100330

72040.0120.071921Single101330

148710.0150.0583125Multi100330

30

0

500

1000

1500

2000

2500

3000

3500

Traditional SinglePop. SinglePop. and

RTR

Multi Pop.

Po

pu

lati

on

Siz

e

Different Approaches

0

2000

4000

6000

8000

10000

12000

14000

16000

Traditional Single Pop. Single Pop. and

RTR

Multi Pop.

Ave

rage

Fit

ne

ss C

alls

Different Approaches

Multi-structural Problems

■ Deceptive Behavior!

𝑀𝑆𝑃1 𝑥 = 𝛼 + 𝑓𝑡𝑟𝑎𝑝 𝑚,𝑘 𝑥2, … , 𝑥𝑛 𝑥1 = 1

𝑛 − 1 − 𝑓𝑂𝑛𝑒𝑀𝑎𝑥 𝑥2, … , 𝑥𝑛 𝑥1 = 0

𝑀𝑆𝑃2 𝑥 = 𝑚𝑎𝑥 𝛼 + 𝑓𝑡𝑟𝑎𝑝 𝑚,𝑘 𝑥 , 𝑛 − 1 − 𝑓𝑂𝑛𝑒𝑀𝑎𝑥 𝑥

31

MSP1 Test Results

32

Average fitness

calls

Average model-

building time

Average

algorithm time

Final Population

sizeParent Replacement?RTR?#ModelsProblem Size

248970.0660.0935750-----0131

278400.0220.1687510331

233470.0220.163487511331

339070.0420.311825010531

318120.0190.218625011531

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

Traditional Three

Models

Three

Models and

RTR

Five Models Five Models

and RTR

Po

pu

lati

on

Siz

e

Different Approaches

0

5000

10000

15000

20000

25000

30000

35000

40000

Traditional Three

Models

Three

Models and

RTR

Five Models Five Models

and RTR

Ave

rage

Fit

ne

ss C

alls

Different Approaches

MSP2 Test Results

33

Average fitness

calls

Average

model-building

time

Average

algorithm

time

Final Population sizeParent Replacement?RTR?#ModelsProblem Size

5969602.9494.231164000-----0130

2386800.3792.3277800010330

2017500.0890.6972500011330

2566600.2372.7978200010530

1859250.0670.871850011530

0

20000

40000

60000

80000

100000

120000

140000

160000

180000

Traditional Three

Models

Three

Models and

RTR

Five Models Five Models

and RTR

Po

pu

lati

on

Siz

e

Different Approaches

0

100000

200000

300000

400000

500000

600000

700000

Traditional Three

Models

Three

Models and

RTR

Five Models Five Models

and RTR

Ave

rage

Fit

ne

ss C

alls

Different Approaches

Conclusions and Suggestions

■ The proposed method seems Nice!

■ Method of learning: Sporadic or Incremental

■ Number of Models

34

References

■ E. Bengoetxea, "Inexact Graph Matching Using Estimation of Distribution Algorithms," Paris, 2002

■ M. Pelikan, D. E. Goldberg and E. E. Cantú-paz, "Linkage Problem, Distribution Estimation, and Bayesian Networks," Evolutionary Computation, pp. 311-340, 2000

■ T. Blickle and L. Thiele, "A comparison of selection schemes used in evolutionary algorithms," Evolutionary Computation, vol. 4, no. 4, pp. 361-394, 1996

■ A. E. Eiben and E. J. Smith, Introduction to Evolutionary Computing, Heidelberg: springer, 2003

■ C. F. Lima, C. Fernandes and F. G. Lobo, "Investigating restricted tournament replacement in ECGA for non-stationary environments," in Genetic and evolutionary computation, New York, 2008

■ C.-Y. Chuang and W.-L. Hsu, "Multivariate multi-model approach for globally multimodal problems," in Genetic and evolutionary computation, New York, 2010.

■ A. Nikanjam and H. Karshenas, "Multi-Structure Problems: Difficult Model Learning in Discrete EDAs," in IEEE Congress on Evolutionary Computation (CEC2016), Vancouver, Canada, 2016

■ M. Pelikan, K. Sastry and D. E. Goldberg, "Sporadic model building for efficiency enhancement of the hierarchical BOA," Genetic Programming and Evolvable Machines, vol. 9, no. 1, p. 53–84, 2008

■ Martin Pelikan’s personal website

■ Wikipedia35

Special Thanks to…

36

Dr. Nikanjam

Dr. Naser Sharif

Saeed Ghadiri

Special Thanks to…

37

Dr. Taghirad

KN2C Robotic Team

38