Review

•  Search, heuristics and game playing •  Tabu Search •  Simulated Annealing •  GA •  ACO •  PSO

Search Algorithms

n  Tree search algorithms n  Uninformed search

n  Depth-first, n  Breadth-first. n  Bi-directional n  Iterated Deepening

n  Informed search n  Hill climbing, n  Beam search,

n  Best-first. n  A*

Evaluating search strategies

n  Completeness: strategy guaranteed to find a solution if it exists.

n  Time complexity. n  Space complexity. n  Optimality: does it find the best solution

(shortest path)

Evaluating DFS

Assume a branching factor b and depth is d

n  Complete: n  Time:O ( ) worst case n  Space: O (db) n  Optimal:no

db

Evaluating BFS

n  Complete: n  Time: O ( ) n  Space:max no of nodes generated= n  Optimal:yes

2 11 .... ( 1) /( 1) ( )d d db b b b b O b++ + + + = − − =

db

Comments on DFS and BFS n  Depth-first

n  Low memory requirement. n  Could get stuck exploring infinite paths. n  Used if there are many solutions and you need

only one.

n  Breadth-first n  High memory requirement. n  Doesn’t get stuck. n  Finds the shortest path (minimum number of

steps).

Other uninformed strategies n  Bidirectional Search

n  Search forward from initial state as well as backward from a goal state (if known)

n  Complete: n  Time: O( ) n  Space:same n  Optimal:

/ 2db

Other uninformed Strategies

n  Iterative Deepening To avoid guessing the depth limit (in DFS), do

DFS with all depth limits till finding the solution.

Complete: Time:O ( ) Space: O (db) as we don’t keep previous

iterations Optimal:yes

db

Informed Search

n  Hill climbing n  Tries to improve the efficiency of depth-first. n  Informed depth-first algorithm. n  It sorts the successors of a node (according

to their heuristic values) before adding them to the list to be expanded.

Informed Search

n  Beam search n  Tries to minimize the memory requirement

of the breadth-first algorithm. n  Informed breadth-first algorithm. n  Expands only the first m promising nodes

at each level.

Informed Search

n  Best-first n  Adds the successors of a node to the

expand list. n  All nodes on the list are sorted according to

the heuristic values. n  Expand most desirable unexpanded node n  Special Cases: Greedy search, A*

Greedy Search

n  Greedy search is best first that uses heuristic h(n) =estimate of cost from n to the closest goal node

n  Greedy search expands nodes that appear to be closest to goal

A* n  A* tries to avoid expanding paths that are

already expensive. n  It uses an evaluation function f(n)=g(n)+h(n) n  g(n) is cost from start node to n n  h(n) is estimate cost from n to the goal n  A* uses admissible heuristic h(n)<= h*(n)

(the actual cost from n to the goal, and h(n)>=0)

n  A* is optimal

Game Playing As Search

n  In Games there is an opponent. n  Need to search for best move and wait

for the opponent response to search again from the new board configuration for best move.

n  Time is limited to find goal in each search

Two Players, perfect information

n  Player and Opponent. n  Take the player’s point of view n  Ideally the player expands the game

tree taking into consideration all the possible moves of the opponent till end of the game with leaf nodes as win, lose, draw.

Max Min strategy n  Player – MAX n  Opponent – MIN n  Minimax principle n  Label each level in Game Tree with MAX (player) and

MIN (opponent) n  Label leaves with evaluation of player n  Go through the game tree

n  if father node is MAX then label the node with the maximal value of its successors

n  if father node is MIN then label the node with the minimal value of its successors

Max Min strategy

n  Complete: Yes (if tree is finite) n  Time complexity: O(bd) n  Space complexity: O(bd) (depth-first

exploration) n  Optimal: Yes (against an optimal

opponent)

Limited depth and evaluation function n  Expanding the complete game tree is only feasible

for simple games. Tic Tac Toe has average branching factor b of 5 and average (moves) depth d of 9. Checkers has b =35 and d =100. Taking into consideration symmetrical and repeated states the game tree may be less than that. Still for checkers 1040. For Chess it is

n  It is only feasible to explore a limited depth of the game tree and to use a board evaluation function for the worth of this node to the player.

14010

Alpha Beta Pruning n  A way to reduce the number of nodes that need to

generated and evaluated n  A branch and bound idea n  Alpha Cutoff: when the value of a min position is less

than or equal to the alpha-value of its parent, stop generating further successors

n  Beta Cutoff: when the value of a max position is greater than the beta-value of its parent, stop generating further successors.

α-β pruning example [13]

α-β pruning example

α-β pruning example

α-β pruning example

α-β pruning example

Alpha Beta Pruning n  It is guaranteed to return exactly the same

value as the Min-Max algorithm. It is a pure optimization without any approximations or tradeoffs.

n  In a perfectly ordered tree, with the best moves on the left, alpha beta reduces the cost of the search from order bd to order b(d/

2) , that is, we can search twice as deep. n  Worst case it won’t prune any nodes.

Meta-Heuristics n  Meta-heuristics are algorithms that combine

heuristics in a higher level framework aimed at efficiently and effectively exploring the search space,

n  Types:

n  Trajectory methods, which handles one solution at a time,

n  Population-based, which handles multiple solutions.

Meta-Heuristics

Meta-Heuristics

Population based

Trajectory method

ACO PSO

EC GRASP ILS TS SA SI

ES EP GP GA

Meta-Heuristics

n  Trajectory methods n  Iterated Local Search (ILS), n  Guided Local Search (GLS), n  Greedy Randomized Adaptive Search

Procedure (GRASP), n  Variable Neighbourhood Search (VNS), n  Tabu Search (TA), n  Simulated Annealing (SA).

TS Basic Idea

n  Forbidding or penalizing moves which take the solution, in the next iteration, to points in the solution space previously visited (tabu).

n  Accepting non-improving solutions deterministically in order to escape from local optima where all the neighbouring solutions are non-improving.

TS- Use of Memory

n  Uses of memory Structures: n  Short term memory based on recency of

occurrence to prevent the search from revisiting previously visited solutions;

n  Recency memory can also be used to keep track of good components to return to in order to intensify the search

TS- Short term memory

n  Tabu lists length refer to the number of iterations T for which we keep a certain move (or its attributes) in the list (tabu tenure).

n  The list may record tabu active attributes

TS – Aspiration n  Sometimes it’s useful to allow a certain

move even if it is tabu, n  This could be done to prevent

stagnation, n  Approaches used to cancel the tabus

are referred to as aspiration criteria,

TS- Neighborhood

n  Like other search method we need a neighborhood structure N(s)

n  In selecting a new state we consider neighbours that are not on the Tabu list N(s) - T(s)

n  N(s) can be reduced and modified based on history and knowledge

Examples

n  Solving Permutation Optimization Problems: n  Insulating Modules From [9] n  Travelling Sales Person n  Job Scheduling

Example: Insulating Modules

3

3 2

1 5 1

4 4

1 2

2 3

Tabu list

Frequency (counts of moves over past iterations)

1,4 3 3

2.4 -1 -6

3,7 -3 -3

1,6 -5 -5

6,5 -4 -6

Swap ∆value +penalty

1 3 6 2 7 5 4

Value=12

1 2 3 4 5 6 7 1 2 3 4 5 6 7

Iteration 26 T *

Penalize non-improving moves according to their frequencies

TS for TSP - Example

d25=5 1

2

5

3 4

d12=4

d23=7

d34=4

d45=3

d24=5

d15=12

d35=10

d13=2 d14=9

TS: 7- queen Problem [2]

Q1

Q2

Q3

Q4

Q5

Q6

Q7

4 5 3 6 7 1 2 Q C

1 2 3 4 5 6 7

Collisions=4

Representation as ordering

TS: 7 queen problem iteration 0

1,7 -2

2,4 -2

2,6 -2

5,6 -2

1,5 -1

2 3 4 5 6 7

1

2 3

4 5

6

4 5 3 6 7 1 2

collisions =4

Select (1,7) as a move

*

Q 1 2 3 4 5 6 7

Queen ∆ Swap value

C

Min Spanning Tree with constraints

n  Finding a spanning tree that also satisfies extra constraints (on edges or nodes or both)

n  Example constraints: certain edges (or nodes) are inclusive (have to occur together); edges (or nodes) are exclusive only some of them need to be in the tree.

Example MST with constraints [3]

A

B

D

C E

20 30

15 40

10 5

25 A

B

D

C E

20 30

15 40

10 5

25

An optimal MST without considering constraints, Cost=50

Constraints 1: Link AD can be included only if link DE also is included. (penalty:100) Constraints 2: At most one of the three links – AD, CD, and AB – can be included. (Penalty of 100 if selected two of the three, 200 if all three are selected.)

TS Model

n  N(x) is defined by edge exchange moves

n  TL: the inserted edge n  Tabu tenure = 2 n  Aspiration criterion: Best Objective n  Use MST without constraints as initial

solution

QAP n  Given n objects and the flows between the object i

and the object j (i, j = 1. . . n), and given n sites with distance between the sites r and s (r, s = 1. . . n), the problem deals with placing the n objects on the n sites so as to minimize the sum of the products, flows × distances. Mathematically, this is equivalent to find a permutation p, whose ith component denotes the object assigned to site i, which minimizes

∑∑= =

n

i

n

jijpp df

ji1 1

ijf

rsd

ip

QAP

n 

Bipartite graph

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

s j

i r rsdijf

objects sites

n  Adaptation n  Tabu tenure changes dynamically n  Cooperation: n  P parallel TS n  synchronous communication n  asynchronous communication

Simulated Annealing

n  The approach is part of neighborhood search approaches to move from current solution to new solutions.

n  It also tries to escape from local optimum by allowing a non-improving solution but in a controlled manner.

Simulated Annealing n  The acceptance probability is defined as:

where: c is the change in the solution cost, t is the current temperature (control parameter, no physical role).

Notice that we dropped Boltzmann’s constant

⎩⎨⎧

>Δ≤Δ

= Δ− 0 0 1

/ cifecif

P tc

SA Basic Algorithm n  Set current solution to an initial solution s= n  Set current temperature to an initial temperature t = n  Select a temperature reduction function n  Repeat

n  Repeat n  Select a solution from the neighborhood n  Calculate change in cost according to the new solution n  If <0 then accept new solution (it is improving) s= n  Else generate a random number x in the range (0,1)

n  If x< then accept new solution s=

n  Until max no. of iterations for the temperature n  Decrease t using

n  Until stopping conditions are satisfied n  s is the final solution

0s0t

α

)(sNCΔ

CΔ

tce /Δ−

is

is

is

α

Annealing Schedule n  The value of the temperature is adjusted according

to the annealing schedule, n  The annealing schedule is a mapping from time to

temperature, n  The annealing schedule is determined by:

n  The initial temperature, n  The final temperature, n  The temperature decrement rule, n  Number of iterations at each temperature.

Temperature

n  Temperature decrement rules n  Linear

n  Geometric

α−= tt

α×= tt

Scheduling problems http://riot.ieor.berkeley.edu/riot/Applications/Scheduling/objectives.html

n  Scheduling jobs to be processed on machine(s) n  Objective functions n  Makespan Let Ci denote the completion time of the ith job in a batch of n jobs

given. The makespan, Cmax, is defined as, Cmax = max (C1, C2, … , Cn) n  The makespan is the completion time of the last job. n  A common problem of interest is to minimize Cmax, n  This criterion is usually used to measure the level of utilization

of the machine.

Objective Functions n  Total (Weighted) Completion Time Let Ci denote the completion time of the ith job in a batch of n jobs given. The total completion time is defined as,

n Ci i=1

n  The total completion time is the sum of all the completion times of the jobs. It is commonly referred to as the flow time.

Let wi denote the weight assigned to the ith job in the a batch of n jobs given. The total weighted completion time is defined as,

n wiCi i=1

A common problem is in minimizing the total (weighted) completion time. This problem allows one to find an indication to the total holding or inventory caused by the schedule.

Objective Functions n  Lateness Difference between completion time and the due date. Li=Ci-di, where Ci

is the completion of job i and di is the due date of job i. Some of the problems involving lateness are: 1. Minimizing the total lateness. The total lateness is defined as,

n Li i=1

2. Minimizing the total weighted lateness. Let wi denote the weight assigned to the ith job. The total weighted lateness is defined as, n wi Li i=1

3. Minimizing the maximum lateness, Lmax. Lmax is defined as, Lmax = max (L1, L2, … , Ln)

Objective Functions n  Tardiness The tardiness of job i, Ti, is defined as Ti=Max (0, Ci-di), where Ci

is the completion of job i and di is the due date of job i. Some of the problems involving tardiness are: 1. Minimizing the maximum tardiness

Min (max (Ti)) 2. Minimizing the number of tardy jobs. The number of tardy

jobs is defined as, n

NT= (Ti) i=1

where ( x) = 1 if x > 0 ( x) = 0 otherwise

Tardiness (cont’d

n  Minimizing the total weighted tardiness. Let wi denote the weight assigned to the ith job. The total weighted tardiness is defined as,

n wi Ti i=1

Machine environments n  Single Machine: Only one machine is available to process

jobs. Each job has a single task. Every job is performed on the same machine.

n  Parallel Machines: Multiple machines are available to process

jobs. The machines can be identical, of different speeds, or specialized to only processing specific jobs. Each job has a single task.

Multiple Machines Minimum Makespan Problem

n  Given processing times for n jobs, p1, p2, . . . , pn, and an integer m, the Minimum Makespan Scheduling problem is to find an assignment of the jobs to m machines, M1,M2, . . . ,Mm so that the final completion time (the makespan) is minimized.

Flow Shop Scheduling n  In the general flow shop model, there are a series of machines

numbered 1,2,3…m. Each job has exactly m tasks. The first task of every job is done on machine 1, second task on machine 2 and so on. Every job goes through all m machines in a unidirectional order. However, the processing time each task spends on a machine varies depending on the job that the task belongs to. The precedence constraint in this model requires that for each job, task i-1 on machine i-1 must be completed before the ith task can begin on machine i.

Job shop n  In the general job shop model, there are a set of m

machines. Jobs have tasks (or operations) not necessarily m as in the flow shop.

n  Each job uses the machines (or subset of them) in a specified sequence

n  The flow of the tasks in a job does not have to be unidirectional. Each job may also use a machine more than once.

SA for the Graph Coloring Problem, [1]

n  Given a graph with n nodes and m edges. Allocate a color to each node such that adjacent nodes are not give the same color.

n  The objective is to find an allocation which uses the smallest number of colors.

n  Another variant is given a number of colors, k, and a graph, can we answer the question: does the graph have k-coloring?

n  NP-hard problem

Dynamic Layout Problem [3] n  Facility layout in manufacturing may be required to

be modified during different time periods due to change in processing sequence of products, mix of parts, volume of production,..etc. The Dynamic Layout Problem (DLP) can be defined as:

n  Given N departments, N locations, T time periods, distance between locations D, flow costs of product from one department to the other F and assignment( location) cost R and rearrangement costs of one department to another in the locations C, we want to minimize the sum of flow, location and rearrangement costs subject to some constraints.

Adaptation n  Adaptation in SA is in the parameters

used by the algorithm: n  The initial temperature, the cooling

schedule and no. of iterations per temperature being the most critical.

n  Other components such the cost function and method of generating neighborhood solutions and acceptance probability affect the computational costs.

Cooperative SA n  Cooperative SA (COSA) was proposed by Wendt et al.

in 1997 [5], n  COSA implements concurrent and synchronous runs

of multiple SA processes, n  The concurrent processes are coupled through the

cooperative transitions, n  The cooperative transition replaces the uniform

distribution used to select the neighbours.

Parent Selection

SGA - Introduction

Population

Parents

Offspring Termination

Initialization

Crossover + Mutation

n  The general scheme [1]:

survivor

selection

Parents and Offspring Selection

n  Selection can occur in two places: n  Selection from current generation to take part

in mating (parent selection) n  Selection from parents + offspring to go into

next generation (survivor selection)

Parent Selection Fitness-Proportionate Selection (FPS) Probability of parent selection

=

n  Expected number of copies of an individual i E( ni ) =

( is the fitness and is the average fitness

ip ∑ j

i

ff

if f

ff

FPS n  Roulette wheel algorithm:

n  Given a probability distribution, spin a 1-armed wheel n times to make n selections

n  No guarantees on actual value of ni

n  Baker’s Stochastic Universal Sampling algorithm: n  n evenly spaced arms on wheel and spin once

n  Guarantees floor(E( ni ) ) ni ceil(E( ni ) )

n  Problems include: n  One highly fit member can rapidly take over if rest

of population is much less fit: Premature Convergence

n  At end of runs when fitnesses are similar, lose selection pressure

n  Highly susceptible to function transposition

FPS

GAs – representations n  Binary Representation n  Other representations are also possible:

n  Integers, n  Real-valued or floating-point numbers, n  Permutations.

n  The crossover and mutation actual implementation depends on the representation.

Population Models n  SGA uses a Generational model (GGA):

n  each individual survives for exactly one generation n  the entire set of parents is replaced by the offspring

n  Steady-State model: n  Only part of the population is replaced by the

offspring, n  Usually one member of population is replaced, n  the proportion of the population replaced is called

the Generational Gap n  1.0 for GGA, 1/pop_size for SSGA

SGA - Crossover

n  Crossover is applied according to a probability Pc,

n  Otherwise, the two parent are copied to

the offspring unmodified,

SGA - Crossover n  1-point crossover (the binary case):

n  Choose a random point on the two parents, n  Split parents at this crossover point, n  Create children by exchanging tails, n  Pc typically in range (0.6, 0.9).

Example from [1]

SGA - Crossover n  n-point crossover (the binary case):

n  Choose n random crossover points, n  Split along those points, n  Glue parts, alternating between parents, n  Generalisation of 1 point (still some positional bias).

Example from [1]

SGA - Crossover n  Uniform crossover (the binary case):

n  Assign 'heads' to one parent, 'tails' to the other, n  Flip a coin for each gene of the first child, n  Make an inverse copy of the gene for the second child, n  Inheritance is independent of position.

Example from [1]

SGA - Mutation

n  Alter each gene independently with a probability pm n  pm is called the mutation rate

n  Typically between 1/pop_size and 1/ chromosome_length

Example from [1]

Real-valued GAs

n  Two types of crossover operators: n  Discrete, use any of the crossover

operators identified before for binary representations,

n  Intermediate (arithmetic): n  Single arithmetic, n  Simple arithmetic, n  Whole arithmetic.

Real-valued GAs

n  Single arithmetic crossover: n  Parents: x1,…,xn and y1,…,yn, n  Pick random gene (k) at random, n  Child 1 is:

n  Reverse for other child.

1 1, ..., , (1 ) , ..., k k k nx x y x xα α− ⋅ + − ⋅

Real-valued GAs

n  Simple arithmetic crossover: n  Parents: x1,…,xn and y1,…,yn, n  Pick random gene (k), after this point mix

the values, n  Child 1 is:

n  Reverse for other child.

nxkxkykxx ⋅−+⋅+⋅−++⋅ )1(ny ..., ,1)1(1 , ..., ,1 αααα

Real-valued GAs

n  Whole arithmetic crossover: n  Most commonly used, n  Parents: x1,…,xn and y1,…,yn, n  Child 1 is:

n  Reverse for other child.

yaxa ⋅−+⋅ )1(

Real-valued GAs n  The general scheme for mutation is:

n  Analogous to bit-flipping mutation.

ll xxxx xx ′′=′→= ..., , ...,, 11

[ ]iiii UBLBxx ,, ∈′

[ ]iii UBLBx , from (uniform)randomly drawn ′

Real-valued GAs n  Another mutation scheme is to add

random noise, for each gene:

where N(0,) is a random Gaussian number with mean zero and standard deviation .

( )σ,0' Nxx ii +=

Permutations GAs – TSP example

n  Four types of crossover operators: n  Adjacency-based:

n  Partially Mapped Crossover (PMX), n  Edge crossover.

n  Order-based: n  Order 1 crossover, n  Cycle crossover.

GAs – When are they useful? n  Highly multimodal functions, n  Discrete or discontinuous functions, n  High-dimensionality functions, including many

combinatorial ones, n  Non - l i n e a r d ependen c i e s b e tween

parameters.

Difficulties that may occur with GA

n  Premature convergence n  Unable to overcome deception n  Need more evaluations than time permits n  Bad match of representation/mutation/crossover,

making operators destructive n  Biased or incomplete representation n  Problem too hard n  (Problem too easy, makes GA look bad)

Adaptive GAs: A mutation example

n  Recall the Gaussian mutation defined as:

n  We can adaptively set the value of during the run.

( )σ,0' Nxx ii +=

Adaptive GAs: A mutation example n  The deterministic approach could set

the value of as: where t is the generation number ranging from 0 to T (maximum generation number).

( )Ttt 9.01−=σ

Adaptive GAs: A mutation example

n  The adaptive approach incorporates feedback from the search process, a well-known method is the Rechenberg’s ‘1/5 success rule’,

n  The rule states that the shown percentage should hold:

51

mutations#mutations successful# =

Adaptive GAs: A mutation example n  A successful mutation is a mutation that

produces a better individual, n  The values of (t) are selected as non-

deterministic based on the ratio n  If the ratio is greater than 1/5, the mutation

step size should be increased, and if the ratio is less than 1/5, the step size should be decreased.

Adaptive GAs: A mutation example n  The update rule is applied every n

generations as follows:

where ps is the calculated percentage and 0.82 < c < 1.

/ , 1/ 5. , 1/ 5, 1/ 5

s

s

s

c if pc if pif p

σσ σ

σ

>⎧⎪= <⎨⎪ =⎩

Adaptive GAs: A mutation example n  The self-adaptive approach adds the mutation

step size to the individual (used originally in ES):

n  The value of is evolved with the individual, n  Each individual will have a different mutation

step size.

σ,,...,, 21 nxxxx =

Adaptive GAs: A crossover example n  One can also adapt the crossover operator, n  This operator could be adapted on three

levels [3]: n  Top level, adapting the operator itself, n  Medium level, adapting the probability of

crossover n  Bottom level, adapting the crossing position or the

swapping probability.

Parallel GAs

n  Different approaches have been proposed to study parallelization in GAs [4]: n  Master-slave approaches, n  Fine-grained GAs, n  Coarse-grained GAs, n  Hierarchal parallel GAs.

Parallel GAs – Multiple-deme Individual migrating from one population to another Population 1

Population 2

Multiple-deme - Factors

n  There are a lot of different issues to consider in multiple-deme GAs: n  Number and sizes of populations, n  Different topologies, n  Different migration polices, n  Different migration frequencies, n  Different migration rates.

Cooperative GAs n  The general idea is to have different populations

optimizing different variables of the problem (different dimensions of the solution),

n  The fitness of any individual is determined by its

value and the value of the best individuals in all the other populations,

n  Performs best If the problem variables are independent.

overall solution

Parallel GAs

Population 1 Population 3

B B

Population 2

B B

ACO - Algorithm n  At each node i, the ant has a choice to

move to any of the j nodes connected to it, n  Each node j Ni connected to i has a

probability to be selected by ant K equal to:

⎪⎪⎩

⎪⎪⎨

⎧

∉

∈= ∑

∈

i

i

Nninin

ijij

kij

Njif

Njifdd

pi

0

/

/βα

βα

ττ

ACO - Algorithm n  and are chosen to balance the local vs. the

global search ability of the ant. n  Evaporate the artificial pheromone:

n  The ant deposits extra pheromone on the arc it chooses:

n  This will increase the probability of a subsequent ant choosing the same arc and referred to as online step_by_step pheromone update.

τττ Δ+= ijij

ACO - Algorithm

n  Different approaches exist for choosing the value of : n  Ant density model, adding Q, a constant value,

hence the final pheromone added to the edge will be proportional to the number of ants choosing it. This doesn’t take the edge length into account,

n  Ant quantity model, adding Q/dij, taking the edge length into account, hence enforcing the ant local search ability.

τΔ

n  Another approach is the online delayed pheromone update, sometimes referred to as the ant cycle model,

n  After the ant builds the solution, it traces the path backward and updates the pheromone trails on the visited arcs according to the solution quality.

ACO - Algorithm

ACO - Algorithm n  The online delayed pheromone update is done by adding

(done after the evaporation phase) :

for every arc (i, j) on the path, where Lk is the length of the path found by ant k.

n  Repeat the process for different initialization of pheromone

kij LQ=Δτ

Ant System Algorithm

n  AS : n  Proposed by Dorigo et al. [6], n  Uses the same basic steps outlined in ACO, n  The online delayed pheromone update is

adopted using all the solutions of the current iteration.

kij LQ=Δτ

Max-Min Ant System Algorithm

n  MMAS: n  Proposed by Stutzle and Hoos [7] to

overcome stagnation. n  The update is done using the best solution

(best ant) in the current iteration or the best solution over all, also decay in the update of the pheromone

)()()1()1( ttt bestijijij τρτρτ Δ+−=+

MMAS n  The values of the pheromone are restricted

between min and max , allows high exploration in the beginning (max) and more intensification later.

n  The values of min and max are chosen experimentally, although they could be calculated analytically if the optimal solution is known,

n  Improved the performance significantly over AS.

Ant Colony System Algorithm

n  ACS: n  Proposed by Gambardella and Dorigo [8], n  Based on AS but uses a different transition

rule based on Elitist strategy n  If q <qo (q is a random number, qo is in

(0,1)

{ }βητ ijijNj )t(maxargj ki∈

=

ACS n  Else use:

otherwise 0

allowed j if k

⎪⎪⎩

⎪⎪⎨

⎧∈

∑=∈ kallowedk

ikik

ijij

kij

][)]t([][)]t([

)t(pβα

βα

ητητ

This creates bias towards choices of better quality, qo is a user specified parameter to control this bias.

ACS n  Uses the offline pheromone update, where

all the ants update their last traversed edge only,

n  Introduced a new pheromone update rule

(using the best solution only).

)()()1()1( 11 ttt bestijijij τρτρτ Δ+−=+

022 )1( τρτρτ +−= ijij

Best here can be iteration best or global best

ACO Applications

n  Travelling Salesman Problem (TSP), n  Assembly Line Balancing (ALB), n  Cell Assignment in PCS Networks (CA). n  Timetabling Problem

Adaptation

n  Severa l approaches have been proposed to adapt the parameter values of ACO, which includes: n  ACSGA-TSP, Pilat and White [12], n  Near parameter free ACS, Randall [13].

Adaptation - ACSGA-TSP n  In [12], the parameter values in ACS were optimized

using a genetic algorithm,

n  The idea was to have a GA running on top of the ACS to optimize its parameter values,

n  The optimized parameters were: n  q0, the parameter determining whether the greedy or

probabilistic selection is adopted, n  , the local pheromone updating factor, n  , the relative importance of the visibility heuristic.

Adaptation - Near parameter free ACS n  Each ant is allowed to:

n  Select the suitable values of its parameters, n  Select the next solution component.

n  Each ant has its own parameter values adaptively selected using an ant approach: n  A separate pheromone matrix is kept for learning

these values, n  No heuristic information (similar to the distance

information in TSP) is used.

Cooperation

n  Two approaches have been studied to investigate the cooperation between different ant colonies: n  A heterogeneous approach, in which the

ants in the two colonies have different behaviours (different optimization criteria),

n  A homogeneous approach, in which all the ants show a similar behaviour.

Cooperation n  In [15], the multi-colony algorithm was

studied by having colonies connected in a directed-ring fashion,

n  Four information exchange approaches were

studied: n  All colonies get the same global best solution, n  Circular exchange of locally best solutions, n  Circular exchange of a number of ants (migrants), n  A mixture of the two previous approaches.

PSO - Motion n  Each particle holds:

n  Its current position xi , n  Its current velocity vi , n  The best position it achieved so far, personal best,

pbesti , or sometimes pi for short, n  The best position achieved by particles in its

neighbourhood Nbest n  If the neighbourhood is the whole swarm, the best

achieved by the whole swarm is called global best, gbesti , or sometimes pg for short.

n  If the neighbourhood is restricted to few particles, the best is called local best, lbest or lp

PSO - Motion n  Equations of motion:

where n  w is the inertia weight, n  c1, c2 are the acceleration coefficients, n  r1,r2 are randomly generated numbers in [0, 1], n  t is the iteration number, n  i and d are the particle number and the dimension.

( ) ( )idt

idt

idt

idt

idt

ididt

idt

ididt

idt

vxxxNbestrcxpbestrcvwv

11

22111 *

++

+

+=

−+−+=

PSO - Motion

n  After that, each particle updates its own personal best (assuming a minimization problem):

( ) ( )⎩⎨⎧ ≤

= +++ otherwise,

, 111 i

t

it

it

iti

t pbestpbestfxfifx

pbest

PSO - Motion

n  After that, each swarm updates its global best (assuming a minimization problem):

( )i1t1 pbestf min argNbest

1+∈+

+

=Npbest

it i

t

PSO - Motion n  An important factor to set is the maximum

velocity allowed for the particles Vmax : n  If too high, particles can fly past optimal solutions, n  If too low, particles can get stuck in local optima.

n  Usually set according to the domain of the search space.

PSO – A simple algorithm (Synchronous update)

n  Initialize the swarm, n  While termination criteria is not met

n  For each particle n  Update the particle’s velocity, n  Update the particle’s position, n  Update the particle’s personal best,

end for n  Update the Nbest,

end while

PSO – A different algorithm (Asynchronous update)

n  Initialize the swarm, n  While termination criteria is not met

n  For each particle n  Update the particle’s velocity, n  Update the particle’s position, n  Update the particle’s personal best, n  Update the Nbest,

end for

end while

The neighbourhood best update is moved into the particles update loop

PSO - Neighbourhoods n  gbes t mode l : e a ch

particle is influenced by all the other particles,

n  The fastest Propagation of information in a population,

n  Particles can get stuck easily in local minima.

PSO - Neighbourhoods n  l b e s t m o d e l : e a c h

particle is influenced only by particles in its own neighbourhood,

n  The propagat ion of

i n f o r m a t i o n i s t h e slowest,

n  Doesn’t get stuck easily in local minima but m igh t i nc rease the computational cost.

PSO - Neighbourhoods

The complete population A local region

The Von Neumann model [10]

PSO - Convergence n  B y a n a l y z i n g t h e

characteristic equation, we can find the conditions necessary for stability:

n  The convergence domain would be inside the triangle shown.

02*2,0,1

>+−><

cwcw

w

c

4

1 -1

2

A B

PSO - Convergence n  Two simulations are run

to optimize the Spherical f u n c t i o n u s i n g 2 0 particles in a domain range of [-100, 100],

n  The values used for [c,

w] are: n  A=[1.2, 1.49] for the first

run, n  B=[0.79, 1.49] for the

second run. w

c

4

1 -1

2

Discrete PSO n  PSO was originally developed for continuous-

valued spaces n  Many problems are defined for discrete

valued spaces n  Examples: Feature selection, TSP, Assignment

problems, Scheduling,.. n  Changes to PSO can be as simple as

discretization of the position vectors or complex as redefining of the arithmetic operations (addition, multiplication)

Binary PSO

n  Velocities are defined as probabilities that one element will be in one state or the other,

n  If 3.0=idtV

1=idtx 0=id

tx30% 70%

Binary PSO

n  Since velocities represent probabilities, the values of the velocity elements need to be restricted in the range [0,1],

n  This is done by using the sigmoid function:

( ) idtV

idt

eVsig

−+=11

Binary PSO n  The sigmoid function:

-4 -3 -2 -1 0 1 2 3 40

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Vid

S(Vid)

Permutation PSO n  In [12], PSO was applied to solve the TSP,

n  The position of a particle was the solution to a problem (permutation of cities),

n  The velocity of a particle was defined as the

set of swaps to be performed on a particle.

Permutation PSO

n  Three operations were re-defined for the new search space: n  Adding a velocity to a position, n  Subtracting two positions, n  Multiplying a velocity by a constant.

Permutation PSO n  A different PSO approach was applied to the

TSP in [15],

n  A continuous PSO version was used, n  In order to evaluate the particle fitness, they

performed a space transformation from a particle in the continuous domain to a permutation in the solution space.

Permutation PSO n  The used rule known as Great Value Priority

(GVP), n  First, all the elements in the position vector

(along with their indices) were sorted to get a sorted list in descending order,

n  The sorted indices were taken as the

permutation.

Permutation PSO

n  The idea was that if xi had the highest value in the position vector, it means that city number i comes first in the permutation vector,

n  This transformation was carried before calculating the particles fitness.

Applications

n  PSO has been successfully applied to many applications including: n  FPGA Placement n  Neural network training, n  Robot motion planning. n  Clustering.

Adaptation n  An adaptive parameter free PSO, referred to

as TRIBES, was proposed in [17],

n  The purpose was to have the user to call PSO algorithm without the need to set any parameters,

n  The main point was to adapt the number

particles used in the search.

Cooperation n  Different approaches have been

proposed to introduce cooperation among different swarms,

n  These approaches include: n  Concurrent PSO, n  Cooperative PSO, n  Hybrid Cooperative PSO.