population based algorithms - school of computer science

Artificial Intelligence Search Methodologies

Dr Rong Qu

School of Computer Science

University of Nottingham

Nottingham, NG8 1BB, UK [email protected]

Population Based Algorithms

Konstanz, May 2012 AI Search Algorithms – Population Based 2

Optimisation Problems: Methods

Meta-heuristics Guide an underlying heuristic/search to escape

from being trapped in a local optima and to explore better areas of the solution space

Single solution approaches Simulated Annealing, Tabu Search, variable

neighbourhood search, etc;

Population based approaches Genetic algorithm, Memetic algorithm, Ant Algorithms,

Particle Swarm Intelligence, etc;


Population Based Algorithms

Simulated annealing, tabu search Concerning only one solution at a particular time during the

search Search is very much restricted to local regions, so called

local

Population based algorithms concern a population of solutions at a time

GENETIC ALGORITHMS


Charles Darwin

1809 - 1882


GA Algorithm – basic idea

Based on survival of the fittest

Algorithm uses terms from genetics: population,

chromosome and gene

Developed extensively by John Holland in mid 70’s

Three modules

the evaluation module, the population module and the

reproduction module

Solutions (individuals) often coded as bit strings



1859

Origin of the Species

Survival of the Fittest



1975

Genetic Algorithms

Artificial Survival of the Fittest



1989

Genetic Algorithms

Foundations and Applications


GA Algorithm – basic steps

Initial population

Evaluations on individuals

Breeding

Choose suitable parents (proportion to evaluation rating)

Produce two offspring (Probability of breeding)

Mutation

Domain knowledge – evaluation function



1. Initialise a population of chromosomes

2. Evaluate each chromosome (individual) in the population

Create new chromosomes by mating chromosomes in the

current population (using crossover and mutation)

Delete members of the existing population to make way for the

new members

Evaluate the new members and insert them into the population

Repeat (evolve) until some termination condition is reached

(normally based on time or number of populations produced)

3. Return the best chromosome as the solution



Generate Initial Population

Population Generation 'n'

Crossover Population

Mutate Population

n = n + 1

n = 1

Final Population

Selection

n<20?


GA Algorithm – encoding

The decision variables of a problem are normally encoded into a finite length string

This could be a binary string or a list of integers For example :

or 0 1 1 0 1 1 0 1 0 2 3 4 1 1 4 5

We could also represent numbers as coloured boxes


Evaluation Module

Responsible for evaluating a chromosome

Only part of the GA that has any knowledge about

the problem. The rest of the GA modules are simply

operating on (typically) bit strings with no

information about the problem

A different evaluation module is needed for each

problem


Population Module

Responsible for maintaining the population Initilisation

Random Known Solutions

Population Size Elitism


Population Module

Deletion

Delete-All : Deletes all the members of the current population and replaces them with the same number of chromosomes that have just been created

Steady-State : Deletes n old members and replaces them with n new members; n is a parameter But do you delete the worst individuals, pick them at random or delete the chromosomes that you used as parents?

Steady-State-No-Duplicates : Same as steady-state but checks that no duplicate chromosomes are added to the population. This adds to the computational overhead but can mean that more of the search space is explored


Reproduction Module

Parent selection

Fitness techniques

Crossover & mutation


Parent Selection

Chromosome 1 2 3 4 5 6 7 8 9 10

Fitness 12 18 15 17 3 136 12 15 20 15

Running Total 12 30 45 62 65 201 213 228 248 263

1 2 3 4 5 6 7 8 9 10

Random Number 234 156 8 174 219 255 143 94 210 31

Chromsome Chosen 9 6 #N/A 6 8 10 6 6 7 3

NotesPress F9 to regenerate random numbers

Notice how C6 tends to dominate due to its dominant fitness

Roulette Wheel Selection

•Sum the fitnesses of all the population members, TF

•Generate a random number, m, between 0 and TF

•Return the first population member whose fitness added to the preceding population members is greater than or equal to m


Parent Selection

Tournament

Select a pair of individuals at random. Generate a random number, R, between 0 and 1. If R < r use the first individual as a parent. If the R >= r then use the second individual as the parent. This is repeated to select the second parent. The value of r is a parameter to this method

Select two individuals at random. The individual with the highest evaluation becomes the parent. Repeat to find a second parent


Fitness Techniques

• Fitness-Is-Evaluation : Simply have the fitness of the chromosome equal to its evaluation

• Windowing : Takes the lowest evaluation and assigns each chromosome a fitness equal to the amount it exceeds this minimum

• Linear Normalization : The chromosomes are sorted by decreasing evaluation value. Then the chromosomes are assigned a fitness value that starts with a constant value and decreases linearly. The initial value and the decrement are parameters to the techniques


Crossover Operators

One Point Crossover inOne Point Crossover in

Genetic AlgorithmsGenetic Algorithms

© Graham Kendall

[email protected]

http://cs.nott.ac.uk/~gxk

Uniform Crossover inUniform Crossover in

Genetic AlgorithmsGenetic Algorithms

© Graham Kendall

[email protected]

http://cs.nott.ac.uk/~gxk

Order Based Crossover

Cycle Crossover

Partially matched crossover


Mutation

• A method of ensuring premature convergence does not occur

• Usually set to a small value

• Dynamic mutation and crossover rates


Example I

• Crossover probability, PC = 1.0 • Mutation probability, PM = 0.0 • Maximise f(x) = x3 - 60 * x2 + 900 * x +100 • 0 <= x >= 31 • x can be represented using five binary digits

0 100

1 941

2 1668

3 2287

4 2804

5 3225

6 3556

7 3803

8 3972

9 4069

10 4100

11 4071

12 3988

13 3857

14 3684

15 3475

16 3236

17 2973

18 2692

19 2399

20 2100

21 1801

22 1508

23 1227

24 964

25 725

26 516

27 343

28 212

29 129

30 100

0

500

1000

1500

2000

2500

3000

3500

4000

4500

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37

Max : x = 10

f(x) = x^3 - 60x^2 + 900x + 100


Example I

• Generate random initial individuals Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100 (0 <= x <= 31)

chromosome binary string x f(x)

P1 11100 28 212

P2 01111 15 3475

P3 10111 23 1227

P4 00100 4 2804

Total 7718

Average 1929.50


Example I

• Choose Parents, using roulette wheel selection • Crossover point, 1, is chosen randomly

Roulette wheel Parents

4116 P3

1915 P2

1 0 1 1 1

0 1 1 1 1

P3

P2

1 1 1 1 1

0 0 1 1 1

C1

C2

0 0 1 0 0

0 1 1 1 1

P4

P2

0 0 1 1 1

0 1 1 0 0

C3

C4


Example I

New generation

chromosome binary string x f(x)

P1 11111 31 131

P2 00111 7 3803

P3 00111 7 3803

P4 01100 12 3889

Total 11735

Average 2933.75

Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100 (0 <= x <= 31)


Example I

chromosome binary string

x f(x)

P1 11100 28 212

P2 01111 15 3475

P3 10111 23 1227

P4 00100 4 2804

Total 7718

Average 1929.50

chromosome binary string

x f(x)

P1 11111 31 131

P2 00111 7 3803

P3 00111 7 3803

P4 01100 12 3889

Total 11735

Average 2933.75

Two generations

Mutation

Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100 (0 <= x <= 31)

What problem do you see with the populations?

what chance is there of finding the global optimum?


GA - performance

There are a number of factors which affect the performance of a genetic algorithm The size of the population The initial population Selection pressure (elitism, tournament) The cross-over probability The mutation probability Defining convergence Local optimisation


GA - applications

Combinatorial optimisation problems bin packing problems

vehicle routing problems

job shop scheduling


GA - applications

Combinatorial optimisation problems portfolio optimization

multimedia multicast routing

knapsack problem

ANT ALGORITHMS


Ants are practically blind but they still manage to find their way to and from food. How do they do it?


Ant Algorithms

Ant systems are a population based approach. In this respect it is similar to genetic algorithms

There is a population of ants, with each ant finding a solution and then communicating with the other ants


Ant Algorithms

A

B

C

H

D

F

E

G


Ant Algorithms

A

B

C

D

F

E

d=0.5

d=0.5

d=1

d=1

d=1

d=1


Ant Algorithms

Time, t, is discrete

At each time unit an ant moves a distance, d, of 1

Once an ant has moved it lays down 1 unit of pheromone

At t=0, there is no pheromone on any edge


Ant Algorithms

At t=1 there will be 16 ants at B

and 16 ants at D.

At t=2 there will be 8 ants at D

and 8 ants at B. There will be 16

ants at E

The intensities on the edges will

be as follows

FD = 16, AB = 16, BE = 8, ED =

8, BC = 16 and CD = 16

A

B

C

D

F

E

0.5

0.5

1

1

1 1

16 ants are moving from

A - F and another 16 are

moving from F - A


Ant Algorithms

We are interested in exploring the search space, rather than simply plotting a route

We need to allow the ants to explore paths and follow the best paths with some probability in proportion to the intensity of the pheromone trail

We do not want them simply to follow the route with the highest amount of pheromone on it, else our search will quickly settle on a sub-optimal (and probably very sub-optimal) solution


Ant Algorithms

The probability of an ant following a certain route is a function, not only of the pheromone intensity but also a function of what the ant can see (visibility)

The pheromone trail must not build unbounded. Therefore, we need “evaporation”


Ant Algorithms – initial ideas

Dorigo (1996)

Based on real world phenomena

Ants, despite almost blind, are able to find their way to the food source using the shortest route

If an obstacle is placed, ants have to decide which way to take around the obstacle.



Dorigo (1996)

Initially there is a 50-50 probability as to which way they will turn

Assume one route is shorter than the other

Ants taking the shorter route will arrive at a point on the other side of the obstacle before the ants which take the longer route.



Dorigo (1996)

As ants walk they deposit pheromone trail.

Ants have taken shorter route will have already laid trail

So ants from the other direction are more likely to follow that route with deposit of pheromone.



Dorigo (1996) Over a period of time, the shortest route will have

high levels of pheromone.

The quantity of pheromones accumulates faster on the shorter path than on the longer one

There is positive feedback which reinforces that behaviors so that the more ants follow a particular route, the more desirable it becomes.


Ant Algorithms


Ant Algorithms - TSP

At the start of the algorithm one ant is placed in each city

Time, t, is discrete. t(0) marks the start of the algorithm. At t+1 every ant will have moved to a new city

Assuming that the TSP is being represented as a fully connected graph, each edge has an intensity of trail on it. This represents the pheromone trail laid by the ants

Let Ti,j(t) represent the intensity of trail edge (i,j) at time t

Variations have been tested by Dorigo



When an ant decides which town to move to next, it does so with a probability that is based on the distance to that city AND the amount of trail intensity on the connecting edge

The distance to the next town, is known as the visibility, nij, and is defined as 1/dij, where, dij, is the distance between cities i and j.

At each time unit evaporation takes place

The amount of evaporation, p, is a value between 0 and 1




In order to stop ants visiting the same city in the same tour a data structure, Tabu, is maintained

This stops ants visiting cities they have previously visited

Tabuk is defined as the list for the kth ant and it holds the cities that have already been visited




After each ant tour the trail intensity on edge (i,j) is updated using the following formula

Tij (t + n) = p . Tij(t) + ΔTij

otherwise

ntandttimebetween

touritsinjiedgeusesantkththeif

L

Q

kk

ijT)(

),(

0

Q is a constant

Lk is the tour length of the kth ant

p is the evaporation coefficient

By using this rule, the probability increases that forthcoming ants will use edge (i, j)



Transition Probability

otherwise

allowedjif

ntTallowedk

ntTt

k

ikikk

ijijk

ijp

0

][)]([

][)]([)(

..

where and are control parameters that control

the relative importance of trail versus visibility



• If you are interested (and willing to do some

work) there is a spreadsheet that implements

some of the above formula

Numerator Denominator

Move A to A TRUE 1.73 0.00 Visibility A to A 1.00 0.00 1.73 0.00

Move A to A FALSE 1.73 1.73 Visibility A to A 1.00 1.00 1.73 1.73

Move A to B FALSE 1.73 1.73 Visibility A to B 0.89 0.89 1.55 1.55

Move A to B FALSE 1.73 1.73 Visibility A to B 0.89 0.89 1.55 1.55 Distance TableMove A to C FALSE 1.73 1.73 Visibility A to C 0.93 0.93 1.62 1.62 A A B B C D E

Move A to D FALSE 1.73 1.73 Visibility A to D 0.99 0.99 1.72 1.72 A 1.00

Move A to E FALSE 1.73 1.73 Visibility A to E 0.77 0.77 1.33 1.33 A 1.00

Move A to F FALSE 1.73 1.73 Visibility A to F 0.79 0.79 1.37 1.37 B 0.80

Move A to F FALSE 1.73 1.73 Visibility A to F 0.79 0.79 1.37 1.37 B 0.80

Move A to G FALSE 1.73 1.73 Visibility A to G 0.88 0.88 1.52 1.52 C 0.87

Move A to H FALSE 1.73 1.73 Visibility A to H 0.98 0.98 1.70 1.70 D 0.99

Move A to H FALSE 1.73 1.73 Visibility A to H 0.98 0.98 1.70 1.70 E 0.59

Move A to I FALSE 1.73 1.73 Visibility A to I 0.97 0.97 1.69 1.69 F 0.63

F 0.63

SUM's 22.52 20.78 11.88 10.88 20.58 18.8514898738 G 0.77

H 0.96

H 0.96

Probability A to A 0.00000 I 0.95

Probability A to A 0.09188

Probability A to B 0.08218 Trail Edge TableProbability A to B 0.08218 A A B B C D E

Probability A to C 0.08570 A 3.00

Probability A to D 0.09142 A 3.00

Probability A to E 0.07057 B 3.00

Probability A to F 0.07293 B 3.00

Probability A to F 0.07293 C 3.00

Probability A to G 0.08062 D 3.00

Probability A to H 0.09002 E 3.00

Probability A to H 0.09002 F 3.00

Probability A to I 0.08955 F 3.00

G 3.00

H 3.00

This spreadsheet models the transition probability shown in the paper [ref12] H 3.00

See notes, if necessary I 3.00

Trail Edge Constant0.5

Visibility Constant0.5



Left: Trail distribution at the beginning;

Right: Trail distribution after 100 cycles. (Dorigo et al., 1996)


Ant Algorithms - Applications

Travelling Salesman Problem (TSP)

Facility Layout Problem

Vehicle Routing

Stock Cutting

…

Marco Dorigo maintains a page devoted to the subject at http://iridia.ulb.ac.be/~mdorigo/ACO/ACO.html

contains information about ant algorithms as well as links to the main papers published on the subject

APPENDIX


Examples of Genetic Algorithms

Genetic Algorithm Example II

Traveling Salesman Problem a number of cities costs of traveling between cities

a traveling sales man needs to visit all these cities exactly once and return to the starting city What’s the cheapest route?

Konstanz, May 2012 52 AI Search Algorithms – Population Based

Traveling Salesman Problem



Initial generation 5 8 1 … … 84 32 27 54 67 P1

78 81 27 … … 9 11 7 44 24 P2

8 1 7 … … 9 16 36 24 19 P30

…

6.5

7.8

6.0

Any idea of other ways to generate the initial population?



Choose pairs of parents

78 81 27 … … 9 11 7 44 24 P2

8 1 7 … … 9 16 36 24 19 P30 6.0

7.8

Crossover

78 81 27 … … 9 16 36 24 19 C2

8 1 7 … … 9 11 7 44 24 C1 5.9

6.2

13



Next generation

78 81 27 … … 9 16 36 24 19 P2

8 1 7 … … 9 11 7 44 24 P1 5.9

6.2

6.0

…


7 8 2 … … 5 10 76 4 79 P2


Traveling Salesman Problem No. of cities: 100 Population size: 30

Cost: 6.37 Generation: 88

Cost: 6.34 Generation: 1100



There are many diverse applications of genetic algorithms. They are best suited to problems where the efficient solutions are not already known. If they are applied to solvable problems, they will be easily out-performed by efficient standard computing methods. The strength of GA's is their ability to heuristically search for solutions when all else fails. If you can represent the solutions to the problem in a suitable format, such as a series of 1's and 0's, then the GA will do the rest.


Genetic Algorithm Example III

Applying Genetic Algorithms

to Personnel Scheduling

Personnel scheduling in healthcare is usually a very complex operation

which has a profound effect upon the efficient usage of expensive resources.



A number of nurses A number of shifts each day A set of constraints

shift coverage one shift per day resting time workload per month consecutive shifts working weekends …



Genetic Algorithm -Initial population

-construct rosters -repair infeasible ones



Genetic Algorithm -Select parents -Recombine rows in the two rosters

-repair infeasible ones

+



Genetic Algorithm -Mutation -Local optimiser


Population Size Crossover Probability Mutation Probability Local Optimiser

50 0.7

0.001 ON



population based algorithms - school of computer science

Documents