artificial intelligence search methodologies

Artificial Intelligence Search Methodologies

Dr Rong QuSchool of Computer Science

University of NottinghamNottingham, NG8 1BB, UK

[email protected]

Population Based Algorithms

Konstanz, May 2014 AI Search Algorithms – Population Based 2

Optimisation Problems: Methods

Meta-heuristics Guide an underlying heuristic/search to

escape from being trapped in a local optima and to explore better areas of the solution space

Single solution approaches Simulated Annealing, Tabu Search, Variable

Neighbourhood Search, etc.; Population based approaches

Genetic algorithm, Memetic algorithm, Ant Algorithms, Particle Swarm Intelligence, etc.;

AI Search Algorithms – Population Based 3

Population Based Algorithms

Local search Concerning only one solution at a particular

time during the search Search is very much restricted to local regions,

so called local Population based algorithms

concern a population of solutions at a time

Konstanz, May 2014

GENETIC ALGORITHMS


Charles Darwin1809 - 1882


GA Algorithm – basic idea

Based on survival of the fittest Algorithm uses terms from genetics: population,

chromosome and gene Developed extensively by John Holland in

mid 70’s Three modules

the evaluation module, the population module and the reproduction module

Solutions (individuals) often coded as bit strings



1859

Origin of the Species

Survival of the Fittest



1975

Genetic Algorithms

Artificial Survival of the Fittest



1989

Genetic Algorithms

Foundations and Applications


GA Algorithm – basic steps

Initial population Evaluations on individuals Breeding

Choose suitable parents (proportion to evaluation rating)

Produce two offspring (Probability of breeding)

MutationDomain knowledge – evaluation function



1. Initialise a population of chromosomes Ci2. Evaluate each Ci (individual) in the population

Create new C by using Ci in the current population (using crossover and mutation)

Delete members of the existing population to make way for the new members

Evaluate the new members and insert them into the population

Repeat (evolve) until some termination condition is reached (normally based on time or number of populations produced)

3. Return the best Ci as the solution



Generate Initial Population

Generate Initial Population

Population Generation 'n'

Population Generation 'n'

Crossover Population

Crossover Population

Mutate Population

Mutate Population

n = n + 1 n = n + 1

n = 1

Final Population

Final Population

SelectionSelection

n<20?


GA Algorithm – encoding

The decision variables of a problem arenormally encoded into a finite length string

This could be a binary string or a list of integersFor example :

or0 1 1 0 1 1 0 1 0 2 3 4 11 45

We could also representnumbers as coloured boxes


Evaluation Module

Responsible for evaluating a chromosome Only part of the GA that has domain

knowledge. The rest of the GA modules are simply operating on (typically) bit strings with no information about the problem

A different evaluation module is needed for each problem


Population Module

Responsible for maintaining the population

Initialization Random Known Solutions Heuristics

Population Size Elitism


Population Module

Deletion Delete-All : replaces all the members of the

current population by the same number of chromosomes created

Steady-State : replaces n old members by n new members.But: replace the worst, random or the parents individuals?

Steady-State-No-Duplicates : Same as steady-state but also checks that no duplicate chromosomes are added to the population.


Reproduction Module

Parent selection Fitness techniques Crossover & mutation


Parent Selection

Roulette Wheel Selection Select the parents with a probability in

proportion to their fitness Tournament

Select two individuals at random. The individual with the highest evaluation becomes the parent. Repeat to find a second parent


Fitness Techniques

• Fitness-Is-Evaluation : Simply use the fitness of the chromosome equal to its evaluation

• Linear Normalization : The chromosomes are sorted by decreasing the evaluation value. Then the chromosomes are assigned a fitness value that starts with a constant value and decreases linearly. The initial value and the decrement are parameters to the techniques


Crossover Operators

One Point Crossover inOne Point Crossover inGenetic AlgorithmsGenetic Algorithms

© Graham Kendall

[email protected]

http://cs.nott.ac.uk/~gxk

Uniform Crossover inUniform Crossover inGenetic AlgorithmsGenetic Algorithms

© Graham Kendall

[email protected]

http://cs.nott.ac.uk/~gxk

Order based crossoverCycle crossoverPartially matched crossover


Mutation

• A method of ensuring premature convergence does not occur

• Usually set to a small value

• Dynamic mutation and crossover rates


Example I

• Crossover probability, PC = 1.0• Mutation probability, PM = 0.0• Maximise f(x) = x3 - 60 * x2 + 900 * x +100• 0 <= x >= 31• x can be represented using five binary digits

0 100

1 9412 16683 22874 28045 32256 35567 38038 39729 4069

10 410011 407112 398813 385714 368415 347516 323617 297318 269219 239920 210021 180122 150823 122724 96425 72526 51627 34328 21229 12930 100

0

500

1000

1500

2000

2500

3000

3500

4000

4500

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37

Max : x = 10

f(x) = x^3 - 60x^2 + 900x + 100


Example I

• Generate random initial individualsMaximise f(x) = x^3 - 60 * x^2 + 900 * x +100 (0 <= x <=

31)

chromosome binary string x f(x)

P1 11100 28 212

P2 01111 15 3475

P3 10111 23 1227

P4 00100 4 2804

Total 7718

Average 1929.50


Example I

• Choose Parents, using roulette wheel selection

• Crossover point, 1, is chosen randomlyRoulette wheel Parents

4116 P3

1915 P2

1 0 1 1 1

0 1 1 1 1

P3

P2

1 1 1 1 1

0 0 1 1 1

C1

C2

0 0 1 0 0

0 1 1 1 1

P4

P2

0 0 1 1 1

0 1 1 0 0

C3

C4


Example I

New generation

chromosome binary string x f(x)

P1 11111 31 131

P2 00111 7 3803

P3 00111 7 3803

P4 01100 12 3889

Total 11735

Average 2933.75

Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100 (0 <= x <= 31)


Example I

chromosome

binary string

x f(x)

P1 11100 28 212

P2 01111 15 3475

P3 10111 23 1227

P4 00100 4 2804

Total 7718

Average 1929.50

chromosome

binary string

x f(x)

P1 11111 31

131

P2 00111 7 3803

P3 00111 7 3803

P4 01100 12

3889

Total 11735

Average 2933.75

Two generations

Mutation

Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100 (0 <= x <= 31)

What problem do you see with the populations?

what chance is there of finding the global optimum?


GA - performance

There are a number of factors which affect the performance of a genetic algorithm The size of the population The initial population Selection pressure (elitism, tournament) The cross-over probability The mutation probability Defining convergence Local optimisation


GA - applications

Combinatorial optimisation problems Cutting and packing problems vehicle routing problems job shop scheduling


GA - applications

Combinatorial optimisation problems portfolio optimization multimedia multicast routing knapsack problem

ANT ALGORITHMS


Ants are practically blind but they still manage to find their way to and from food. How do they do it?


Ant Algorithms

Ant systems are a population based approach. In this respect it is similar to genetic algorithms

There is a population of ants, with each ant finding a solution and then communicating with the other ants


Ant Algorithms

A

B

C

H

D

F

E

G


Ant Algorithms

A

B

C

D

F

E

d=0.5

d=0.5

d=1

d=1

d=1

d=1


Ant Algorithms

Time, t, is discrete

At each time unit an ant moves a distance, d of 1

Once an ant moved, it lays down 1 unit of pheromone

At t = 0, there is no pheromone on any edge


Ant Algorithms

At t=1 there will be 16 ants at B and 16 ants at D.At t=2 there will be 8 ants at D and 8 ants at B. There will be 16 ants at EThe intensities on the edges will be as follows

FD = 16, AB = 16, BE = 8, ED = 8, BC = 16 and CD = 16

A

B

C

D

F

E

0.5

0.5

1

1

11

16 ants are moving from A - F and another 16 are

moving from F - A


Ant Algorithms

We need to allow the ants to explore paths and follow the best paths with some probability in proportion to the intensity of the pheromone trail

We do not want them simply to follow the route with the highest amount of pheromone on it, else our search will quickly settle on a sub-optimal (and probably very sub-optimal) solution


Ant Algorithms

The probability of an ant following a certain route is a function, not only of the pheromone intensity but also a function of what the ant can see (visibility)

The pheromone trail must not build unbounded. Therefore, we need “evaporation”


Ant Algorithms – initial ideas

Dorigo (1996) Based on real world phenomena Ants, despite almost blind, are able to find

their way to the food source using the shortest route

If an obstacle is placed, ants have to decide which way to take around the obstacle.



Dorigo (1996) Initially there is a 50-50 probability as to

which way they will turn Assume one route is shorter than the other Ants taking the shorter route will arrive at a

point on the other side of the obstacle before the ants which take the longer route.



Dorigo (1996) As ants walk they deposit pheromone trail. Ants have taken shorter route will have

already laid trail So ants from the other direction are more

likely to follow that route with deposit of pheromone.



Dorigo (1996) Over a period of time, the shortest route will

have high levels of pheromone. The quantity of pheromones accumulates faster

on the shorter path than on the longer one There is positive feedback which reinforces that

behaviors so that the more ants follow a particular route, the more desirable it becomes.


Ant Algorithms


Ant Algorithms - TSP

At the start of the algorithm, one ant is placed in each city

Time, t, is discrete. t(0) marks the start of the algorithm. At t+1 every ant will have moved to a new city

Assuming that the TSP is represented as a fully connected graph, each edge has an intensity of trail on it. This represents the pheromone trail laid by the ants

Let Ti,j(t) represent the intensity of trail edge (i,j) at time t

Variations have been tested by Dorigo



An ant decides which town to move to next, with a probability that is based on the distance to that city AND the amount of trail intensity on the connecting edge

The distance to the next town, is known as the visibility, nij, defined as 1/dij, dij is the distance between cities i and j.

At each time unit evaporation takes place, at a rate p, a value between 0 and 1




In order to stop ants visiting the same city in the same tour a data structure, Tabu, is maintained

Tabuk is defined as the list for the kth ant and it holds the cities that have already been visited




After each ant tour the trail intensity on edge (i,j) is updated using the following formula

Tij (t + n) = p . Tij(t) + ΔTij

otherwise

ntandttimebetween

touritsinjiedgeusesantkththeif

L

Q

kk

ijT )(

),(

0

Q is a constantLk is the tour length of the kth antp is the evaporation coefficient

otherwise

allowedjifntTallowedk

ntTt

k

ikikk

ijijk

ijp

0

][)]([

][)]([)( .

.



Transition Probability

where and are control parameters that control the relative importance of trail versus visibility



Left: Trail distribution at the beginning;Right: Trail distribution after 100 cycles. (Dorigo et al., 1996)


Ant Algorithms - Applications

Travelling Salesman Problem (TSP) Facility Layout Problem Vehicle Routing Stock Cutting …

Marco Dorigo maintains a page devoted to the subject at

http://iridia.ulb.ac.be/~mdorigo/ACO/ACO.html contains information about ant algorithms as well as

links to the main papers published on the subject

APPENDIX


Examples of Genetic Algorithms

Genetic Algorithm Example II Genetic Algorithm Example II

Traveling Salesman Problema number of citiescosts of traveling between cities

a traveling sales man needs to visit all these cities exactly once and return to the starting city

What’s the cheapest route?

Konstanz, May 2014 50AI Search Algorithms – Population Based

Traveling Salesman Problem



Initial generation5 8 1 … … 84 32 27 54 67P1

78 81 27 … … 9 11 7 44 24P2

8 1 7 … … 9 16 36 24 19P30

…

6.5

7.8

6.0

Any idea of other ways to generate the initial population?



Choose pairs of parents

78 81 27 … … 9 11 7 44 24P2

8 1 7 … … 9 16 36 24 19P30 6.0

7.8

Crossover

78 81 27 … … 9 16 36 24 19C2

8 1 7 … … 9 11 7 44 24C1 5.9

6.2

13



Next generation

78 81 27 … … 9 16 36 24 19P2

8 1 7 … … 9 11 7 44 24P1 5.9

6.2

6.0…


7 8 2 … … 5 10 76 4 79P2


Traveling Salesman ProblemNo. of cities: 100Population size: 30

Cost: 6.37Generation: 88

Cost: 6.34Generation: 1100



Many applications of genetic algorithms.Best suited to problems where the efficient solutions are not already known.Strength of GA's: ability to heuristically search for solutions when all else fails.If you can represent the solutions to the problem in a suitable format, such as a series of 1's and 0's, then the GA will do the rest.


Genetic Algorithm Example III Genetic Algorithm Example III

Applying Genetic Algorithms

to Personnel Scheduling

Applying Genetic Algorithms

to Personnel Scheduling

Personnel scheduling in healthcareis usually a very complex operation

which has a profound effect upon theefficient usage of expensive resources.



A number of nursesA number of shifts each dayA set of constraints

shift coverageone shift per dayresting timeworkload per monthconsecutive shiftsworking weekends…



Genetic Algorithm- Initial population

- construct rosters- repair infeasible ones



Genetic Algorithm- Select parents- Recombine rows in the two rosters

- repair infeasible ones

+



Genetic Algorithm- Mutation- Local optimiser


Population SizeCrossover ProbabilityMutation ProbabilityLocal Optimiser

500.7

0.001ON



artificial intelligence search methodologies

Documents

population module

current population

existing population

population of solutions

tabu search

ant algorithms

population of chromosomes

variable neighbourhood