an investigation into the suitability of the ant colony

An Investigation into the Suitability of the Ant Colony OptimizationHeuristic in Noisy and Dynamic Environments

by

Andrew Schoenrock100326415

Honours Project (COMP 4905)Submitted in fulfilment of therequirements for the degree of

Bachelor of Computer Science, Honours

at

Carleton UniversityOttawa, OntarioDecember, 2006

Supervised by Dr. Tony White

Contents

1 Abstract 5

2 Acknowledgements 7

3 Introduction 8

3.1 Inspiration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.2 NP-Hard Problems . . . . . . . . . . . . . . . . . . . . . . . . 9

3.3 Travelling Salesman problem . . . . . . . . . . . . . . . . . . . 9

3.4 Extensions to TSP . . . . . . . . . . . . . . . . . . . . . . . . 11

4 ACO Heuristic 13

4.1 Ant System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4.2 Ant Colony System . . . . . . . . . . . . . . . . . . . . . . . . 14

4.2.1 Transition Rule . . . . . . . . . . . . . . . . . . . . . . 14

4.2.2 Pheromone Trail Update Rule . . . . . . . . . . . . . . 15

4.2.3 Local Pheromone Updates . . . . . . . . . . . . . . . . 15

4.2.4 Pheromone Decay . . . . . . . . . . . . . . . . . . . . . 16

4.2.5 Candidate Lists . . . . . . . . . . . . . . . . . . . . . . 16

4.2.6 ASC Pseudocode . . . . . . . . . . . . . . . . . . . . . 17

5 Problem Descriptions and Algorithm

Modifications 18

5.1 Problem 1: Noisy Environment . . . . . . . . . . . . . . . . . 18

1

5.1.1 Adding Noise . . . . . . . . . . . . . . . . . . . . . . . 19

5.1.2 Adapting to Noise . . . . . . . . . . . . . . . . . . . . 20

5.2 Problem 2: Dynamic Environment . . . . . . . . . . . . . . . 22

5.2.1 Pheromone Lower Bound . . . . . . . . . . . . . . . . . 24

5.2.2 Local Pheromone Smoothing . . . . . . . . . . . . . . . 24

6 Test Results 26

6.1 Problem 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

6.2 Problem 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

7 Discussion 43

7.1 Problem 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

7.2 Problem 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

8 Looking Forward 47

9 References 49

List of Tables

1 Paramaters Used in Tests . . . . . . . . . . . . . . . . . . . . 26

2 51 City Test Problem Used . . . . . . . . . . . . . . . . . . . . 27

3 Problem 1 Test Results on Low to High Noise Values . . . . . 28

4 Problem 1 Test Results on Extremely High Noise Values . . . 32

5 Problem 2 Test Results for 5 Moving Cities . . . . . . . . . . . 37

2

6 Problem 2 Test Results for 15 Moving Cities . . . . . . . . . . 37




10 Problem 2 Overall Test Results . . . . . . . . . . . . . . . . . 39

List of Figures

1 6 City Travelling Salesman Problem Example . . . . . . . . . 11

2 6 City TSP with 3 Moving Cities at Time t . . . . . . . . . . 22

3 6 City TSP with 3 Moving Cities at Time t + n . . . . . . . . 23

4 Best Tour Lengths Found at Different Noise Levels . . . . . . 29

5 Average Tour Lengths Found at Different Noise Levels . . . . 30

6 Standard Deviation of Tour Lengths Found at Different Noise

Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

7 Best Tour Lengths Found at Extremely High Noise Levels . . 33

8 Average Tour Lengths Found at Extremely High Noise Levels 34

9 Standard Deviation of Tour Lengths Found at Extremely High

Noise Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

10 Average Difference of Solution Lengths with Different Num-

bers of Moving Cities . . . . . . . . . . . . . . . . . . . . . . . 40

11 Standard Deviation of Solution Lengths with Different Num-

bers of Moving Cities . . . . . . . . . . . . . . . . . . . . . . . 41

3

12 Solution Quality with Different Numbers of Moving Cities . . 42

4

1 Abstract

This paper will discuss the ability that the Ant Colony Optimization heuris-

tic has to adapt to different environmental disturbances. Ant Colony Opti-

mization is a popular swarm intelligence heuristic which maps problems onto

graphs where an optimal solution of the problem is represented as a short

Hamiltonian path through said graph. Two separate problems will be looked

at. The first problem deals with a noisy environment where the length of

the edges of the graph are subjected to normally distributed noise every time

they are polled by the algorithm. The second problem deals with dynamic

environments where a random subset of the vertices of the graph are con-

stantly moving. With a thorough examination of these problems and the

measures taken to counteract them, more will be known about the overall

abilities of the ACO heuristic. The changes to the original algorithm were

done in a very efficient manner, which allows the heuristic to overcome these

disturbances without significantly increasing the running time.

The paper will begin with an introduction to the field of swarm intelli-

gence, Ant Colony Optimization and the the Travelling Salesman Problem. A

detailed look at the original Ant Colony Optimization is then examined. The

two problems are then looked at in detail and general algorithm modifica-

tions are discussed. In depth testing results are then given for each problem,

followed by a discussion of their implications. The paper concludes with a

5

look at how these ideas can be extended for further study.

6

2 Acknowledgements

I would like to thanks Dr. White for supervising this project and his continual

guidance and suggestions. I would also like to thank my friends and family

for their support.

7

3 Introduction

3.1 Inspiration

Swarm intelligence has been a very popular and important research area

in computer science in the last 15 years. These artificial intelligence tech-

niques are based on using collective behaviour in a decentralized manner to

accomplish a desired task. These systems are typically made up of simple

agents who can communicate with other agents either directly or indirectly

through modifications of their environment. These agents then act on this

information which helps the task at hand get accomplished. This method of

indirect communication to guide a group of simple agents is called stigmergy

and is vital to these heuristics. Many swarm intelligence ideas were inspired

from nature, be it the way birds search for their prey, or the way that ants

manoeuvre themselves in an organized manner. This paper will be looking

at a swarm intelligence heuristic known as Ant Colony Optimization. This

heuristic was inspired by the way in which real ants actually organize them-

selves to find short paths between destinations. The way it works is quite

simple, but it has impressive results. At first the ants seem to travel in a

random fashion, not really paying attention to their surroundings or fellow

ants. As these ants move around, they release a chemical called pheromone,

which other ants are attracted to. Over time, popular paths are carved out

by many ants travelling through the same areas, and other, worse, paths are

8

forgotten as the older pheromone evaporates. The notion of indirect com-

munication between ants, through the use of pheromone, allows them to find

short paths between their initial and destination points quite quickly. This

behaviour is the basis for the Ant Colony Optimization heuristic.

3.2 NP-Hard Problems

Swarm Intelligence heuristics, in particular ACO, are a popular choice when

dealing with NP-Hard problems. When dealing with these kind of problems,

one generally does not expect to achieve the optimal solution, rather reach

a solution that meets certain requirements. Swarm intelligence heuristics

generally offer a good compromise between computation time and solution

quality. These heuristics can give a very good solution in a fraction of the

time of solving a NP-Hard problem completely. Specifically, the Ant Colony

Optimization heuristic has been used to find solutions for popular NP-Hard

problems like the set-packing problem [6], wireless network routing [9], com-

binatorial optimization [7], assembly line balancing [8] and many, many more.

3.3 Travelling Salesman problem

The Travelling Salesman problem is a very popular problem in computer

science. Essentially this problem consists of a salesman who has to travel

to a number of different cities and is interested in finding the shortest route

between all of these cities. Since this problem is in the class of NP-Hard prob-

9

lems, there is no efficient, polynomial time algorithm to solve it. There are

numerous ways to find acceptable solutions to this, one being Ant Colony Op-

timization. This is the problem which this paper will focus on. The travelling

salesman problem is a convenient problem to use Ant Colony Optimization

on because it is intuitively easy to understand and also easy to implement.

Since the ACO heuristic solves problems by finding short paths through a

corresponding graph, when dealing with the travelling salesman problem the

cities will just become nodes in a graph, with their edges representing the dis-

tances between cities. This paper will look at the undirected version of this

problem where the distance between cities A and B is equal to the distance

between cities B and A. Below is an example of the Travelling Salesman

Problem with 6 cities. All possible paths are shown, with the optimal path

highlighted.

10

Figure 1: 6 City Travelling Salesman Problem Example

3.4 Extensions to TSP

It is already widely known that the ACO heuristic works very well at find-

ing very short, if not the shortest, paths in many instances of the travelling

salesman problem. This paper will deal with two extensions on this problem.

The first instance to be looked at is when we are dealing with the travel-

ling salesman problem in a noisy environment. This means that whenever

an ant queries the environment as to how far city x is away from the city

it is currently at, it will receive a noisy value. The noisy response will be

11

normally distributed around the actual value, with the standard deviation

being a runtime parameter. The second problem will focus on an instance of

the TSP where a random subset of cities move in a constant, random direc-

tion. The number of cities to be moved will be a runtime parameter. Both

of these problems will be discussed as far as implementation and original

algorithm modifications in further sections. The aim of the modifications

are to let the algorithm adapt to the new, noisy or dynamic environments

without them being expensive in terms of computation time or memory use.

These extensions, although maybe not both relevant to the travelling sales-

man problem, both occur in the real world and can effect potential problems

that the ACO heuristic could be used to solve. These tests will be either

expanding the bounds of what the ACO heuristic can solve or will show the

heuristic’s weakness in certain inconsistent environments. Inconsistent envi-

ronments are a part of life, and as such we should know how this well used

heuristic will perform in these types of situations.

12

4 ACO Heuristic

This paper will be using the Ant Colony System implementation of the ACO

heuristic [3]. This implementation is similar to the original Ant System

implementation [3] with a few improvements. This code was built off of a

basic Ant Colony System obtained through private communication with Dr.

Tony White.

4.1 Ant System

The AS implementation worked is the following way. First, the problem

must be transformed into a graph problem where an optimal solution can be

mapped onto a short path over the graph which visits each node exactly once

and ends up where it began (a Hamiltonian cycle). The Ant System then

builds its solutions gradually in the following manner. During each iteration,

each ant in the colony builds a tour throughout the graph, visiting each node

exactly once. For each ant, the transition from city a to b depends on the

following things:

1. If city b has already been visited. Each ant has a memory of which

cities it has already visited on this tour and makes sure not to revisit

previously visited cities.

2. The visibility of city b. This is calculated by taking the inverse of the

distance between a and b.

13

3. The amount of pheromone on the edge between a and b. The pheromone

trails are updated on each tour by each ant and represent how desir-

able a path is. This is where the indirect communication is used to

guide ants along previously known, good solutions through the use of

pheromone.

The way in which each city is chosen and how the pheromone levels are

update will be discussed in the next section, as they have been altered.

4.2 Ant Colony System

The Ant Colony System generally works in the same manner as the Ant

System, with a few minor alterations. These new special features will be

discussed in the following subsections.

4.2.1 Transition Rule

The transition rule is defined as follows. An ant k on city i chooses the city

j to move to following the rule:

j =

argmaxuεJk

i{[τiu(t)] · [ηiu]

β} if q ≤ q0;

J if q > q0,

where q is a random number uniformly distributed over [0,1], q0 is a tunable

runtime parameter where 0 ≤ q0 ≤ 1, τiu(t) is the amount of pheromone on

edge iu on iteration t, ηiu is the visibility of u from i, β is a tunable runtime

14

parameter, and JεJki is a city randomly selected, according to the following

formula:

pki =

[τiJ(t)] · [ηiJ ]β∑lεJk

i[τil(t)] · [ηil]β

4.2.2 Pheromone Trail Update Rule

The global pheromone updating formula only applies to the best tour found.

It is applied, on iteration t in the following manner:

τij(t)← (1− p) · τij(t) + p ·∆τij(t)

Where each ij is an edge of the best tour found thus far, p is the tunable

runtime parameter that governs the decay of the pheromone, and

∆τij(t) = 1/L,

where L is the length of the best tour found thus far.

4.2.3 Local Pheromone Updates

The local pheromone updates take place as each ant is performing a tour.

When an ant travels over edge ij the following rule is applied:

τij(t)← (1− p) · τij(t) + p · τ0,

15

where τ0 is the initial amount of pheromone on each edge prior to the heuristic

running. This has the effect of decreasing the pheromone on all travelled

paths, which encourages exploration of other paths, which in turn could lead

to better solutions.

4.2.4 Pheromone Decay

Pheromone decay happens after every iteration. It is done in the following

manner.

τij(t) = (1− p) · τij(t) + p · τ0

4.2.5 Candidate Lists

The final new aspect of the ACS is the use of candidate lists. These lists

help the heuristic run faster and allow it to scale to larger problems easily.

A candidate list is a list of cities who’s visibility are the highest out of all

other cities. When performing a tour, an ant will first exhaust the candidate

list of a specific city before moving on to other cities. This means we no

longer have to keep track of the distances between all cities, only between

cities which exist in each other’s candidate list.

16

4.2.6 ASC Pseudocode

1 for every edge (i,j)2 do3 τij(0) = τ04 CalculateDistance(i, j)5 end for6 for k = 1 to numAnts7 do8 Place ant k on a randomly chosen city9 end for

10 Let T+ be the shortest tour found and L+ be its length11 /* Main Loop */12 for t = 1 to numIterations13 do14 for k = 1 to numAnts15 do16 build Tour T k by applying numCities− 1 times

the following steps:17 if there exists at least one city j ε candidate list18 then19 choose the next city j, jεJk

i from thecities in the candidate list according to the transition rule

20 else21 choose the closest jεJk

i

22 end if23 After each transition any k applies the local pheromone update24 end for25 for k = 1 to numAnts26 do27 Compute Lk(t) of T k(t) produced by ant k28 end for29 if an improved tour is found30 then update T+ and L+

31 end if32 for every edge (i, j)εT+

33 do34 update pheromone trails35 end if36 for every edge (i, j)37 do38 decay pheromone trails39 end if40 end for41 output T+

17

5 Problem Descriptions and Algorithm

Modifications

As mentioned in previous sections, the problem we will look at in this paper

will be the travelling salesman problem. The travelling salesman problem is

a perfect problem to test new improvements of the ACO heuristic on because

the problem or any extensions to the problem can be easily understood in

terms of implementation, goals and results. In the following two subsections

the specific problems that this paper will deal with will be explained.

5.1 Problem 1: Noisy Environment

This problem deals with a situation where the environment in which the ants

live in is noisy. It is important to understand that in the real world we do not

always encounter static environments where no noise exists whatsoever, and

it is important to be able to deal with these situations when we encounter

them. In this case, every time an ant polls the system for information about

how far a certain city is from the city they are currently at they receive a

noisy response. The noise is normally distributed about the actual distance,

making it more realistic situation then a simple uniform distribution. The

standard deviation of the noise is a tunable runtime parameter. This allowed

the testing of low noise levels (a small standard deviation) or very high

levels of noise (large standard deviation). The following two subsections will

18

describe how the noise was added to the environment and then how this noise

was adapted to to allow for the heuristic to continue functioning.

5.1.1 Adding Noise

The method for inserting noise into this algorithm was done in the follow-

ing manner. It changed the original function used to calculate the distance

between two cities, which is used in the original ACS algorithm on line 4 of

the pseudocode in section 4.1.6, by taking a randomly generated Gaussian

number (which is a normally distributed random number with mean 0 and

standard deviation 1) multiplying it by the distribution defined by the user,

then adding this to the actual distance which results in a randomly gener-

ated, normally distributed number about the real distance value. The pseudo

code for the original method and the altered method is shown below.

Original code:

Distance(City1, City2)

return√

(City1.x− City2.x)2 + (City1.y − City2.y)2)

Modified code:

Distance(City1, City2)

value←√

(City1.x− City2.x)2 + (City1.y − City2.y)2)

value← value + RandomGaussian() ∗NOISE SD

return value

19

Where NOISE SD is the tunable runtime argument for the standard devi-

ation of the noise.

5.1.2 Adapting to Noise

The original heuristic only needed to calculate the distance between each

city once at the beginning of the computation and these values were stored

in an array to be accessed later, which is done on line 4 of the pseudocode.

Since these initial values can no longer be trusted to accurately describe the

distance between two cities, we must rebuild the distance array after each

iteration, and instead of just putting a the noisy value into the distance array,

we successively average these numbers to hopefully close in on the correct

value. This is done in the following way:

Array DistanceArray

Array RunningSumArray

UpdateDistances(iteration)

for i← 1 to numCities

for j ← 1 to numCities

RunningSumArray[i][j]← RunningSumArray[i][j] + Distance(i, j)

DistanceArray[i][j]← RunningSumArray[i][j]/iteration

20

Where iteration is the current tour number that the heuristic has made.

This code would be added to the original ACS code on line 40 to ensure these

values are updated once every iteration.

This algorithm basically builds a better idea of all of the distances between

each city by averaging the noisy values received when polling the system over

time, meaning after a few hundred iterations it should have a very close es-

timation of the distance. This means that before the algorithm really has a

chance to begin to solve the TSP problem, it should already have a pretty

good estimation of each of the distances.

This modification is also relatively light weight. It only requires an ad-

ditional array to store the running sums of sample distances, which is O(n)

and computationally this modification requires the recalculating of all of the

distances after each iteration, which runs in O(n2), which is no more then

each iteration originally took. These two facts make this modification an easy

to implement and inexpensive in terms or memory space and computation

time.

21

5.2 Problem 2: Dynamic Environment

In this problem we are dealing with an instance of the TSP where a randomly

chosen subset of cities are constantly moving in a randomly chosen direction.

Below is an illustrative example of a 6 city problem at two different times,

showing the movement on the cities.

Figure 2: 6 City TSP with 3 Moving Cities at Time t

22

Figure 3: 6 City TSP with 3 Moving Cities at Time t + n

To deal with the dynamic nature of this problem pheromone smoothing

was employed to act as a counter measure and to keep the heuristic func-

tioning. The pheromone smoothing simply changes the ratio of the amount

of pheromone on each edge adjacent to a specific city, while preserving the

previous relative ordering. This type of approached has been used before in

other dynamic ACO environments. One instance was on a problem where

edges suddenly dropped off of the graph [2] and another was where ”traffic

jams” occurred, causing path lengths to increase and decrease in cost over

time [5]. The dynamic nature of the environments in these two problems were

23

overcome relatively well by using simple pheromone smoothing. The method

used here will be called local smoothing, where only the edges connected to

the moving cities are changed. This should allow paths previously deemed

sub-optimal to be used again, and insures that it is possible to deviate from

paths that are currently very good in case they become sub-optimal. The

modifications done to the original ACS heuristic are described below.

5.2.1 Pheromone Lower Bound

The first modification of the ACS heuristic was the use of a lower bound

on the amount of pheromone allowed to exist on an edge. This was done

so that no edges are essentially ”lost” and never used again due to a lack

of pheromone. This was needed because the fundamental structure of the

underlying graph is always changing where good solutions can become poor

quite quickly and vice-versa. The pheromone update rule, which is used in

multiple places in the heuristic, was changed to the following:

τij(t) =

(1− p) · τij(t) + p · τ0 if ((1− p) · τij(t) + p · τ0) > τ0;

τ0 otherwise,

5.2.2 Local Pheromone Smoothing

A smoothing technique was used to change the amount of pheromone levels on

specific edges, while leaving them in the same order relative to their neighbour

edges (ie. in the case of local smoothing about city i, if τij > τik was true

24

prior to smoothing, it will remain true after smoothing.) This technique

just ensures that certain paths do not become exceedingly popular and other

paths do not become forgotten. This is needed because the the shortest

path throughout the graph can change fundamentally at any moment. The

formula [5] to smooth an edge ij is given below.

τij ← τ0 · (1 + log(τij/τ0))

Applying this formula will only cause pheromone levels close to τ0 to move

slightly closer towards τ0 and larger pheromone levels to move more close to

τ0. Local smoothing (the applying of that formula to edges leaving a particu-

lar city) will be applied to the moving cities only. This does not need to hap-

pen every iteration, as the tour lengths do not increase dramatically from one

iteration to the next. The moving cities will have local pheromone smooth-

ing applied to them after every 10 iterations to ensure that the pheromone

levels stay in check, and is inserted at the bottom of the main loop in the

pseudocode in section 4.2.6, on line 40.

25

6 Test Results

The following values were used for the tunable parameters of the ACS imple-

mentation of the ACO heuristic. They were found to yield very good results

in static ACS trials.

Variable Valueβ 8

p (evaporation rate) 0.2q0 0.9

cl (candidate list size) 5τ0 10−7

number of ants 25number of iterations 2000

Table 1: Paramaters Used in Tests

Both problems were run on the following 51-city problem [1]. The optimal

tour has a length of 426.

26

City# Location City# Location City# Location City# Location1 37,52 14 12,42 27 30,48 40 5,62 49,49 15 36,16 28 43,67 41 10,173 52,64 16 52,41 29 58,48 42 21,104 20,26 17 27,23 30 58,27 43 5,645 40,30 18 17,33 31 37,69 44 30,156 21,47 19 13,13 32 38,46 45 39,107 17,63 20 57,58 33 46,10 46 32,398 31,62 21 62,42 34 61,33 47 25,329 52,33 22 42,57 35 62,63 48 25,5510 51,21 23 16,57 36 63,69 49 48,2811 42,41 24 8,52 37 32,22 50 56,3712 31,32 25 7,38 38 45,35 51 30,4013 5 25 26 27,68 39 59,15

Table 2: 51 City Test Problem Used

6.1 Problem 1

To test how well the noise filtering modifications worked it was tested with

a wide range of amounts of noise. The standard deviation of the noise is

the tunable parameter in this case. Increasing this parameter would cause a

wider range a values further away from the actual values to be given from the

algorithm, which can be seen as a ”noisier” environment then with a lower

noise standard deviation. Of course an environment with a noise standard

deviation of zero would an environment free of noise. The following table

describes tests run on various environments spanning a wide range of noise

amounts. Each trial consists of 25 distinct tests run at a specific noise level.

The best tour and average tour length, along with the standard deviation of

27

each trial are given.

Noise Best Tour Found Average Tour Found Standard Deviation of Tours0.0 430 448.88 6.59893.0 440 448.2 4.25216.0 436 448.04 5.42949.0 437 445.68 5.489712.0 436 445.8 4.939615.0 427 443.76 6.784018.0 439 447.28 3.954921.0 429 443.92 6.770124.0 431 444.32 5.815227.0 432 445.28 6.684430.0 435 447.36 6.565933.0 430 446.44 7.787636.0 430 445.68 8.365339.0 436 451.52 7.705242.0 438 448.16 6.460245.0 430 451.32 10.3989

Table 3: Problem 1 Test Results on Low to High Noise Values

Below the results of best tour lengths, average tour lengths and the stan-

dard deviations are given in a graph to illustrate the effect that the amount

of noise has.

28

Figure 4: Best Tour Lengths Found at Different Noise Levels

29

Figure 5: Average Tour Lengths Found at Different Noise Levels

30

Figure 6: Standard Deviation of Tour Lengths Found at Different NoiseLevels

Although an environment with a noise standard deviation of 45 could be

seen as very noisy, further tests were done to examine how well the noise

cancelling modifications adapted to extremely noisy environments. The data

about these tests are given in the following table.

31

Noise Best Tour Found Average Tour Found Standard Deviation50.0 437 452.04 7.8713100.0 442 462.64 11.8216150.0 453 480.4 11.9833200.0 479 503.88 13.2524250.0 514 532.68 12.7427300.0 531 561.0 18.1769

Table 4: Problem 1 Test Results on Extremely High Noise Values

Below the results of best tour lengths, average tour lengths and the stan-

dard deviations in these extremely noisy environments are illustrated in the

following three graphs.

32

Figure 7: Best Tour Lengths Found at Extremely High Noise Levels

33

Figure 8: Average Tour Lengths Found at Extremely High Noise Levels

34

Figure 9: Standard Deviation of Tour Lengths Found at Extremely HighNoise Levels

35

6.2 Problem 2

This problem is significantly harder to test due to the fact that nothing will

be known about the state of the underlying graph after any given test. This

is due to the fact that for each test a randomly selected subset of cities are

chosen to move, and the direction they move in is also randomly chosen. The

direction can be any multiple of 45◦. The only controllable parameter was the

number of cities to move. To test how well the modified algorithms worked,

it was run for 2000 iterations, and then the underlying graph was exported

and a new test with the original ACS heuristic was done. This would give us

a basis for comparison. Below are the results for 5 tests, of 10 trials each, all

with different amounts of moving cities. Also given is the difference between

the path lengths given as a percentage of how much better of a solution the

standard ACS gave on the same graph as the dynamic ACS ended on.

36

ACO on Dynamic Environment ACO on Static Environment % Difference1893 1878 0.82410 2396 0.61950 1935 0.81906 1893 0.71699 1685 0.82063 2059 0.22223 2215 0.41416 1416 0.01836 1820 0.91871 1862 0.5

Table 5: Problem 2 Test Results for 5 Moving Cities



37





38



Below is a table giving the statistical information for all of the tests done

for problem 2, followed by graphs visually displaying these results. Solution

Quality is defined as 100% - the average difference between the dynamic ACO

solutions and the static ACO solutions.

# of Moving Cities Average Difference Standard Deviation Solution Quality5 0.57 0.29 99.4315 0.78 0.79 99.2225 0.7 0.45 99.335 0.58 0,43 99.4245 2.34 3.01 97.66

Table 10: Problem 2 Overall Test Results

39

Figure 10: Average Difference of Solution Lengths with Different Numbersof Moving Cities

40

Figure 11: Standard Deviation of Solution Lengths with Different Numbersof Moving Cities

41

Figure 12: Solution Quality with Different Numbers of Moving Cities

42

7 Discussion

7.1 Problem 1

As we can see the noise cancellation worked very well across most levels of

noise and we saw a slight increase of path length when dealing with extreme

levels of noise. This can be attributed to the fact that the modifications,

through averaging, can reach a point where the estimation of the distances

between cities is very close to the actual value. Even after as little as 25 it-

erations, the heuristic will already have a pretty good idea about the actual

distances between the cities, and this estimation will only get better over

time. This approach takes advantage of the fact that the nature of the ACO

heuristic is to build solutions over many iterations through the graph, and it

can estimate the distance between the cities well enough before the heuris-

tic begins to converge upon a solution. This implies that the pheromone

updates during the early stages of the solution building process are either

reasonably accurate or that previous updates done on incorrect estimations

can be corrected through the accurate application of pheromone on good

edges and the decay of pheromone on older, suboptimal edges. The algo-

rithm seems to breakdown slightly when running in environments of extreme

noise which lead to a slight drop in solution quality. From this we can see

that the algorithm has a harder time to successfully estimate the distances,

meaning that it must prematurely chooses a suboptimal path based on inac-

43

curate pheromone trails. This type of noise would be very hard to deal with

in any situation without a more intrusive noise cancellation scheme, and con-

sidering how good the solutions the algorithm still gives at this point, we can

conclude that it performs extremely well in a wide range of noise conditions.

There are a few other benefits of implementing the noise cancellation

in this manner. First of all, it is efficient in both computation time and

memory space. This makes the running time of both the standard ACS and

the modified ACS almost indistinguishable, which is always a desirable factor.

Secondly, this method of noise cancellation does not need to know how much

noise exists in the environment, or even if it is noisy at all. Since no prior

knowledge of the noise is needed these modifications can be implemented

in instances where noise could exist or where it is known to exist but the

intensity of the noise is not known.

7.2 Problem 2

When it came to testing this problem we could not rely on any previous

benchmarks obtained, the only way to evaluate how well the dynamic ACS

worked was to run it for a given number of iterations, in this case 2000, letting

the underlying map randomly grow. This map was then exported and the

static ACS heuristic was ran on it. The two solutions were then compared.

As it turns out, the use of pheromone smoothing around the moving cities

44

worked very well and let the heuristic adapt to the dynamic aspects of the

problem. When looking at both the average solution found and the standard

deviation of these tests, it is obvious that the solutions obtained were quite

good. In all cases very reasonable solutions were reached, especially when

there were less then 35 moving cities where the static ACS heuristic reported

solutions less then 1% better then the dynamic ACS. When more and more

cities are moving the heuristic has a harder time, but still gives very reason-

able solutions with the worst solution found was 8.7% worse then the static

ACS. Its very important to realize that the dynamic ACS not only gave good

results, but also tracked them as time went on meaning that it could always

provide a very good path at any time, while running the static ACS on a

given map at a given time will not give very relevant information about the

map when the computation is completed due to the fact the environment is

always evolving.

This heuristic performed very well for one simple reason, it allowed paths

that were previously travelled and found not to be too desirable to be revis-

ited in hopes of finding a new, better solution in the ever-changing environ-

ment. When dealing with a smaller amount of moving cities, the algorithm

correctly established the link between the moving cities and the static cities,

allowing the pheromne levels on the dynamic edges to change as their quality

in the final solution changed. Even when almost all the cities were moving,

45

the algorithm still produced very respectable results. The use of a lower

bound on the pheromone levels was also very important. Simply making the

lowest possible pheromone level on any given edge be the initial pheromone

level, which in this case was 10−7, made it impossible for any path to be com-

pletely forgotten and the pheromone smoothing ensured that paths would not

become too popular. The combination of these two rather simple modifica-

tions allowed the dynamic ACS to perform so well.

The new modifications to the dynamic ACS were also very efficient. The

modifications only moved the cities and updating the distances, and updating

the pheromone levels. Both of these operations run in O(n · c), where c is the

number of moving cities. The only added memory needs was keeping track

of the moving cities, which was only O(c). Both the running time and space

complexity of the modifications are practically insignificant with respect the

the overall running time and space complexity of the ACO heuristic in gen-

eral.

46

8 Looking Forward

Since the field of swarm intelligence has been evolving so quickly, it is always

a good idea to think of the future. Both of these modifications seem to work

very well, but there could be room for a slight improvement. This could be

obtained through the fine tuning of all of the runtime parameters. Some of

these parameters, such as pheromone evaporation rate or initial pheromone

levels, could potentially have a positive effect in either of the two problems.

Noisy environments are quite common in the real world, and as such they

effect a lot of different problems. Although the first problem seems to be

pretty self contained, these noise cancellation techniques could be tested in

conjunction with other environmental disturbances, like the dynamic envi-

ronments introduced in the second problem for instance, to try to realize

more of the ACO heuristic’s potential.

An interesting extension to the second problem would be to contain all

cities within a certain area, and if a city hit the boundary it would be redi-

rected back to the center of the map. This would take away the possibility

of ever reaching a stable state because, after a while, the current problem

reaches a state where the tour lengths keep increasing but the structure of

these tours remain constant. Another interesting extension would be to in-

vestigate different wandering paths for the cities other then just straight lines,

47

or the ability for the cities to change speed and direction.

All of these extension to the base problems discussed in this paper would

be very relevant by testing the boundaries of that the ACO heuristic can

handle.

48

References

[1] Tsplib. http://www.iwr.uni-heidelberg.de/groups/comopt/software/TSPLIB95/.

[2] Daniel Angus and Tim Hendtlass. Ant colony optimisation applied to a

dynamically changing problem. In Developments in Applied Artificial In-

telligence: Proceedings of the 15th international conference on Industrial

and engineering applications of artificial intelligence and expert systems,

IEA/AIE 2002, pages 618–627, London, UK, 2002. Springer-Verlag.

[3] Eric Bonabeau, Marco Dorigo, and Guy Theraulaz. Swarm Intelligence:

From Natural to Artificial Systems. Oxford, New York, USA, 1999.

[4] Marco Dorigo and Thoms Stutzle. Ant Colony Optimization. MIT Press,

Cambridge, Massachusetts, USA, 2004.

[5] Casper Joost Eyckelhof and Marko Snoek. Ant systems for a dynamic

tsp. In Ant Algorithms: Third International Workshop, ANTS 2002,

pages 88–99, London, UK, 2002. Springer-Verlag.

[6] Xavier Gandibleux, Xavier Delorme, and Vincent T’Kindt. An ant colony

optimisation algorithm for the set packing problem. In Ant Colony Opti-

mization and Swarm Intelligence: Fourth International Workshop, ANTS

2004, pages 49–60, Berlin, Heidelberg, Germany, 2004. Springer-Verlag.

[7] Walter J. Gutjahr. S-aco: An ant-based approach to combinatorial op-

timization under uncertainty. In Ant Colony Optimization and Swarm

49

Intelligence: Fourth International Workshop, ANTS 2004, pages 238–

249, Berlin, Heidelberg, Germany, 2004. Springer-Verlag.

[8] Joaquin and Jordi Pereira. Ant algorithms for assembly line balancing.

In Ant Algorithms: Third International Workshop, ANTS 2002, pages

65–75, London, UK, 2002. Springer-Verlag.

[9] Ying Zhang, Lukas D. Kuhn, and Markus P.J. Fromherz. Improvements

on ant routing for sensor networks. In Ant Colony Optimization and

Swarm Intelligence: Fourth International Workshop, ANTS 2004, pages

154–165, Berlin, Heidelberg, Germany, 2004. Springer-Verlag.

50

an investigation into the suitability of the ant colony

Documents