Particle Swarm Optimization

• Swarm Intelligence.

• Particle Swarm Optimization

• The procedure for implementation of PSO.

• modifications in the basic structure of PSO.

• PSO features.

• Parameter selection in PSO.

Agenda

• The term swarm is used to represent an aggregation of the animals or insects which work collectively to accomplish their day to day tasks in an intelligent and efficient manner.

• Swarm may be school of fishes, flocks of birds and social insects like ants, bees etc.

Swarm Intelligence

• Swarms are capable of performing tasks which are complex enough to be performed by an individual animal or insect.

example of Ants

Ants are capable of finding the shortest path for their food and they do it by communicating through their environment using a chemical substance called pheromones as they can’t communicate directly.

• The PSO algorithm is a population-based optimization technique. In a PSO system, multiple candidate solutions coexist and collaborate simultaneously.

• Each solution called a particle, flies in the problem search space looking for the optimal position to land.

• A particle, during the generations, adjusts its position according to its own experience as well as the experience of neighboring particles.

Particle swarm optimization

• A particle status on the search space is characterized by two factors:– its position (Xi) – and velocity(Vi).

And will be updated according to the following equations:

• Eq. (1)• first part , Vi [k], represents particle’s

previous velocity. • It stores previous flight direction .• to prevent the particle from drastically

changing its direction and alignment it to its current direction.

• second part, C1 Rand (.)(pbesti [k]-Xi[k]), • is called cognition part and resembles

individual memoryof the position that was best for the particle.

• its effect is to linearly attract the particle towards its own best experience .

• third part, C2rand (.) (gbest [k] -Xi [k]), is called social or cooperation component• resembles a group standard which individuals

seek to attain.• The effect of this term is to attract the particle

towards the best experience of the all particles in the swarm scaled by random weight .

a. Initialize each particle with a random velocity and position.

b. Calculate the objective value of all particles .

c. Set the position and objective of each particle as Pi and Pbest ,respectively.

d. Set the position and objective of the particle with the best fitness (least objective) as Pg and gbest, respectively.

The procedure for implementation of PSO

e. Update Particles’ Vi and Xi according to Equations (1) and (2).

f. Update each particle’s Pbest and Pi, that is, if the current Pbest of the particle is better(less) than its Pbest, then Pbest and Pi are replaced with current objective value and position vector, respectively.

g. Update Pg and gbest, that is,

if the current gbest of the whole swarm is fitter than gbest, then gbest and Pg are replaced with current best objective and its corresponding position vector, respectively.

h. Steps e –g are repeated until stopping criterion (usually a prespecified number of iterations or a quality threshold for objective value) is reached.

1. Selection of maximum velocity: the velocities may become too high so that the

particles become uncontrolled and exceed search space. Therefore, velocities are bounded to a maximum value Vmax, that is

important modifications in the basic structure of PSO

• Large value of Vmax increases the convergence speed and convergence to a local minimum .

• small value of Vmax decreases the efficiency of the algorithm.

Notes,

2. Adding inertia weight:a new parameter w for the PSO, named inertiaWeight is added in order to better control the scope

of the search .So, Eq. (1) is now becomes:

• high value large velocities find the global

optimum neighborhood in a fast way.

• low value narrow the particles’ search region.

Notes,

3. Constriction factor:

if running algorithm without restraining the velocity, so the system will explodes after a few iterations .

so, induce a constriction coefficient in

order to control the convergence properties.

• with the constriction factor , the PSO equation for computing the velocity is:

• Note that ,– if C = 5 then = 0.38 from Eq. (4), will cause a

very pronounced damping effect.– But , if C is set to 4 then is thus 0.729, which

works fine.

• The advantage of using this factor is that– there is no need to use Vmax nor to guess the

values for any parameters which govern the convergence and prevent explosion.

Notes,

• has less parameters to be tuned by user,compared with other heuristics.

• Simple and coding is very easy.

• provides fast convergence.

• provides high accuracy.

• less affected by initial solutions,compared with other heuristics.

• ...........,other features.

PSO features

1. Using PSO variants with fixed parameters,so parameter values (accepted values , set on trial-and-error basis).

2. Using PSO variants with dynamic or adaptive parameters.

3. Using parameter-free PSO variants which do not require any parameter setting process.

4. Using composite PSO but,it have a big problem that it will increase complexity of the problem.

Strategies for setting PSO parameters

• Swarm size

– Too few particles the algorithm will trap in local optima.

– too many particles slow down the algorithm.

– So no exact rule for selection of swarm size, But when the dimension of problem increases, the swarm size should also be increased.

Parameter selection in PSO

• Maximum velocity– If it is too high , the particles move erratically and

are swiftly attracted to global best (gbest) without enough exploration of search space and may exceed search space.

– if it is too small, the computational overhead increases and the algorithm may be unable to converge.

– Value for Vmax

– NI is the number of intervals in the dth dimension.– Xmax and Xmin are the maximum and minimum values

of particles .

• Inertia weight– There are different procedures for setting inertia

weight:– fixed inertia weight.– Linearly decreasing.– linearly increasing.– non-linear ,fuzzy adaptive, random....etc.

– Taking into account both simplicity and efficiency, linearly decreasing inertia weight is the most appropriate method for setting inertia weight.

Accepted values of w

• Acceleration coefficients(C1,C2)– if too high the particles move abruptly and

fall in false optima.– if too low the particles move too slowly and

the algorithm couldn’t converge.– If C1 increases it could increase attraction to

Pbest and decrease attraction to Gbest.– If C2 increases it could increase attraction to

Gbest and decrease attraction to Pbest.– So, setting C1 ,C2 =2 for most of problems.

Any Questions ??