bio-inspired optimization. our journey – for the remainder of the course a brief review of...
TRANSCRIPT
Bio-Inspired Optimization
Our Journey – For the remainder of the courseA brief review of classical optimization methods
The basics of several stochastic optimization methods evolutionary algorithms (EA) artificial neural networks (ANN) particle swarm optimization (PSO) ant colony optimization (ACO)
The basic biology and physics behind the methods
How to implement and apply the methods
http://www.youtube.com/watch?v=A042J0IDQK4
Optimization:Highest achievable performance by maximizing desired factors
and minimizing undesired onesMaximization: achieving the highest result regardless of cost
or expense $$ Quality of life Etc.
OptimizationIn general, optimization is the
problem of finding the (global) minimum (or maximum) of an objective function
Sometimes (but not always!) the objective function is a specific, well-defined mathematical function
F = f(x1 ,x2, x3 ,...)
Optimization methodsOptimization methods can be divided into two broad
categories Classical (deterministic) optimization methods Stochastic optimization methods
Classical optimization (Covered in ME517) Gradient descent (following the steepest slope) Newton’s method Penalty methods Lagrange multiplier methods
Optimization methodsClassical optimization (Covered in ME517)
Gradient descent (following the steepest slope)http://www.youtube.com/watch?v=HvLJUsEc6dw
Newton’s methodPenalty methodsLagrange multiplier methods
Limitations of classical optimizationClassical methods are less useful in cases with
non-differentiable objective functionsobjective functions whose values can only be obtained as a
result of a (lengthy) simulationVarying number of variables (as in optimization of neural
networks)For such problems, stochastic optimization methods
are more suitable.
Stochastic optimization methodsStochastic optimization algorithms can be used for solving many
problems where classical methods are insufficient More frequently used in industry, particularly in large and complex
problems The number of application areas is steadily increasing
As the name implies, stochastic optimization methods contain an element of stochasticity (randomness)
Many stochastic optimization methods are inspired from biological phenomena
An important subset of stochastic optimization methods are biologically inspired optimization methods.
Biologically inspired optimizationDiscussion
Why would someone use inspiration from biology when defining an optimization method?
What are good examples?
Biologically inspired optimization characteristicsIn comparison with other optimization algorithms, bio-Inspired
optimization algorithms have the following characteristics:The individual components are distributed and autonomous,
there is no central control, and the fault of an individual cannot influence solving the whole problem, these characteristics ensure this kind of algorithms is more robust Is the individual intelligent?
Biologically inspired optimization characteristics
They don’t demand to meet the requirement of differentiability, convexity, and other conditions for mathematical description of the problem This is because of the random search
Use basic mathematical operations, therefore, they are simple and easy to be implemented computationally
These advantages enabled bio-mimetic optimization algorithms to be widely used in a very short period, such as power system, vehicle routing, mechanical design, robotics, etc.
Biologically inspired optimizationNature is all about adaptation, which can be seen as a kind of
optimizationFinding the ideal solution for a specific case (types of eyes)
However, note that in nature the target is constantly moving – unlike the case in engineering problemsMoving surface
Example 1: Shark skinA good example of adaptation (biological optimization)The skin of sharks has evolved to allow low friction very fast
swimming.
Example 1: Shark skinThe rib-like structures affect the interaction between the
surface of the shark and the surrounding water, essentially reducing drag
Prime example of biological inspiration for engineering concepts: Similar ideas are being applied in order to reduce drag (and noise) of aircraft and other vehicles
Shark skin has also inspired the development of the fabric used in swimsuits worn by athletes
Example 2: Evolution of the eyeThe evolution of eyes is another interesting exampleFaced with the problem of generating light-gathering devices,
evolution has come up with no less than 40 completely independent solutions to the problem
Example 3: SwarmingSome species (particularly ants, bees, and termites) display
very advanced forms of cooperationSpecific example: Weaver ants (Oecophylla)
Example 4: SwarmingMany organisms (e.g. some bird and fish species) display
swarming behavior for protection against predators, efficient food gathering, etc...
Biologically inspired optimizationSeveral different optimization methods have been developed,
based on biological phenomena:Evolutionary algorithms (inspired by evolution) Artificial neural networks (inspired by the brain)Ant colony optimization (inspired by cooperative behavior) Particle swarm optimization (inspired by swarming)
EA
Brief introduction to EAsEvolutionary algorithms = EAs.
Based on darwin evolution, which is a process involving gradual, hereditary changes of biological organisms, over long periods of time
Basic biological concepts: Population, fitness, selection, mutation
Individuals that are well adapted to their environment have a high probability of generating offspring
Brief introduction to EAsRandom changes to the
genotype (mutations), sometimes leading to large changes of the phenotype.
Mutations provide new material for evolution to work with
Mutations are random, however selection is not
Basic mode of operation of EAs In an EA, a population of candidate solutions (to the problem at hand) is formed (random
initialization)
Each individual in the population is evaluated and assigned a fitness score based on its performance.
New individuals are formed through the processes of selection, crossover, and mutation.
The new individuals form the second generation, which is evaluated in the same way as the first generation etc.
The process is repeated until a satisfactory solution has been found to the problem at hand.
Evolutionary Algorithms Encoding: the variables of the problem are encoded in strings of digits
known as chromosomes.
The exact encoding procedure varies from problem to problem – in the simple case of optimization of a function f(x1,x2,x3,...xn) the variables xi, i = 1,...,n should be encoded. In more complex problems (e.g. optimization of artificial brains for robots or optimization of neural networks), the specification of the encoding procedure can be much more complicated.
Evolutionary Algorithms Random initialization of a population containing N chromosomes
Evolutionary AlgorithmsCrossover
Mutation
Evolutionary Algorithms
Evolutionary Algorithms
Introduction to Genetic Algorithms http://www.youtube.com/watch?v=zwYV11a__HQ
Optimization Project – More details on wiki 1. Describe the problem.
Identify the specific performance characteristics that are being min/max-imized against each other.
Contrast classical and stochastic optimization, and identify which is more appropriate for your optimization problem.
2. Identify a suitable stochastic optimization algorithm, and explain why you chose it. (You may want to do some outside research.)
3. Solve the optimization problem using a programming language
The fun begins: Boxcar2d.com