Download - Simulated Annealing
![Page 1: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/1.jpg)
Term Paper on-Global Optimization Search
Techniques (A I – Soft computing Domain)
PRESENTED BY -JOY DUTTA
ROLL – 91/CSE/101006BTECH 6 TH SEM
COMPUTER SCIENCE & ENGINEERINGCALCUTTA UNIVERSITY
04/12/2023 1
![Page 2: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/2.jpg)
MAIN PROBLEM -> OPTIMIZATION
Local
GlobalOptimiza
tion search
techniques
04/12/2023 2
TABU SEARCH , GREEDY APPROACH , STEEPEST DESCEND,
ETC
SIMMULATED ANNEALING,PARTICLE SWARM OPTIMIZATION (PSO),GRADIENT DESCENT ETC
![Page 3: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/3.jpg)
Difficulty in Searching Global Optima
04/12/2023 3
startingpoint
descenddirection
local minima
global minima
barrier to local search
![Page 4: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/4.jpg)
Simulated Annealing(SA)• SA is a global optimization technique.• SA distinguishes between different local optima.SA is a memory less algorithm, the algorithm
does not use any information gathered during the search
SA is motivated by an analogy to annealing in solids.
Simulated Annealing – an iterative improvement algorithm.
04/12/2023 4
![Page 5: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/5.jpg)
Consequences of the Occasional Ascents
04/12/2023 5
Help escaping the local optima.
desired effect
Might pass global optima after reaching it
adverse effect
![Page 6: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/6.jpg)
Background: AnnealingSimulated annealing is so named because of its analogy to
the process of physical annealing with solids,.A crystalline solid is heated and then allowed to cool very
slowly until it achieves its most regular possible crystal lattice
configuration (i.e., its minimum lattice energy state), and thus is free of crystal defects.
If the cooling schedule is sufficiently slow, the final configuration results in a solid with such superior structural integrity.
Simulated annealing establishes the connection between this type of thermodynamic behaviour and the search for global minima for a discrete optimization problem.
04/12/2023 6
![Page 7: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/7.jpg)
Background (cont..)Solid is heated to melting point
High-energy, high-entropy stateRemoves defects/irregularities
Temp is very slowly reducedRecrystallization occurs (regular structure)New internal state of diffused atomsFast cooling induces fragile structure
04/12/2023 7
![Page 8: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/8.jpg)
Control of Annealing Process
04/12/2023 8
Acceptance of a search step (Metropolis Criterion):
Assume the performance change in the search direction is .
Accept a ascending step only if it pass a random test,
Always accept a descending step, i.e. 0
1,0exp randomT
![Page 9: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/9.jpg)
Relationship between Physical Annealing and Simulated Annealing
Thermodynamic Simulation
Combinatorial Optimization
System states
solutions
Energy Cost
Change of State Neighbouring Solutions
Temperature Control Parameter T
Frozen State Heuristic Solution
04/12/2023 9
![Page 10: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/10.jpg)
Stopping Criterion • A given minimum value of the temperature
has been reached.
• A certain number of iterations (or temperatures) has passed without acceptance of a new solution.
• A specified number of total iterations has been executed
04/12/2023 10
![Page 11: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/11.jpg)
11
Flow Chart:Start With an initial solution
Add new random stand at random period
Improvement?
Accept new Solution
Stop criteria?
Stop
P(delta)>rand?
yesyes
yes
no
no
noP(delta) 1 when c is very high.P(delta) 0 when c is very smallrand (0,1)
![Page 12: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/12.jpg)
Simulated Annealing Algorithm
• Initial temperature (TI)• Temperature length (TL) : number of
iterations at a given temperature• cooling ratio (function f): rate at which
temperature is reduced . f(T) = aT , where a is a constant, 0.8 ≤ a ≤ 0.99 (most often closer to 0.99)stopping criterion
04/12/2023 12
![Page 13: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/13.jpg)
Simulated Annealing Algorithm construct initial solution x0; xnow = x0 set initial temperature T = TI repeat for i = 1 to TL do generate randomly a neighbouring solution x′ ∈
N(xnow) compute change of cost ΔC = C(x′) - C(xnow) if ΔC ≤ 0 then xnow = x′ (accept new state) else Generate q = random(0,1) if q < exp(-ΔC /T) then xnow = x′ end if end if end for set new temperature T = f(T) until stopping criterion return solution corresponding to the minimum cost
function04/12/2023 13
![Page 14: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/14.jpg)
Convergence of simulated annealing
HILL CLIMBING
HILL CLIMBING
HILL CLIMBINGCO
ST
FU
NC
TIO
N,
C
NUMBER OF ITERATIONS
AT INIT_TEMP
AT FINAL_TEMP
Move accepted withprobability= e-(^C/temp)
Unconditional Acceptance
04/12/2023 14
![Page 15: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/15.jpg)
Implementation of Simulated Annealing
04/12/2023 15
Understand the result:
• This is a stochastic algorithm. The outcome may be different at different trials.
• Convergence to global optima can only be realized in asymptotic sense.
![Page 16: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/16.jpg)
Qualitative AnalysisRandomized local search.
Is simulated annealing greedy?Controlled greed.
Is a greedy algorithm better? Where is the difference?Explain with - The ball-on-terrain example.
04/12/2023 16
![Page 17: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/17.jpg)
Ball on terrain example – Simulated Annealing vs Greedy Algorithms
• The ball is initially placed at a random position on the terrain. From the current position, the ball should be fired such that it can only move one step left or right. What algorithm should we follow for the ball to finally settle at the lowest point on the terrain?
04/12/2023 17
![Page 18: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/18.jpg)
Ball on terrain example – SA vs. Greedy Algorithms
Greedy Algorithmgets stuck here!Locally Optimum
Solution.
Simulated Annealing exploresmore. Chooses this move with asmall probability (Hill Climbing)
Upon a large no. of iterations,SA converges to this solution.
Initial positionof the ball
04/12/2023 18
![Page 19: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/19.jpg)
Jigsaw puzzles – Intuitive usage of Simulated Annealing
• Given a jigsaw puzzle such that one has to obtain the final shape using all pieces together.
• Starting with a random configuration, the human brain unconditionally chooses certain moves that tend to the solution.
• However, certain moves that may or may not lead to the solution are accepted or rejected with a certain small probability.
• The final shape is obtained as a result of a large number of iterations.
04/12/2023 19
![Page 20: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/20.jpg)
ApplicationsCircuit partitioning and placement. Hardware/Software PartitioningGraph partitioningVLSI: Placement, routing.Image processingStrategy scheduling for capital products with
complex product structure.Umpire scheduling in US Open Tennis
tournament! Event-based learning situations. etc
04/12/2023 20
![Page 21: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/21.jpg)
04/12/2023 21
Advantages:• can deal with arbitrary systems and cost functions• statistically guarantees finding an optimal solution• is relatively easy to code, even for complex
problems.• generally gives a ``good'' solution
This makes annealing an attractive option for optimization problems where heuristic (specialized or problem specific) methods are not available.
![Page 22: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/22.jpg)
04/12/2023 22
• Repeatedly annealing with a 1/log k schedule is very slow,
especially if the cost function is expensive to compute.
• For problems where the energy landscape is smooth, or there are
few local minima, SA is overkill - simpler, faster methods (e.g., gradient descent) will work better. But generally don't know what the energy landscape is for a particular problem.
• Heuristic methods, which are problem-specific or take advantage of
extra information about the system, will often be better than general methods, although SA is often comparable to heuristics.
• The method cannot tell whether it has found an optimal solution.
Some other complimentary method (e.g. branch and bound) is required to do this.
Disadvantages
![Page 23: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/23.jpg)
ConclusionsSimulated Annealing algorithms are usually
better than greedy algorithms, when it comes to problems that have numerous locally optimum solutions.
Simulated Annealing is not the best solution to circuit partitioning or placement. Network flow approach to solving these problems functions much faster.
Simulated Annealing guarantees a convergence upon running sufficiently large number of iterations.
04/12/2023 23
![Page 24: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/24.jpg)
Reference:
04/12/2023 24
• P.J.M. van Laarhoven, E.H.L. Aarts, Simulated Annealing: Theory and Applications, Kluwer Academic Publisher, 1987.
• A. A. Zhigljavsky, Theory of Global Random Search, Kluwer Academic Publishers, 1991.
![Page 25: Simulated Annealing](https://reader033.vdocument.in/reader033/viewer/2022061420/556183afd8b42aac268b5779/html5/thumbnails/25.jpg)
Thank You!