single-solution based metaheuristics. outline local search simulated annealing tabu search …
Post on 14-Dec-2015
246 Views
Preview:
TRANSCRIPT
Single-solution based metaheuristics
Outline
Local Search Simulated annealing Tabu search …
3
local search
LS()
1 choose an initial solution X randomly
2 while the stop criterion is not yet satisfied do3 choose a neighbor X'∈N(X)
4 if f(X')<f(X) then X ← X'
5 return X
Neighbourhood
For each solution X S, N(X) S is a neighborhoodIn some sense each X' N(X) is in some sense “close” to S
Defined in terms of some operation Very like the “action” in search Exchange two elements Change the value of elements
Local search Elements of Local Search
Representation of the solution Evaluation function; Neighbourhood function: to define solutions which can be
considered close to a given solution. Neighbourhood search strategy: random and systematic
search; Acceptance criterion: first improvement, best
improvement, best of non-improving solutions, random criteria;
Simulated annealing
Combinatorial search technique inspired by the physical process of annealing
A stochastic local search algorithm Simulated annealing is an approach for solving all
sorts of optimization problems: Say you have a really huge search space. You want to find the global optimum for some function in
that space.
Simulated annealing
Basic ideas: like hill-climbing identify the quality of the local
improvements instead of picking the best move, pick one randomly say the change in objective function is Δf if Δf is positive, then move to that state otherwise:
move to this state with probability proportional to Δf thus: worse moves (very large negative Δf ) are executed less often
8
SA()
1 choose an initial solution X0 randomly
2 give an initial temperature T0 , X ← X0, T ← T0 3 while the stop criterion is not yet satisfied do4 for i ← 1 to L do5 pick a solution X'∈N(X) randomly6 Δf ← f(X')-f(X)7 if Δf<0 then X ← X'8 else X ← X' with probability exp(- Δf/T)9 T← g(T) //generally, T ← aT10 return X
Generic choices for annealing schedule
initial temperature T0
(example: based on statistics of evaluation function) Cooling schedule-how to change temperature over time
(example: geometric cooling, T ← aT) L: number of iterations at each temperature
(example :multiple of the neighborhood size Stopping criterion
(example: no improved solution found for a number of temperature values)
Simulated Annealing in Practice
method proposed in 1983 by IBM researchers for solving VLSI layout problems (Kirkpatrick et al, Science, 220:671-680, 1983). theoretically will always find the global optimum (the best
solution) useful for some problems, but can be very slow
slowness comes about because T must be decreased very gradually to retain optimality
In practice how do we decide the rate at which to decrease T? (this is a practical problem with this method)
11
Tabu search
Tabu Search (TS) is a metaheuristic which is concerned with imposing restrictions to guide a search process. These restrictions operate in several forms both by direct exclusion of search alternatives classed as tabu and by modifying evaluations and probabilities of selection of such alternatives
Tabu search begins in the same way as ordinary local or neighborhood search, proceeding iteratively from one solution to another until a chosen termination criterion
is satisfied. Each X∈S has an associated neighborhood
N(X)S, and each solution X' ∈N(X) is reached from X by an operation called a move.
12
Tabu search
TS( )
1 choose an initial solution X
2 X* ← X (record the best solution found so far)
3 while the stop criterion is not yet satisfied do
4 N*(X) ←{X' ∈N(X) |the move from X to X' is not tabu OR X' satisfies the aspiration criterion}
5 choose the best neighbor X' ∈N(X)
6 X ← X' //even if f(X') > f(X)
7 if f(X) < f(X*) then X* ← X
8 return X*
Elements of Tabu Search
Neighborhood structure stop criterion Tabu List(short term memory) Tabu tenure Aspiration criteria Long term memory attributes
Elements of TS: Recency
Memory related - recency (How recent the solution has been reached) Tabu List (short term memory): to record a limited
number of attributes of solutions (moves, selections, assignments, etc) to be discouraged in order to prevent revisiting a visited solution;
Tabu tenure (length of tabu list): number of iterations a tabu move is considered to remain tabu;
Elements of Tabu Search
Memory related – recency (How recent the solution has been reached) Tabu tenure
List of moves does not grow forever – restrict the search too much
Restrict the size of list FIFO Other ways: dynamic
Elements of Tabu Search
Long term memory: to record attributes of elite solutions to be used in: Intensification: giving priority to attributes of
a set of elite solutions (usually in weighted probability manner)
Diversification: Discouraging attributes of elite solutions in selection functions in order to diversify the search to other areas of solution space;
trade-off
Memory related: frequency observe frequency of selected attributes Penalization of moves
Making use of frequency can modify the evaluation function so that the attributes with less frequency are given the bigger evaluation value
Elements of TS: Aspiration
If a move is good, but it’s tabu-ed, do we still reject it? Aspiration criteria: accepting an improving solution
even if generated by a tabu move Similar to SA in always accepting improving
solutions, but accepting non-improving ones when there is no improving solution in the neighbourhood;
Adaptive Tabu Search
Tabu tenure denotes an attribute is tabu in recent t iteration
If t (tenure) to small, we will return to the same local min
Adaptively modify t If we see the same local min, increase t When we see evidence that local min escaped (e.g. improved
sol), lower t
Path RelinkingBasic idea: Given 2 good solutions, perhaps a better solution lies
somewhere in-between Try to combine “good features” from two solutions Gradually convert one solution to the other
Applications
Simulated annealing for optimization problem 3-CNF SAT, Strip packing, Vehicle Routing, …
Tabu search for optimization problem 3-CNF SAT, Strip packing, Vehicle Routing, …
22
Homework
Experiments(1 or 2):
1. Implement TS or SA for strip packing problem
2. Implement TS or SA for 3-CNF SAT problem Download the following paper to read Qisen Cai, Defu Zhang, Wei Zheng, Stephen C.H. Leung. A new
fuzzy time series forecasting model combined with ant colony optimization and auto-regression. Knowledge-Based Systems. 74(1) (2015) 61–68.
top related