design & analysis of algorithms combinatory optimization school of computing pasi fränti...
TRANSCRIPT
![Page 1: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/1.jpg)
Design & Analysis of Algorithms
Combinatory optimization
SCHOOL OF COMPUTING
Pasi Fränti20.10.2014
![Page 2: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/2.jpg)
• Local search• Stochastic variations of local search• Genetic algorithms• Swarm Intelligence
Optimization techniques
![Page 3: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/3.jpg)
Here decision tree picture!!!
Optimization techniques in context
![Page 4: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/4.jpg)
Local search
![Page 5: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/5.jpg)
Select one and move
Main principle of local search
![Page 6: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/6.jpg)
LocalSearch:
Generate initial solution. REPEAT
Generate a set of new solutions. Evaluate the new solutions. Select the best solution.
UNTIL stopping criterion met.
Structure of local search
![Page 7: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/7.jpg)
• Representation of solution• Neighborhood function• Search strategy
Components of local search
![Page 8: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/8.jpg)
Study neighbor solutions
Movement in neighborhood
![Page 9: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/9.jpg)
Accept only better solutions
Hill climbing
![Page 10: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/10.jpg)
Local and global maxima
![Page 11: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/11.jpg)
Combining local search and hill-climbing
![Page 12: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/12.jpg)
• Represent solution as bit string (x1x2,…xn), where xi {0,1}.
• Problem instance: wi= (2,3,5,7,11), S=15.
• Solution with elements 2,3 and 7 is represented as 11010.
Local search for knapsack
![Page 13: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/13.jpg)
0 1 0 1 0
1 0 0 1 0
1 1 1 1 0
1 1 0 0 0
1 1 0 1 1
1 1 0 1 0
12
10
9
5
17
23
Single bit change: 01 or 10
S=15 W=[2, 3, 5, 7, 11]Move in knapsack
![Page 14: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/14.jpg)
1 1 0 1 012
0 11 1 015
0 1 0 11 21
1 0 11 0
14
11 1 1 0 17 11 0 11 23
1 0 1 0 1
1811 1 0 010
11 0 0 1
16
Two operations:
• 01 or 10
• Swap bit location
Extended neighborhoodS=15 W=[2, 3, 5, 7, 11]
10011
![Page 15: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/15.jpg)
1 0 0 0 113 1 1 0 0 0 5
1 0 1 0 0 7
0 1 0 0 114
0 0 1 0 1 16
100109
0 0 0 1 1 18
Getting stuck into local maximumS=15 W=[2, 3, 5, 7, 11]
![Page 16: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/16.jpg)
0 1 0 0 114 11 0 0 0 5
0 11 0 0 8
1 0 0 0 113
0 0 1 0 1 16
0 1 0 1 010
0 0 0 11 18
Prevents search to return previously visited solutions
Select the next best
Tabu!
S=15 W=[2, 3, 5, 7, 11]Tabu search
![Page 17: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/17.jpg)
0 1 0 1 010
0 1 0 1121
11 0 1 0 12
0 11 1 0 15
Tabu search (2nd iteration)S=15 W=[2, 3, 5, 7, 11]
![Page 18: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/18.jpg)
Traveling salesman problem
... pi-1 pi pi+1 ...
... pi-1 pi+1 pi ...
... pi pi-1 pi+1 ...
... pi pi+1 pi-1 ...
... pi+1 pi pi-1 ...
... pi+1 pi-1 pi ...
Permute local changes in given route
![Page 19: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/19.jpg)
Local search algorithm for TSP
LocalSearchTSP(G:graph): solution; S {1,2,...,N}. FOR i:=1 TO N DO
Swap( S[i], S[random(1,N)] ).
REPEAT i Random(2..N-1). {S1,S2,...,S6 } PermutateTriple(i-1 i, I+1). S SelectBest(S1,S2,...,S6).
UNTIL no improvement.
![Page 20: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/20.jpg)
TSP example
E F G H A 4 + + 2 + = 2 + 6
E F H G A 4 + 3 + 2 + 2 = 11 min!
E G F H A + + 3 + = 3 + 3
E G H F A + 2 + 3 + = 2 + 5
E H G F A 3 + 2 + + = 2 + 5
E H F G A 3 + 3 + + 2 = 1 + 6
A
B C
D E F
G H
22
24 4
3 32
23
43 5
![Page 21: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/21.jpg)
Genetic algorithm
![Page 22: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/22.jpg)
Genetic algorithm (GA)
Needs more material!
![Page 23: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/23.jpg)
Generate a set of initial solutions.
REPEAT Generate new solutions by crossover. Mutate the new solutions (optional). Evaluate the candidate solutions. Retain best candidates and delete the rest.
UNTIL stopping criterion met.
Main structure of GA
![Page 24: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/24.jpg)
Permuting pairs for crossover
Select next pair(i, j): REPEAT
IF (i+j) MOD 2 = 0 THEN i max(1, i-1); j j+1; ELSE j max(1, j-1); i i+1;
UNTIL ij. RETURN(i, j)
Elitist approach using zigzag scanning among the best
solutions
![Page 25: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/25.jpg)
Optimizing chess
playing
Revise
![Page 26: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/26.jpg)
Tic-tac-toe example
![Page 27: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/27.jpg)
Evaluation function for tic-tac-toe
![Page 28: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/28.jpg)
Minmax example
Redraw
![Page 29: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/29.jpg)
Minmax playing: Min’s move
![Page 30: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/30.jpg)
Minimax maximizes the worst-case outcome for max
Minmax playing: Max’s move
![Page 31: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/31.jpg)
Chess Game tree
![Page 32: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/32.jpg)
Beyond the horizon
![Page 33: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/33.jpg)
Evaluating Chess position
![Page 34: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/34.jpg)
Positional factors
![Page 35: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/35.jpg)
0 queen [800-1000] 1 rook [440-540] 2 bishop [300-370] 3 knight [290-360] 4 pawn [85-115] 5 bishop pair (+) [0-40] 6 castling done (+) [0-40] 7 castling missed (-) [0-50] 8 rook on an open file (+) [0-30] 9 rook on a semi-open file (+) [0-30] 10
connected rooks (+) [0-20]
11
rook(s) on the 7th line (+) [0-30]
12
(supported) knight outpost (+)
[0-40]
13
(supported) bishop outpost (+)
[0-30]
14
knights’ mobility >5 (>6) (+)
[0-30]
Initial value range
![Page 36: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/36.jpg)
290
300
310
320
330
340
350
360
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39
generation
aver
age
gene
ratio
n va
lue
Bishop
Knight
Result of optimization
![Page 37: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/37.jpg)
Swarm intelligence
![Page 38: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/38.jpg)
• Social intelligence: individual behavior maybe naive but joint effect can be intelligent.
• Decentralized: no central control of the individuals of the colony
• Self-organized: individual adapts to environment and other members of colony
• Robust: Task is completed even if some individuals fail
Swarm intelligence (SI)
![Page 39: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/39.jpg)
Ant colony optimization (ACO)
Main principle:
• Emitting pheromone between nest and food• Joint efforts to carry loads
Solving TSP by ants:
• Sending ants to make randomized tours• Short links chosen more often than long ones• Good tracks are marked by pheromone• Tracks with high pheromone chosen more often
![Page 40: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/40.jpg)
Ant colony optimization
Initialization:
• Generate randomized tours.
• Smaller links chosen more often than longer ones
Simulate pheromone:
• Subtract cost of the links in best solution (-1)
• Increase the ones in the worst solution (+1)
• Tours are evaluated using the original graph.
![Page 41: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/41.jpg)
A F
C
D
H
B
G
E
A F
C
D
H
B
G
EA F
C
D
H
B
G
E
A F
C
D
H
B
G
E2 1 2 3
2 4 2 6
A F
C
D
H
B
G
E
0
-1
+ 1
0
-1
+ 1
000
-1+
1
+1
-1
A F
C
D
H
B
G
E
2
2
2
2
2
3
453
34 4
3
In p u t grap h :
A d d it io n s m ad e:
Ant colony optimization (1st round)
![Page 42: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/42.jpg)
A F
C
D
H
B
G
E
0
0
+ 1
0
-1
+ 1
000
-10 +1
-1
A F
C
D
H
B
G
E
2
1
3
2
1
4453
25 5
2
A F
C
D
H
B
G
E A F
C
D
H
B
G
E
2 1 2 2
A F
C
D
H
B
G
E
2 3A F
C
D
H
B
G
E
2 2
M o d ified grap h :
N ew ad d it io n s :
Ant colony optimization (2nd round)
![Page 43: Design & Analysis of Algorithms Combinatory optimization SCHOOL OF COMPUTING Pasi Fränti 20.10.2014](https://reader036.vdocument.in/reader036/viewer/2022062321/56649ed35503460f94be335e/html5/thumbnails/43.jpg)