combining metaheuristics with ilp solvers in combinatorial ...€¦ · combining metaheuristics...
TRANSCRIPT
![Page 1: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/1.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Combining Metaheuristics with ILPSolvers in Combinatorial Optimization
Christian Blum
University Of The Basque Country
Ikerbasque, Basque Foundation For Science
![Page 2: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/2.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Preliminaries: Preparing the Grounds
...
...
![Page 3: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/3.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Motivation and Outline (1)
Motivation:
▶ In the field of metaheuristics we have rules of thumb :
1. If, for your problem, there is a good greedy heuristic
apply GRASP or Iterated Greedy
2. If, for your problem, there is an efficient neighborhood
apply Iterated Local Search or Tabu Search
▶ In contrast, for hybrid metaheuristics nothing like that is known
⋆ We only have very few generally applicable techniques
⋆ We do not really know for which type of problem they work well
![Page 4: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/4.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Motivation and Outline (2)
Outline:
▶ Short Intro: Hybrid metaheuristics
▶ How to combine metaheuristics with ILP solvers?
⋆ Standard method: Large neighborhood search (LNS)
▶ Hypothesis about the conditions in which LNS does NOT work
▶ What can we use instead of LNS?
⋆ New hybrid: Construct, Merge, Solve & Adapt (CMSA)
![Page 5: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/5.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Hybrid Metaheuristics
Hybrid Metaheuristics
Short introduction
![Page 6: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/6.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Hybrid metaheuristics: definition
Definition: What is a hybrid metaheuristic?
▶ Problem: a precise definition is not possible
Possible characterization:
A technique that results from the combination of a metaheuristic with
other techniques for optimization
What is meant by: other techniques for optimiation ?
▶ Metaheuristics
▶ Branch & bound
▶ Dynamic programming
▶ Integer Linear Programming (ILP) techniques
![Page 7: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/7.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Hybrid metaheuristics: history
History:
▶ For a long time the different communities co-existed quite isolated
▶ Hybrid approaches were developed already early, but only sporadically
▶ Only since about 15 years the published body of research grows
significantly:
1. 1999: CP-AI-OR Conferences/Workshops
2. 2004: Workshop series on Hybrid Metaheuristics (HM 200X)
3. 2006: Matheuristics Workshops
Consequence: The term hybrid metaheuristics identifies a new line of research
![Page 8: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/8.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Specific topic of this presentation
Specific topic today: Combination between metaheuristics and ILP Solvers
Available general purpose ILP solvers:
▶ IBM ILOG CPLEX: free for academic purposes
▶ Gurobi: free for academic purposes
▶ FICO Xpress: 30 days free trial. Restricted student version for free
▶ MOSEK: free for academic purposes
![Page 9: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/9.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Example of an integer linear program (1)
Optimization problem: Minimum weight dominating set (MWDS)
▶ Given: An undirected graph G = (V,E); each vi ∈ V has a weight w(vi) ≥ 0
▶ Valid solutions: Any subset S ⊆ V is a valid solution iff
∀ vi ∈ V : N [vi] ∩ S = ∅
▶ Optimization goal: Find a solution S∗ that minimizes
f(S∗) :=∑
vi∈S∗ w(vi)
1 2
3 1
4
1 2
3 1
4
1 2
3 1
44
1
1
![Page 10: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/10.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Example of an integer linear program (2)
Stating the MWDS problem as an ILP:
min∑vi∈V
w(vi) · xi
subject to:∑vj∈N [vi]
xj ≥ 1 for vi ∈ V
xi ∈ {0, 1} for vi ∈ V
(1)
(2)
Beware:
▶ In this ILP: linear number of variables and constraints
▶ Any problem may be expressed as an ILP in various different ways
![Page 11: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/11.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Why combining metaheuristics with ILP Solvers?
General advantage of metaheuristics:
▶ Very good in exploiting information on the problem (greedy heuristics)
▶ Generally very good in obtaining high-quality solutions for medium and even
large size problem instances
However:
▶ Metaheuristics may also reach their limits with growing problem instance size
▶ Metaheuristics fail when the information on the problem is misleading
Goal: Taking profit from valuable optimization expertise that went into the
development of ILP solvers even in the context of large problem instances
![Page 12: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/12.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Hybrid metaheuristics
ILP-based Large Neighborhood Search
Principle: Use of an ILP solver for finding the best neighborin a large neighborhood of a solution
David Pisinger, Stefan Ropke. Large Neighborhood Search, Handbook of
Metaheuristics, International Series in Operations Research & Management Science
Volume 146, 2010, pp 399-419
![Page 13: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/13.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Neighborhood search (1)
▶ Crucial ingredient of neighborhood search: Choice of a neighborhood
▶ Usual in metaheuristics based on neighborhood search: rather small
neighborhoods
Example of a small neighborhood: 2-opt neighborhood for the TSP. Each
solution has O(n2) neighbors.
Traveling Salesman Problem: 2-opt move
current solution possible neighbor
![Page 14: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/14.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Neighborhood search (2)
Example of a large neighborhood: in the context of the TSP
Traveling Salesman Problem: large neighborhood
current solution destroy partially: find best solution contining
![Page 15: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/15.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
General tradeoff in neighborhood search
▶ Small neighborhoods:
1. Advantage: It is fast to find an improving neighbor (if any)
2. Disadvantage: The average quality of the local minima is low
▶ Large neighborhoods:
1. Advantage: The average quality of the local minima is high
2. Disadvantage: Finding an improving neighbor might itself be NP -hard
due to the size of the neigbhorhood
Ways of examining large neighborhoods:
▶ Heuristically
▶ Exact techniques: for example an ILP solver
![Page 16: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/16.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
ILP-based large neighborhood search: Ilp-Lns
![Page 17: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/17.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Crucial aspect of Ilp-Lns
▶ Important: Applying an ILP solver to find the best solution containing a
specific partial solution Spartial means applying the ILP to a
reduced search space .
▶ Consequence: In comparison to the ILP solver, Ilp-Lns can be applied to
much bigger problem instances
Search Space
LNS
![Page 18: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/18.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Application example: Minimum weight dominating set
Original ILP:
min∑vi∈V
w(vi) · xi
subject to:∑vj∈N [vi]
xj ≥ 1 for vi ∈ V
xi ∈ {0, 1} for vi ∈ V
How to search for the best solution containing Spartial?
Adding the following constraints: xi = 1 for vi ∈ Spartial
![Page 19: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/19.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Generating the initial solution: Greedy (1)
Definitions:
▶ Vcov: set of covered nodes (w.r.t. a partial solution S)
▶ d(v|Vcov): current degree of v only considering covered nodes
Example:
v1 2
v6 1
v4
v2 v3
v7 v8
v5
S = {v1, v7}, Vcov = {v1, v2, v4, v6, v7, v8}, d(v3|Vcov) = 1
![Page 20: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/20.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Generating the initial solution: Greedy (2)
Pseudo-code of Greedy:
1: input: a graph G = (V,E) with node weights
2: S := ∅3: Vcov := ∅4: while Vcov = V do
5: v∗ := argmaxv∈V \Vcov
{d(v|Vcov)
w(v)
}6: S := S ∪ {v∗}7: Vcov := Vcov ∪N [v∗]
8: end while
9: output: S
![Page 21: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/21.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Partial destruction of a solution
General principle: removing a certain percentage percdest of the nodes in Scur
How to select nodes to be removed?
▶ Destruction type typedest = random : nodes are chosen uniformly at random
▶ Destruction type typedest = heuristically guided): node choice biased by
greedy function
Choice of value for percdest:
▶ A value is chosen dynamically from [percldest, percudest]
▶ Initially percdest := percldest
▶ When no better solution found: percdest := percdest + 5
▶ When better solution found of upper bound reached: percdest := percldest
![Page 22: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/22.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Benchmark instances
▶ Random graphs with |V | = {100, 1000, 5000, 10000} nodes
▶ Different edge probabilities ep (low, medium, high):
⋆ For |V | = 100: ep ∈ {0.03, 0.04, 0.05}⋆ For |V | > 100: ep ∈ {0.01, 0.03, 0.05}
1
7 9
8 7
9 8
2
8
9 6
9
2 8
3 7
4 3
1 0 0
3
3 5
6 5
6 7
8 5
8 8
6 1
6 6
7 0
9 4 4
1 5
5 7
6 9
9 0
8 9
6 4
8 0
8 3
5
2 6
4 1
6 2
7 8
8 6
4 0
5 3
9 9
7 5
9 7
7 6
8 4
9 5
9 3
6
2 7
9 2
5 4
3 3
7
4 7
5 0
9 1
3 8
5 5
7 2
1 0
5 2
6 8
1 1
1 8
3 9
6 0
3 6
5 9
1 2
2 4
4 6
1 3
2 2
1 4
1 6
1 7
4 4
4 5
7 3
1 9
2 0
4 8
2 1
2 3
2 5
4 9
8 1
5 8
2 9
3 0
7 7
3 1
5 6
3 2
7 1
3 4
7 4
8 2
4 2
5 1
6 3
![Page 23: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/23.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Tuning of Ilp-Lns
▶ typedest can be random or heuristically guided
▶ Lower and upper bound (percldest, percudest) for the destruction percentage:
1. (X,X) where X ∈ {10, 20, 30, 40, 50, 60, 70, 80, 90}2. (X,Y ) ∈ {(10, 30), (10, 50), (30, 50), (30, 70)}
▶ tmax: maximum CPU time for each application of the ILP solver
Selected values after tuning with irace:
|V | typedest (percldest, percudest) tmax
100 1 (60, 60) 2.0
1000 0 (90, 90) 10.0
5000 1 (50, 50) 5.0
10000 1 (40, 40) 10.0
![Page 24: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/24.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
MWDS results: improvement over Greedy (in percent)
10
15
20
25
30
n
Impr
ovem
ent o
f ILP
−LN
S o
ver
GR
EE
DY
100 1000 5000 10000
10
20
30
40
50
n
Impr
ovem
ent o
f ILP
−LN
S o
ver
GR
EE
DY
100 1000 5000 10000
5
10
15
20
25
30
35
n
Impr
ovem
ent o
f ILP
−LN
S o
ver
GR
EE
DY
100 1000 5000 10000
Low density Medium density High density
![Page 25: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/25.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
MWDS results: improvement over Cplex (in percent)
●0
2
4
6
8
n
Impr
ovem
ent o
f ILP
−LN
S o
ver
CP
LEX
100 1000 5000 10000
●●
●
●
●
0
2
4
6
8
10
n
Impr
ovem
ent o
f ILP
−LN
S o
ver
CP
LEX
100 1000 5000 10000
●●
0
2
4
6
8
10
n
Impr
ovem
ent o
f ILP
−LN
S o
ver
CP
LEX
100 1000 5000 10000
Low density Medium density High density
![Page 26: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/26.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Hypothesis and subsequent question
Hypothesis after studying the LNS literature:
LNS works especially well when the number of solution components
(variables) is linear concerning the input parameters of the tackledproblem
Question:
What can we do when the ILP of our tackled problem has a
large number of solution components (variables)???
![Page 27: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/27.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Hybrid metaheuristics
Construct, Merge, Solve & Adapt
Principle: Exact solution of sub-instances obtained by joiningsolutions
Christian Blum, Borja Calvo. A matheuristic for the minimum weight rooted
arborescence problem. Journal of Heuristics, 21(4): 479-499 (2015)
![Page 28: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/28.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Principal Idea
Observation: In the presence of a large number of solutions components, many of
them only lead to bad solutions
Idea: Exclude the presumably bad solution components from the ILP
Steps of the proposed method:
▶ Iteratively generate presumably good solutions in a probabilistic way
▶ Assemble a sub-instance from the used solution components
▶ Solve the sub-instance by means of an ILP solver
▶ Delete useless solution components from the sub-instance
![Page 29: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/29.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Construct, Merge, Solve & Adapt (Cmsa)
![Page 30: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/30.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Application example: Minimum Common String Partition (1)
Input:
1. Two related strings of length n over a finite alphabet Σ
2. Note: Two strings s1 and s2 are related iff the frequency of each letter in each
string is equal.
Valid solutions:
▶ Generate a partition P1 of non-overlapping substrings of s1
▶ Generate a partition P2 of non-overlapping substrings of s2
▶ Solution S = (P1, P2) is a valid solution iff P1 = P2
▶ Obj. function value: f(S) := |P1| = |P2|
Objective: Minimization
![Page 31: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/31.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Minimum Common String Partition (2)
Example:
▶ s1 := AGACTG, s2 := ACTAGG
▶ Trival solution:
⋆ P1 = P2 = {A,A,C,T,G,G}⋆ Obj. function value: 6
▶ Optimal solution S∗:
⋆ P1 = P2 = {ACT,AG,G}⋆ Obj. function value: 3
![Page 32: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/32.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Related Literature
Basic facts:
▶ Introduced in 2005 in the context of genome rearrangement
▶ Problem difficulty: NP-hard
Works from the literature:
▶ 2005: Greedy approach
▶ 2007: Introduction of approximation algorithms
▶ 2008: Study concerning fixed-parameter tractability (FPT)
▶ 2013: An ant colony optimization metaheuristic
![Page 33: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/33.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Preliminaries
Definitions: Given input strings s1 and s2 ...
▶ A common block bi is a triple (ti, k1i, k2i) where
1. ti is a string starting at position 1 ≤ k1i ≤ n in string s1
2. ti is a string starting at position 1 ≤ k2i ≤ n in string s2
▶ Set B is the set of all common blocks of s1 and s2
▶ Any valid (partial) solution S is a subset of B such that
1.∑
bi∈S |ti| = n (in the case of complete solutions)
2.∑
bi∈S |ti| < n (in the case of partial solutions)
3. For any bi, bj ∈ S it holds: ti and tj do not overlap neither in s1 nor in s2
![Page 34: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/34.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Common Block Example
Input strings: s1 = AGACTG and s2 = ACTAGG is as follows:
Set B of all common blocks:
b1 =(ACT, 3, 1) b8 =(A, 3, 4)
b2 =(AG, 1, 4) b9 =(C, 4, 2)
b3 =(AC, 3, 1) b10 =(T, 5, 3)
b4 =(CT, 4, 2) b11 =(G, 2, 5)
b5 =(A, 1, 1) b12 =(G, 2, 6)
b6 =(A, 1, 4) b13 =(G, 6, 5)
b7 =(A, 3, 1) b14 =(G, 6, 6)
Solution {ACT,AG,G}: S = {b1, b2, b14}
![Page 35: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/35.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
ILP Model (1)
Input strings: s1 = AGACTG and s2 = ACTAGG
B =
b1 =(ACT, 3, 1)
b2 =(AG, 1, 4)
b3 =(AC, 3, 1)
b4 =(CT, 4, 2)
b5 =(A, 1, 1)
b6 =(A, 1, 4)
b7 =(A, 3, 1)
b8 =(A, 3, 4)
b9 =(C, 4, 2)
b10 =(T, 5, 3)
b11 =(G, 2, 5)
b12 =(G, 2, 6)
b13 =(G, 6, 5)
b14 =(G, 6, 6)
M1 =
0 0 1 1 1 0
1 1 0 0 0 0
0 0 1 1 0 0
0 0 0 1 1 0
1 0 0 0 0 0
1 0 0 0 0 0
0 0 1 0 0 0
0 0 1 0 0 0
0 0 0 1 0 0
0 0 0 0 1 0
0 1 0 0 0 0
0 1 0 0 0 0
0 0 0 0 0 1
0 0 0 0 0 1
M2 =
1 1 1 0 0 0
0 0 0 1 1 0
1 1 0 0 0 0
0 1 1 0 0 0
1 0 0 0 0 0
0 0 0 1 0 0
1 0 0 0 0 0
0 0 0 1 0 0
0 1 0 0 0 0
0 0 1 0 0 0
0 0 0 0 1 0
0 0 0 0 0 1
0 0 0 0 1 0
0 0 0 0 0 1
![Page 36: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/36.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
ILP Model (2)
min
m∑i=1
xi
subject to:
m∑i=1
|ti| · xi = n
m∑i=1
M1i,j · xi = 1 for j = 1, . . . , n
m∑i=1
M2i,j · xi = 1 for j = 1, . . . , n
xi ∈ {0, 1} for i = 1, . . . ,m
(3)
(4)
(5)
(6)
Note: Very large number of solution components
![Page 37: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/37.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Set of solution components: properties
0
20
40
60
1 2 3 4 5 6 7 8 9Common block length
Num
ber
of b
lock
s (in
per
cent
)
Note: Most common blocks of length 1 and 2 will not appear in good solutions
![Page 38: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/38.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Simple Greedy Algorithm
Given a valid partial solution Spartial: B(Spartial) ⊂ B are the common blocks
that may be used in order to extend Spartial
Pseudo-code:
1. Spartial := ∅
2. while Spartial is not a complete solution
▶ Choose the longest common block bi from B(Spartial)}▶ Spartial := Spartial ∪ {bi}
Note: This algorithm is used in Cmsa in a probabilistic way
![Page 39: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/39.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Benchmark instances and tuning
Benchmark instances: 300 instances
▶ String length n ∈ {200, 400, . . . , 1800, 2000}
▶ Alphabet size |Σ| ∈ {4, 12, 20}
Tuning results with irace:
n na agemax drate lsize tmax
400 50 10 0.0 10 60
800 50 10 0.5 10 240
1200 50 10 0.9 10 480
1600 50 5 0.9 10 480
2000 50 10 0.9 10 480
![Page 40: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/40.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
MCSP results: improvement over prob. Greedy (|Σ| = 4)
2
4
6
8
10
12
n
200 400 600 800 1000 1200 1400 1600 1800 2000Impr
ovem
ent o
f CM
SA o
ver S
OLC
ONS
T
![Page 41: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/41.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
MCSP results: improvement over prob. Greedy (|Σ| = 12)
0
2
4
6
8
10
n
200 400 600 800 1000 1200 1400 1600 1800 2000Impr
ovem
ent o
f CM
SA o
ver S
OLC
ONS
T
![Page 42: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/42.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
MCSP results: improvement over prob. Greedy (|Σ| = 20)
0
2
4
6
8
10
n
200 400 600 800 1000 1200 1400 1600 1800 2000Impr
ovem
ent o
f CM
SA o
ver S
OLC
ONS
T
![Page 43: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/43.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
MCSP results: improvement over Cplex (|Σ| = 4)
0
5
10
15
20
n
200 400 600 800 1000 1200 1400 1600 1800 2000
Impr
ovem
ent o
f CM
SA o
ver C
PLEX
![Page 44: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/44.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
MCSP results: improvement over Cplex (|Σ| = 12)
0
5
10
15
n
200 400 600 800 1000 1200 1400 1600 1800 2000
Impr
ovem
ent o
f CM
SA o
ver C
PLEX
x
![Page 45: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/45.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
MCSP results: improvement over Cplex (|Σ| = 20)
0
5
10
n
200 400 600 800 1000 1200 1400 1600 1800 2000
Impr
ovem
ent o
f CM
SA o
ver C
PLEX
![Page 46: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/46.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Evolution of the (sub-)instance sizes
ILP Σ 4 L
ILP Σ 4 S
ILP Σ 12 L ILP Σ 12 S
ILP Σ 20 L
ILP Σ 20 S
GSA Σ 4 S L
GSA Σ 12 S L
GSA Σ 20 S L
(dashed curves)
0e+00
1e+05
2e+05
200 400 600 800 1000 1200 1400 1600 1800 2000n
Inst
ance
Siz
e
![Page 47: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/47.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Differences between Lns and Cmsa: summarized
How is the original problem instance reduced?
Search Space
LNS
Search Space
CMSA
How is the sub-instance of the next iteration generated?
▶ Lns: Partial destruction of the incumbent solution
▶ Cmsa: Generating new solutions and removing old solution components
![Page 48: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/48.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Summary and Possible Research Directions
Summary:
▶ Cmsa: A new hybrid algorithmm for combinatorial optimization
▶ Hypothesis:
⋆ Lns better for problems with a linear number solution components
⋆ Cmsa better for problems with a super-linear number of components
Possible Research Directions:
▶ Solution construction: adaptive probabilities over time
▶ A more intelligent version of the aging mechanism
▶ Theoretical studies about the differences between Lns and Cmsa
![Page 49: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/49.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
People involved in this research
Christian Blum Borja Calvo Pedro Pinacho
Jose Antonio Lozano Manuel Lopez-Ibanez
![Page 50: Combining Metaheuristics with ILP Solvers in Combinatorial ...€¦ · Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid ⃝c C. Blum Motivation and Outline (1) Motivation:](https://reader036.vdocument.in/reader036/viewer/2022081400/5f0d78797e708231d43a849a/html5/thumbnails/50.jpg)
Combining Metaheuristics with ILP Solvers, INISTA 2015, Madrid c⃝ C. Blum
Outlook
Questions?Literature:
▶ C. Blum, B. Calvo. A matheuristic for the minimum weight rooted
arborescence problem. Journal of Heuristics, 21(4): 479-499 (2015)
▶ C. Blum, J. Puchinger, G. R. Raidl, A. Roli. Hybrid metaheuristics in
combinatorial optimization: A survey. Applied Soft Computing, 11(6):
4135–4151 (2011)
Forthcoming book: C. Blum, G. R. Raidl. Hybrid Meta-
heuristics – Powerful Tools for Optimization, Springer Series on
Artificial Intelligence, 2015