1 sequence grammars for very large scale neighborhood search agustin bompadre james orlin ismp 2006
TRANSCRIPT
1
Sequence Grammars for Very Large Scale
Neighborhood Search
Agustin Bompadre
James Orlin
ISMP 2006
2
On Combinatorial Optimization
Recent incredible advances in our ability to solve IPs
But… combinatorial optimization problems are still difficult to solve, and always will be
We will always rely on other approaches
– Neighborhood search
– Other heuristics
3
Very Large Scale Neighborhood (VLSN) Search
Neighborhood search:
– Iteratively replace a current solution with the
best solution in a neighborhood VLSN Search:
– The neighborhoods are too large to search explicitly
• exponentially large
– Neighborhood can be searched efficiently
• Polynomial time
4
Background on VLSN Search:Collaborators with Ravi Ahuja and Jim OrlinRicha Agarwal, Georgia Institute of
Technology, Atlanta
Agustin Bompadre, MIT
Ozlem Ergun, Georgia Institute of Technology, Atlanta
Jon Goodstein, United Airlines, Chicago
Wei Huang, University of Florida, Gainesville
Krishna C. Jha, University of Florida, Gainesville
Arvind Kumar, University of Florida, Gainesville
Gilbert Laporte, University of Montreal, Montreal
Jian Liu, University of Florida, Gainesville
D. Romero Morales, Maastricht University, Netherlands
Amit Mukherjee, United Airlines, Chicago
Stefano Pallottino, University of Pisa, Pisa, Italy
Abraham P. Punnen, University of New Brunswick, Fredericton
H. Edwin Romeijn, University of Florida, Gainesville
Maria P. Scaparra, University of Pisa, Pisa, Italy
Maria G. Scutellà, University of Pisa, Pisa, Italy
Dushyant Sharma, University of Michigan, Ann Arbor
Zuo J. Shen, University of Florida, Gainesville
Larry Shughart, Charles River Associates, Boston
5
VLSN Search has been applied to …
Classic Combinatorial Problems
– TSP, QAP, GAP, and more Vehicle routing Problems Partitioning Problems Airline Fleet Scheduling Locomotive Scheduling Block Scheduling for Rail and more Weapon Target Assignment Timetabling Problems Telecomunications etc
6
Some References
http://www.ise.ufl.edu/ahuja/vlsn/index.htm
http://web.mit.edu/jorlin/www
R.K. Ahuja, O. Ergun, J.B. Orlin, and A.P. Punnen. 2002.
R.K. Ahuja, O. Ergun, J.B. Orlin, and A.P. Punnen. 2005
Deineko and Woeginger [2000]
7
Overview of this talk
Unifying structure for a variety of VLSN search techniques for sequencing problems
– Based on context free grammars and regular grammars
– Encompasses a number of previous results for the TSP
– Extends to other sequencing problems
8
Two Sequencing Problems
minimize (f() : Sn)
– Sn is the set of permutations on n objects
Example 1: TSP1
( ), (1) ( ), ( 1)1( )
n
n i iif c c
9
,, : ( ) ( )( ) i ji j i j
f c
Linear Ordering Problem (LOP)
e.g., in ranking college football teams
A little known result
M Groetschel, M Juenger, G
Reinelt , 1984
10
Assume initial sequence is 1, …, n
Example: the swap neighborhood
1 2 3 4 5 6 7 8
swap [4,7] 1 5 62 3 7 84
5 3 8 1 2 7 6 4
swap [4,7] 5 3 8 6 2 7 1 4
swap [4,7] swap elements in positions 4 and 7.
4 7
1 6
A neighborhood of the permutation 1, 2, …, n induces a neighborhood for any other permutation.
11
A very large Neighborhood: Pyramidal tours Klyaus [1976]
N = { : 1 = (1) < (2) < … < (k) for some k and (k) > (k+1) > … > (n) }
1 4 6 8 7 5 3 2
Size of neighborhood: 2n-1
Time to search the neighborhood: O(n2)
12
Another Very Large Scale Neighborhood
Example: Balas-Simonides nbhd [2001]
N = { : (1) = 1 and if (i) < (j) then i < j+K for all i, j}
1 3 4 2 5 8 6 7
an example when K = 2
Exponential size: even if K = 2.
Time to search the neighborhood: O(n K2 2K)
13
The research
Uses grammars to describe exponentially large sequence grammars.
Polynomial time DP algorithm for finding the min
cost neighbor for
– the TSP
– the LOP
– many other problems
14
What is a grammar?
A grammar consists of a set of symbols, and production rules that explain how to replace one or more symbols by a different set of symbols.
Context free grammars were developed by Chomsky [1956]
15
What is a context Free Grammar?
S aB
S ab
B Sb
S
ab aB
aSb
aabb aaBb
aaSbb
aaaBbbaaabbbThis generates the language consisting of sequences of the form anbn.
In a Context free grammar each production rule has one non-terminal on the left.
Example
16
Features of a Context Free Grammar
A “start symbol” {S}
A set of “terminals” {a, b}
A set of “non-terminals” {S, B}
Production Rules: S aB
S ab
B Sb
Language: {anbn: n = 1, 2, …}
S aB
S ab
B Sb
17
Sequence grammars (Bompadre and Orlin, 2005)
Terminals of the grammar: {1, 2, …, n}.
– one terminal for each “city.”
Non-terminals of the grammar: a set of symbols.
– May be thought of as subsets of cities
– Applying production rules to non-terminal A will eventually generate a string of cities
– e.g., A 5 - 3 - 9 - 2
– A 9 - 2 - 5 - 3
For each non-terminal A, there is a set of cities called Cities(A). e.g., Cities(A) = {2, 3, 5, 9}.
– A always generates strings on Cities(A)
18
The pyramidal tour neighborhoodKlyaus [1976]
Let Cities(Tj) = { j, j+1, j+2, …, n}
The start non-terminal is T1
Rules
T1 1, T2
Tj j, Tj+1 for j = 2 to n-1
Tj Tj+1, j for j = 2 to n-1
Tn n
24 6 7 5 31 8
T21
1 2T3
21 3T4
231 4 T5
24 31 5T6
24 5 31 6 T7
24 6 5 31 7T8O(n) rules
24 6 7 5 31 8
19
The semi-twisted neighborhood
Let cities(Tij) = { i, i+1, i+2, …, j}
The start non-terminal is T1n.
Rules
T1n 1, T2,n
Tij i, Ti+1,j for i < j
Tij Ti+1,j, i for i < j
Tij j, Ti,j-1 for i < j
Tij Ti,j-1, j for i < j
Tjj j
23 5 6 4 81 7
T2,81
1 2T3,8
21 8T3,7
281 3 T4,7
23 81 4T5,7
23 4 81 5 T6,7
23 5 4 81 6T7,7
23 5 6 4 81 7
If we consider items added from last to first, they are consecutive.O(n2) rules.
20
The twisted neighborhood: Aurenheimer [1988]
Let cities(Tij) = { i, i+1, i+2, …, j}
The start non-terminal is S.
Rules
S T1n
Tik Tij, Tj+1,k for i j k
Tik Tj+1,k , Tij for i j k
Tjj j for all j
O(n3) rules.
T1,3T4,8
T4,8 T1,2 3
T1,2 3T4,5 T6,8
3T1,2T6,85 4
35 4 T6,8 2 1
T1,8
35 4 2 1T6,78
35 4 2 178 6
21
Can every polynomially sized neighborhood be generated in polynomial time?
• A 1, 2, 3, 4, 5
• A 2, 1, 3, 4, 5
• A 3, 2, 1, 4, 5
• A 4, 2, 3, 1, 5
• A 5, 2, 3, 4, 1
• A 1, 3, 2, 4, 5
. . .
Yes!
Example: The two swap neighborhood.
Let cities(A) = {1, 2, 3, 4, 5}
There is a non-terminal called the start non-terminal. In this case it is city A
22
What is the advantage of representing neighborhoods as sequence grammars?
We have a “meta dynamic program” for grammar-induced neighborhood for the TSP
If the grammar has K-rules, the generated DP runs in O(Kn3) time for finding the best neighbor
Often the bound can be improved.
DP Generator
Rules of the grammar
Input
DP for searching the neighborhood
Output
23
More on the dynamic program
DP running time.
– O(Kn3) time for finding the best neighbor
– Time analysis can be improved in some cases
O(n2) time for pyramidal tour nbhd.
O(n3) time for semi-twisted nbhd.
O(n6) time for twisted nbhd.
– previous best was O(n7).
24
Dynamic Programming Illustration
Some rules for pyramidal tour neighborhood
pj: Tj j, Tj+1
qj: Tj Tj+1, j for j = 2 to n-1
Let f(i, Tj, k) = min length, starts at i, ends at k, visits Tj
T4
4 T5
4T5
orf(4, T4, j) = mink {c4k+ f(k, T5, j)}
f(j, T4, 4) = mink {f(j, T5, k) + ck4}
25
On Pyramidal Tours
Rules
O(n) rules
O(n2) states
O(n2) time to run the recursion.
What’s new?
The DP takes the rules as input and forms the DP automatically.
It separates the solver from the formulation
26
Running times for the TSP
Neighborhood Running time
2-opt O(n2)
Pyramidal Tour O(n2)
Tree-based grammars with K rules O(Kn3)
Twisted Sequence O(n6) **
Semi-twisted Sequence O(n3)
Dynasearch Nbhds O(n2)
Balas-Simonetti O(k2kn)
Improves upon O(n7) algorithm by Deineko and Woeginger [2000]
27
The Linear Ordering Problem
g(k) is the min cost ordering of items k to n
T4
4 T5
4T5
or
g(4) = g(5) + c45 + c46 + c47 + c48
Consider the pyramidal tour neighborhood.
g(4) = g(5) + c54 + c64 + c74 + c84
O(n2) time. O(n) time if data is preprocessed.
28
Running times for the LOP
Neighborhood Running time
2-opt O(n2)
Pyramidal Tour O(n2)
Tree-based grammars with K rules O(Kn2)
Twisted Sequence O(n3)
Semi-twisted Sequence O(n2)
Dynasearch Nbhds O(n2)
Balas-Simonetti O(k2kn)
29
Summary so far
Use context free grammars to create exponentially large neighborhoods
time to search the neighborhoods optimally is polynomial time in n and K for TSP and LOP
next: some natural questions arising from this research
30
Do we get new neighborhoods?
Yes! e.g., we can create new grammars by taking unions or subsets
– The union of two sequence grammars induces a new neighborhood
– Any subset of a sequence grammar induces a new grammar
Open question: can one create interesting new neighborhoods?
31
1 3 5 7 9
2 6 84
Can all polynomially searchable neighborhoods be efficiently represented as sequence grammars?
keep odd cities fixed; optimally insert even cities.
1 2 3 4 5 6 7 8 9
This nbhd requires an exponential number of rules if represented as a sequence grammar.
No! e.g., Sarvanov and Doroshko [1981] assignment neighborhood for the TSP..
32
Given a context free grammar, can one efficiently recognize if it is a sequence grammar?
Suppose that G is a context free grammar.
e.g., T1 1, T2
Tj j, Tj+1 for j = 2 to n-1Tj Tj+1, j for j = 2 to n-1Tn n
1. To recognize G, use recursion to determine Cities(A) for all non-terminals A.
2. Then verify that Cities(A) is unique for each A, and that if A B, C, then cities(A) = cities(B) cities(C)…
Tn n implies that Cities(Tn) = {n}
Tn-1 Tn, n-1 implies that Cities(Tn-1} = {n-1, n}…
Yes!
33
A natural extension is “context sensitive grammars”. Are these useful in VLSN Search?
CSGs permit rules such as : ABC ADdC
– This rule involving B is sensitive to its context
– WLOG, we can also permit rules of the form AB BA
The following grammar generates all tours
S A1, A2, …, An
Aj j for j = 1 to n
AiAj AjAi for all i, j
It has O(n2) rules, and the nbhd cannot be searched in polynomial time.
No!
34
Summary and next directions
Sequence grammars for representing many exponential sized neighborhoods for sequence problems
Dynamic programming algorithms for searching these neighborhoods for the TSP, linear ordering problem, and several other sequence problems.
Next directions: grammars that are useful for VLSN Search for other combinatorial problems