overview of mixed-integer (linear) programming: recent ...focapo-cpc.org/pdf/linderoth.pdf ·...
Post on 09-Sep-2018
236 Views
Preview:
TRANSCRIPT
Mixed-Integer (Linear) Programming: Recent Advances,and Future Research Directions
Jeff Linderoth
Dept. of ISyEUniv. of Wisconsin-Madison
linderoth@wisc.edu
@JeffLinderoth
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 1 / 58
Introduction
Mixed Integer Linear Program: (MILP)
minx,z
c>x+ d>z
s.t. Ax+ Bz ≤ bx ∈ Rn+z ∈ Zp
Integer variables always have bounds (e.g. zi ∈ {0, 1})
Powerful Modeling Paradigm
“On-off” constraints
Logical conditions between constraints and variables
Cardinality constraints: Select at most k out of p alternatives
Nonlinearities: Fixed-charges, piecewise linear functions
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 2 / 58
Introduction Make Fun of Christos
Instructions
At this point, we would like to send
you some guidelines:
1) Please limit your talk to *35
minutes*. We would like to have ample
time for discussion.
...5) One of the major pieces of input we
received from the technical advisory
committee was that invited talks should
not only present new results, but also
succinctly (1) review the state-of-the
art, and (2) discuss new challenges and
open questions.
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 3 / 58
Introduction Make Fun of Christos
MISSION IMPOSSIBLE
Talk may be a bit basic for some of the audience
There will be some (biased) discussion of my own team’scontributions
If you aren’t a (recent) MILP user, I have only one take-away message:
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 4 / 58
Introduction Make Fun of Christos
Branch-And-Cut: The “Virginia Slims” of Algorithms
If you haven’t tried MILP because you believe that it is “impossible”to solve, you may be wrong
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 5 / 58
Progress in MIP Solvers Main Solvers
The Big Three
Three Primary Solvers for MILP
1 IBM CPLEX: (Version 12.7)
2 FICO XPRESS: (Version 8)
3 Gurobi: (Version 7)
There are other commercial solvers (e.g.) Mosek, but for MILP theseare the big three
Commercial Solvers remain head and shoulders1 abovefree/open-source solvers
SCIP: http://scip.zib.de/ Free for non-academic useMIP-CL: http://www.mipcl-cpp.appspot.com/Coin-Cbc: Part of open-source Coin-OR initiative coin-or.org
1Maybe 6-7 times fasterLinderoth (UW ISyE) Quo Vadis MIP FOCAPO 6 / 58
Progress in MIP Solvers NEOS
NEOS: www.neos-server.org
Allow users to solve optimization problems on remote resources
Started at Argonne National Lab in 1994
Moved to UW-Madison (Wisconsin Institute of Discovery) in 2010.
Web interface, API (XML-RPC)
Access to 11 different solvers for MILP
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 7 / 58
Progress in MIP Solvers NEOS
NEOS Daily Jobs, 2016
Over one million optimizationproblems solved in 2016!
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 8 / 58
Progress in MIP Solvers NEOS
Most Popular Problems/Solvers
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 9 / 58
Progress in MIP Solvers MILP Speedups
CPLEX MILP Improvement: v1-v11
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 10 / 58
Progress in MIP Solvers MILP Speedups
CPLEX MILP Improvement: v6-v12.6.1
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 11 / 58
Progress in MIP Solvers MILP Speedups
Calculations
Improvement in MIP Software from 1988-2017
Algorithms: 147650x
Machines: 17120x
http://preshing.com/20120208/a-look-back-at-single-threaded-cpu-performance/
NET: (Algorithm × Machine): 2,527,768,000x
What Does This “Mean”?
A “typical” MILP that would have taken 124 years to solve in 1988will solve in 1 second now.
This is amazing, but your mileage may vary
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 12 / 58
Progress in MIP Solvers Stories
In case you don’t believe me...
Electrical Power Industry, ERPI GS-6401, June 1989:
Mixed-integer programming (MIP) is a powerful modelingtool. They are, however, theoretically complicated andcomputationally cumbersome.
As a result, the utility (power) industries did not use mixed-integerlinear programming for many years.
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 13 / 58
Progress in MIP Solvers Stories
Join the New Decade
Bob Bixby, Founder of CPLEX, and now the“Bi” in GuRoBi:
“Give me your most difficult MILP model”
California 7-Day Model
m = 48939 constraints,n = 25755 variables, p = 2856integer variables
“Many Days”—No solution.
(> 1 hour to solve LPrelaxation)
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 14 / 58
Progress in MIP Solvers Stories
Dateline: 9/27/1999
At the DIMACS/EPRI Workshop on Next Generation of UnitCommitment Models
http://dimacs.rutgers.edu/Workshops/NextGeneration/
program.html
Solved model in 22 mins to fulloptimality.
Using CPLEX v6.5
During his talk!
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 15 / 58
Progress in MIP Solvers Stories
Now Even Faster!Gurobi Optimizer version 5.5.0
Read MPS format model from file unitcal_7.mps.bz2
Optimize a model with 48939 Rows, 25755 Columns and 127595 NonZeros
Presolved: 38804 Rows, 19960 Columns, 105627 Nonzeros
Root relaxation: objective 1.945018e+07, 18340 iterations, 0.60 seconds
Nodes | Current Node | Objective Bounds | Work
Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time
0 0 1.9450e+07 0 721 - 1.9450e+07 - - 2s
0 0 1.9596e+07 0 559 - 1.9596e+07 - - 16s
0 0 1.9598e+07 0 487 - 1.9598e+07 - - 20s
H 0 0 2.066856e+07 1.9598e+07 5.18% - 25s
13 11 1.9669e+07 6 217 2.0669e+07 1.9602e+07 5.16% 649 30s
36 28 1.9668e+07 9 219 2.0669e+07 1.9605e+07 5.15% 707 35s
H 93 84 1.998399e+07 1.9605e+07 1.90% 342 37s
100 74 1.9678e+07 17 204 1.9984e+07 1.9606e+07 1.89% 321 43s
* 271 111 54 1.972042e+07 1.9606e+07 0.58% 170 43s
H 417 29 1.964604e+07 1.9606e+07 0.21% 129 43s
858 178 1.9629e+07 11 141 1.9637e+07 1.9609e+07 0.14% 96.9 50s
H 924 187 1.963578e+07 1.9609e+07 0.13% 94.6 51s
H 987 221 1.963563e+07 1.9611e+07 0.12% 93.9 53s
1024 237 1.9630e+07 19 107 1.9636e+07 1.9611e+07 0.12% 93.5 56s
H 1034 237 1.963556e+07 1.9611e+07 0.12% 92.8 56s
1144 288 1.9630e+07 14 501 1.9636e+07 1.9611e+07 0.12% 89.0 63s
1245 294 1.9621e+07 30 399 1.9636e+07 1.9619e+07 0.09% 131 185s
1673 261 1.9623e+07 35 120 1.9636e+07 1.9623e+07 0.07% 117 190s
Explored 2167 nodes (274011 simplex iterations) in
194.37 seconds
Optimal solution found (tolerance 1.00e-04)
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 16 / 58
Progress in MIP Solvers ran14x18
Story #2: ran14x18
In 1998, there was no Twitter. The best search engine was Altavista2,and there were “newsgroups” on the Interwebs where you could askquestions.
Challenge post to sci.op-research, August 4, 1998: (From authorsof paper Sun et al. (1998))
“We tried CPLEX 5.0 on this instance, obtaining a solutionwith objective value 3712 but without proving optimality.The same solution was also found in some runs of thegenetic algorithm. In response to my posting onsci.op-research, some people tried exact solvers on thisinstance (fctp, mps), but without success. I also contactedCPLEX, but they also failed.”
2Google was incoporated in August 1998Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 17 / 58
Progress in MIP Solvers ran14x18
Solving ran14x18
Dateline, Nov 13, 1998:
“I proved the optimality of the solution3712 for the ran14x18 FCTP instance. Inorder to do this I ran my PARINO parallelMIP code for about 52 hours on 32Pentium II (300MHz) processors. 7624677nodes of the branch and bound tree wereevaluated in this time. PARINO adds flowcover inequalities which are usually quiteuseful in helping to improve the LP boundsfor these types of problems. Also, I used aparallel pseudocost-type branching rule
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 18 / 58
Progress in MIP Solvers ran14x18
CPLEX Too
Dateline, Nov 25, 1998
We are in the process of developing anew version of CPLEX, so we decidedto try the ran14x18 problem with ournew version on a large number ofparallel processors. We used an SGIOrigin system with 64 processors. Ourparallel development code solved it in8982253 nodes, and 3 hours 12minutes wall clock time.
NB: This was when Gu (now the “Gu” in GuRoBi) just joined CPLEX
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 19 / 58
Progress in MIP Solvers ran14x18
Now: gurobi v7.0 on my Windows Desktop
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 20 / 58
Progress in MIP Solvers ran14x18
The Seymour Problem
To find a “short” proof of the Four-Color Theorem3, Robertson,Seymour, Thomas wanted to find the smallest set of “configurations”such that the Four-Color Theorem is true if none of theseconfigurations can exist in a minimal counterexample.
This can be posed as an integer program
Although Seymour claimed to have found a solution with objectivevalue 423, nobody (including Seymour himself) had been able eitherto reproduce this solution, or prove a strong lower bound on theoptimal value.
There was some skepticism in the integer programming community asto whether this was indeed the optimal value.
3Any map can be colored using four colors in such a way that regions sharing aboundary segment receive different colors.
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 21 / 58
Progress in MIP Solvers ran14x18
Ferris + Linderoth versus Seymour
Still a young and energetic Postdoc, I visited Michael Ferris atUW-Madison in 1999.
After setting a varety of algorithmic parameters, we set CPLEX 6.5running: It ran (using grid-software computing tool called HTCondorthat could checkpoint, save, and relocate computation to differentmachines) for > 10 years in the UW-Madison HTCondor Pool.
BUMMER!
It never solved!
Hooray!
With a different method using disjunctive cuts, and some“extreme” enumeration (in parallel), Pataki, Schmieta, Ceria,Ferris, and Linderoth solved in in 9000 CPU hours.
Indeed the solution is 423.
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 22 / 58
Progress in MIP Solvers ran14x18
Now It’s Officially Easy!
This is just amazing!
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 23 / 58
MILP Algorithms Components
How Did They Do It?
1 Hired very good people!
2 Mined the academic literature. (Read: stole all of our good ideas).
Key Idea!
Recognize and exploit commonly appearing structures:
Preprocessing techniques
Cutting planes
Improved search/branching
Heuristic methods for finding feasible solutions
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 24 / 58
MILP Algorithms Components
(M)ILP Feasible Region
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 25 / 58
MILP Algorithms Components
Convex Hull
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 26 / 58
MILP Algorithms Components
Cutting Planes
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 27 / 58
MILP Algorithms Components
Cutting Planes
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 28 / 58
MILP Algorithms Components
Branching
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 29 / 58
MILP Algorithms Components
Branching Makes A Tree
x1 =1
x2 = 0
x1 = 0
x2 = 1
Recursively solve IP at each node
Fathom is lower bound at node is larger than value of a knownfeasible solution
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 30 / 58
MILP Algorithms Components
MIP Building Blocks
Presolve: Tighten formulation and reduce problem size
Solve continuous relaxations
Ignoring integralityGives a bound on the optimal integral objective
Cutting planes: Cut off relaxation solutions
Branching variable selection: Intelligently explore search space
Primal heuristics: Find integer feasible solutions
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 31 / 58
MILP Algorithms Components
Branch And Cut Components
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 32 / 58
MILP Algorithms Components
Some Papers
Branching
An old(ish) survey of methods: (Linderoth and Savelsbergh, 1999)
“Reliability Branching”: (Achterberg et al., 2004)
Cutting
Gomory Cuts: (Balas et al., 1999)
(0-1/2) Cuts: (Caprara and Fischetti, 1996)
Disjunctive Cuts: (Balas and Perregaard, 2003; Bonami andMinoux, 2005)
Flow Covers: Gu et al. (1999)
MIR: Marchand and Wolsey (2001)
Surveys: Cornuejols (2008); Conforti et al. (2009)
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 33 / 58
MILP Algorithms Components
More Papers
Heuristics
Local Branching: (Fischetti and Lodi, 2003)
Feasibility Pump: (Fischetti et al., 2005; Achterberg and Berthold,2007)
RINS: (Danna et al., 2005)
Presolve
Basic reduction in MILP: (Savelsbergh, 1994)
Achterberg et al. (2016): A 70(!) page paper that explainspresolve operations in Gurobi
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 34 / 58
MILP Algorithms Components
{1} is a Series!
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 35 / 58
MILP Algorithms Components
Importance of Each Component
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 36 / 58
Orbital Branching
Improvement in Branching: Orbital Branching
One recent branching improvement is for solving symmetric MILPs
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 37 / 58
Orbital Branching
Symmetry is Pretty!
The A matrix of a symmetric integer program
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 38 / 58
Orbital Branching
Symmetry Background
(We assume all variables binary for this discussion)
Let Πn be collection of permutations of {1, 2, . . . , n} := [n]
π ∈ Πn is a symmetry of IP if...1 x feasible ⇔ π(x) feasible2 cTx = cTπ(x)
The set of symmetries of IP (with composition of permutations)forms the symmetry group of IP
G(IP) = {π ∈ Πn | π(x) ∈ F , cTx = cTπ(x) ∀x ∈ F },
where F = {x ∈ {0, 1}n | Ax ≤ b} is the set of feasible solutions
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 39 / 58
Orbital Branching
Why Symmetry is Bad – Wasted Search!
Suppose the permutation (1, 2) ∈ G
x1 =1
x2 = 0
x1 = 0
x2 = 1
You evaluate many completely equivalent (isomorphic) subtrees
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 40 / 58
Orbital Branching
Orbits
For a point z ∈ Rn, the orbit of z under G is the set of all elements towhich z can be sent by permutations in G:
orb(G, z) def= {π(z) | π ∈ G}.
The union of the orbits of each variable forms a partition of [n].
The orbits encode which variables are “equivalent” (symmetric) withrespect to the symmetry G.
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 41 / 58
Orbital Branching Basic Idea
Orbital Branching
Let O be an orbit of the symmetry group of the IP.
Surely we can branch as∑i∈O
xi ≥ 1 or∑i∈O
xi ≤ 0.
If at least one variable i ∈ O is going to be one, and they are all“equivalent”, then you may as well pick (i∗) one arbitrarily.
x∗i = 1 or∑i∈O
xi = 0
No, really. That’s it. :-)
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 42 / 58
Orbital Branching Basic Idea
An Alternative View of Orbital Branching
Suppose that you have found that the variables xe, xf, xg and xhshare an orbit at node a, O = {e, f, g, h}.
Then you can surely branch as:
a
e f g hxe = 1 xf = 1 xg = 1 xh = 1
∑j∈O xj = 0
But the best solution you can find from nodes f, g, and h will be thesame as the best solution you can find from node e
In fact, the solutions will be isomorphic⇒ Prune nodes f, g, and h
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 43 / 58
Orbital Branching Computational Results
Tale of the Tape—Gurobi v3.0
Symmetry = 0 Symmetry = 2Instance Time Gap% Nodes Time Gap% Nodes
ecc105 7200 50.0 150 173 0.0 7ecc83 7200 15.0 724601 6 0.0 372ecc93 7200 20.0 108572 905 0.0 54650
codbt05 7200 7.4 352025 7200 3.7 359268codbt33 8 0.0 604 6 0.0 401codbt42 159 0.0 75569 111 0.0 45912codbt61 10 0.0 1485 7 0.0 950cov1053 7200 5.9 919836 77 0.0 10958cov1054 7200 2.0 189645 2330 0.0 103657cov1075 7200 5.0 549355 17 0.0 665cov954 58 0.0 31950 1 0.0 166sts27 1 0.0 4044 0 0.0 78sts45 18 0.0 61194 23 0.0 34839sts63 7200 4.4 8698168 85 0.0 43135sts81 7200 16.4 3252747 70 0.0 6317
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 44 / 58
Orbital Branching Computational Results
Tale of the Tape—CPLEX v12.1
Symmetry = 0 Symmetry = 5Instance Time Gap% Nodes Time Gap% Nodescod105 7200 52.4 13201 606 0.0 1120cod83 7200 14.3 1418001 79 0.0 15452cod93 7200 18.9 389028 7200 6.3 639001
codbt05 7200 5.6 1035046 150 0.0 23059codbt33 8 0.0 1049 1 0.0 14codbt42 89 0.0 84039 4 0.0 2141codbt61 8 0.0 1833 1 0.0 61cov1053 7200 5.9 1495461 2234 0.0 448008cov1054 7200 2.0 191970 7200 2.0 169371cov1075 7200 6.4 1505168 57 0.0 12227cov954 64 0.0 36563 3 0.0 1351sts27 0 0.0 3532 0 0.0 1307sts45 10 0.0 59890 6 0.0 28775sts63 1585 0.0 7692765 736 0.0 3607609sts81 7200 13.1 23933498 7200 11.5 23415204
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 45 / 58
Future Directions: Nonlinear
The World Is Nonlinear
An anecdote: July, 1948. A young andfrightened George Dantzig, presents hisnewfangled “linear programming” to ameeting of the Econometric Society ofWisconsin, attended by distinguishedscientists like Hotelling, Koopmans, and VonNeumann. Following the lecture, Hotelling(in Dantzig’s words “a huge whale of aman”) pronounced to the audience:
But we all know the world isnonlinear!
The world is indeednonlinear
Physical Processes andProperties
EquilibriumEnthalpy
Abstract Measures
Economies of ScaleCovarianceUtility of decisions
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 46 / 58
Future Directions: Nonlinear
Mixed-Integer Nonlinear Optimization
Mixed Integer Linear Program: (MILP)
zminlp = minx,η
η
s.t. gi(x) ≤ 0, i = 1, . . . ,m
f(x) ≤ ηxI ∈ Z|I|
f : Rn → R, g : Rn → Rm smooth, sometimes convex functions.
I ⊆ {1, . . . , n} subset of integer variables
We may assume that f is a linear function.
NP-Super-Hard
Combines challenges of handling nonlinearitieswith combinatorial explosion of integer variables
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 47 / 58
Future Directions: Nonlinear
Specializations
Functional Form Problem Type
gi(x) = ‖Aix+ bi‖2 − pTi x+ qi MISOCP
gi(x) = xTQix+ p
Ti + qi MIQP
gi(x) = aTi x− bi MILP
MISOCP: Mixed Integer Second Order Cone Program
Are convex-MINLP
MIQP: Mixed Integer Quadratic Program
May be convex or nonconvexConvex MIQP is a special case of MISOCPIf f is convex quadratic and g is an affine mapping, then there arespecialized algorithms for convex-MIQP
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 48 / 58
Future Directions: Nonlinear
Aside – MISOCP
The feasible regions of surprisingly many (convex) MINLP can beexpressed as MISOCP:
Hyperbolic constraints: Product of (non-negative) variables ≥ anorm2.
wTw ≤ xy, x ≥ 0, y ≥ 0 ⇔ ∥∥∥∥[ 2w
x− y
]∥∥∥∥ ≤ x+ yLubin et al. (2016) show that every convex MINLP in the availabletest libraries can be reformulated to be a conic MINLP
Most are quadratic cone, but need also “power cone” and “exponentialcone”
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 49 / 58
Future Directions: Nonlinear
New Math
Once academics could no longer compete against the likes of CPLEX,they turned their attentions to MINLP.
Try to apply same MILP improvement techniques to MINLP
MI
+
NLP
6=
MINLP
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 50 / 58
Future Directions: Nonlinear
MINLP Progress (via MILP)
Undaunted by initial failures, many MILP researchers continue towork in MINLP
At MIP 2014 and MIP 2015, 7 of the 22 talks each each were aboutMINLP
Some Work Product. (There’s LOTS More)
Cutting planes: (Atamturk and Narayanan, 2010; Kılınc-Karzan,2016; Kılınc et al., 2017; S. Modaresi and Vielma, 2016)
Heuristics: (Bonami et al., 2008)
Preprocessing: (Grossmann and Lee, 2003; Gunluk and Linderoth,2010)
Branching: (Bonami et al., 2011)
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 51 / 58
Future Directions: Nonlinear
Leading To Improvements – Convex MIQP
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 52 / 58
Future Directions: Nonlinear
Leading To Improvements – Nonconvex MIQP
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 53 / 58
Future Directions: Nonlinear Wrapping Up
Optima 101
I recently had the opportunity to interview Bob Bixby for Optima(The Mathematical Optimization Society Newsletter):http://www.mathopt.org/Optima-Issues/optima101.pdf
“In some sense, companies like CPLEX,XPRESS, and Gurobi put academics
doing computational IP kind of out ofbusiness. Many people are doing MINLP
now.”
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 54 / 58
Future Directions: Nonlinear Wrapping Up
Bixby Response
“I think there is still plenty of room forMIP actually. The truth is that therehas to be relatively little thats come outof the research community in the last 10years or so.
Certainly RINS was a big thing. Thatshad a huge influence. Symmetry testingwas a big thing, but not a huge thing.That got put in Gurobi, based on orbitalbranchings.
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 55 / 58
Future Directions: Nonlinear Wrapping Up
Conversation Continued
“It seems harder and harder to generateand demonstrate good/impactful ideaswithout access to the internals ofcommercial IP solvers.”
“Its perfectly clear, its very hard to testwhether something is a good idea or not.The most you can hope for these days isthat it looks reasonable when you do alittle bit of testing. Then you knowpeople working for the commercialsolvers will look at it and try to dosomething.Managing cutting planes can be verydifficult, and we would actually be happyif there were some nice thing we coulddo.”
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 56 / 58
Future Directions: Nonlinear Wrapping Up
Conclusions: Misson (NOT) Accomplished
Take Away Messages!
1 Christos Asks The Impossible!2 MILP has come a long way baby!
A positive academic/commercial“partnership”If you are not using MILP, Take herfor a spin!
3 MINLP is a current “frontier” ofprimary improvement for commercialsoftware
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 57 / 58
Future Directions: Nonlinear Wrapping Up
Things We Still Can’t Do Well
Optimization solver so far onlyeffectively deal with algebraicsystems!
Paul is here to help!
We can’t effectively solveoptimization problems with bothinteger variables and uncertainparameters
Shabbir is the man!
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 58 / 58
References
T. Achterberg and T. Berthold. Improving the feasibility pump. Discrete Optimization, 4:77–86,2007.
T. Achterberg, T. Koch, and A. Martin. Branching rules revisited. Operations Research Letters,33:42–54, 2004.
T. Achterberg, R. E. Bixby, Z. Gu, E. Rothberg, and D. Weninger. Presolve reductions in mixedinteger programming. Technical Report ZIB-Report 16-44, Konrad-Zuse-Zentrum furInformationstechnik Berlin, 2016.
A. Atamturk and V. Narayanan. Conic mixed integer rounding cuts. MathematicalProgramming, 122:1–20, 2010.
E. Balas, S. Ceria, G. Cornuejols, and N.R. Natraj. Gomory cuts revisited. Operations ResearchLetters, 19:1–9, 1999.
Egon Balas and Michael Perregaard. A precise correspondence between lift-and-project cuts,simple disjunctive cuts, and mixed integer Gomory cuts for 0-1 programming. MathematicalProgramming, 94:221–245, 2003.
P. Bonami and M. Minoux. Using rank-1 lift-and-project closures to generate cuts for 0-1 MIPs,a computational investigation. Discrete Optimization, 2:288–307, 2005.
P. Bonami, G. Cornuejols, A. Lodi, and F. Margot. A feasibility pump for mixed integernonlinear programs. Mathematical Programming, 119:331–352, 2008.
P. Bonami, J. Lee, S. Leyffer, and A. Wachter. More branch-and-bound experiments in convexnonlinear integer programming. Preprint ANL/MCS-P1949-0911, Argonne NationalLaboratory, Mathematics and Computer Science Division, September 2011.
A. Caprara and M. Fischetti. {0, 12} chvatal-gomory cuts. Mathematical Programming, 74:
221–235, 1996.Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 0 / 17
References
M. Conforti, G. Cornuejols, and G. Zambelli. Polyhedral approaches to mixed integer linearprogramming. In M. Junger, T. Liebling, D. Naddef, W. Pulleyblank, G. Reinelt, G. Rinaldi,and L. Wolsey, editors, 50 Years of Integer Programming 1958—2008. Springer, 2009.
G. Cornuejols. Valid inequalities for mixed integer linear programs. Mathematical Programming,112:3–44, 2008.
E. Danna, E. Rothberg, and C. LePape. Exploring relaxation induced neighborhoods to improveMIP solutions. Mathematical Programming, 102:71–90, 2005.
M. Fischetti and A. Lodi. Local branching. Mathematical Programming, 98:23–47, 2003.
M. Fischetti, F. Glover, and A. Lodi. The feasibility pump. Mathematical Programming, 104:91–104, 2005.
I. Grossmann and S. Lee. Generalized convex disjunctive programming: Nonlinear convex hullrelaxation. Computational Optimization and Applications, pages 83–100, 2003.
Z. Gu, G. L. Nemhauser, and M. W. P. Savelsbergh. Lifted flow covers for mixed 0-1 integerprograms. Mathematical Programming, 85:439–467, 1999.
O. Gunluk and J. Linderoth. Perspective relaxation of mixed integer nonlinear programs withindicator variables. Mathematical Programming Series B, 104:186–203, 2010.
M. Kılınc, J. Linderoth, and J. Luedtke. Effective separation of disjunctive cuts for convex mixedinteger nonlinear programs. Mathematical Programming Computation, 2017. To appear.
Fatma Kılınc-Karzan. On minimal inequalities for mixed integer conic programs. Mathematics ofOperations Research, 41:477–510, 2016.
J. T. Linderoth and M. W. P. Savelsbergh. A computational study of search strategies in mixedinteger programming. INFORMS Journal on Computing, 11:173–187, 1999.
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 0 / 17
Perspective Cuts
M. Lubin, E. Yamangil, R. Bent, and J. P. Vielma. Extended formulations in mixed-integerconvex programming. In IPCO 2016: The Eighteenth Conference on Integer Programmingand Combinatorial Optimization, volume 9682, pages 102–113. Springer, 2016.
H. Marchand and L. Wolsey. Aggregation and mixed integer rounding to solve MIPs.Operations Research, 49:363–371, 2001.
M. R. Kln S. Modaresi and J. P. Vielma. Intersection cuts for nonlinear integer programming:Convexification techniques for structured sets. Mathematical Programming, 155:575–611,2016.
M. W. P. Savelsbergh. Preprocessing and probing techniques for mixed integer programmingproblems. ORSA Journal on Computing, 6:445–454, 1994.
M. Sun, J.E. Aronson, P. McKeown, and D. Drinka. A tabu search heuristic procedure for thefixed charge transportation problem. European Journal of Operational Research, 106(2):441–456, 1998.
D. Vandenbussche and G. L. Nemhauser. A branch-and-cut algorithm for nonconvex quadraticprograms with box constraints. Mathematical Programming, 102:559–575, 2005.
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 1 / 17
Perspective Cuts
Preprocessing for MINLP
MILP Force: Exploit The Structure!
Mixed Integer Linear Programmers carefully study simple problemstructures to come up with “good” formulations for problems
Good formulations closely approximate convex hull of feasiblesolutions
We need to do this for MINLP
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 1 / 17
Perspective Cuts
Indicator MINLPs
Binary variables z are used as indicator variables.
If zi = 0, components of x controlled by zi collapse to a point
If zi = 1, components of x controlled by zi belong to a convex set
Process Flow Applications
z = 0⇒ x1 = x2 = x3 = x4 = 0
z = 1⇒ f(x1, x2, x3, x4) ≤ 0
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 2 / 17
Perspective Cuts Convex Hull Characterizations
A Very Simple Example
Rdef={(x, y, z) ∈ R2 × B | y ≥ x2, 0 ≤ x ≤ uz
}
z = 0⇒ x = 0, y ≥ 0z = 1⇒ x ≤ u, y ≥ x2
x
y
z = 1
z
y ≥ x2
Deep Insights
conv(R) ≡ line connecting (0, 0, 0) to y = x2 in the z = 1 plane
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 3 / 17
Perspective Cuts Convex Hull Characterizations
Characterization of Convex HullDeep Theorem #1
R ={(x, y, z) ∈ R2 × B | y ≥ x2, 0 ≤ x ≤ uz
}conv(R) =
{(x, y, z) ∈ R3 | yz ≥ x2, 0 ≤ x ≤ uz, 0 ≤ z ≤ 1, y ≥ 0
}
x2 ≤ yz, y, z ≥ 0 ≡
The Only NLP I like: Second Order Cone Programming
x2 − yz is not convex
There are effective, robust algorithms for optimizing over conv(R)
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 4 / 17
Perspective Cuts Perspective
Giving You Some Perspective
For a convex function f : Rn → R, the perspective functionP : Rn+1 → R of f is
P(x, z) def=
{0 if z = 0zf(x/z) if z > 0
The epigraph of P(x, z) is a cone pointed at the origin whose lowershape is f(x)
Exploiting Your Perspective
If zi is an indicator that the (nonlinear, convex) inequality f(x) ≤ 0must hold, (otherwise x = 0), replace the inequality with itsperspective version:
zif(x/zi) ≤ 0
The resulting (convex) inequality is a much tighter relaxation ofthe feasible region.
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 5 / 17
Perspective Cuts Computations
Nonlinear Facility Location Problem
Problem studied by Gunluk, Lee, and Weismantel (’07) and classes ofstrong cutting planes derived
M: Facilities
N: Customers
xij: percentage of customer j ∈ N demand met by facility i ∈Mzi = 1⇔ facility i ∈M is built
Fixed cost for opening death star i ∈MCost of serving j ∈ N from i ∈M is proportion to teh square of theamount/percentage
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 6 / 17
Perspective Cuts Computations
Nonlinear Facility Location Formulation
z∗def= min
∑i∈M
cizi +∑i∈M
∑j∈N
qijx2ijyij
subject to
xij ≤ zi ∀i ∈M, ∀j ∈ N∑i∈M
xij = 1 ∀j ∈ N
xij ≥ 0 ∀i ∈M, ∀j ∈ Nzi ∈ {0, 1} ∀i ∈M
x2ij − ziyij ≤ 0 ∀i ∈M, ∀j ∈ N
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 7 / 17
Perspective Cuts Computations
Strength of Relaxations
zR: Value of NLP relaxation
zGLW : Value of NLP relaxation after GLW cuts
zP: Value of perspective relaxation
z∗: Optimal solution value
|M| |N| zR zGLW zP z∗
10 30 140.6 326.4 346.5 348.715 50 141.3 312.2 380.0 384.120 65 122.5 248.7 288.9 289.325 80 121.3 260.1 314.8 315.830 100 128.0 327.0 391.7 393.2
Woo Hoo!
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 8 / 17
Perspective Cuts Computations
Impact of SOCP
m = 30, n = 100
Bonmin B&B, GLW, Original: 16697 CPU seconds, 45901 nodes
Bonmin B&B, GLW, w/ineq: 21206 CPU seconds, 29277 nodes
Bonmin B&B, Perspective, 4201 CPU seconds, 39 B&B nodes
Mosek SOCP, Perspective, 23 CPU seconds, 44 B&B nodes
Larger Instances
m n T N
30 200 141.9 6340 100 76.4 5440 200 101.3 4550 100 61.6 4950 200 140.4 47
“The Force isStrong with This
One”
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 9 / 17
Piecewise Linear
Piecewise Linear
Low-dimensional nonlinearities (even non-convex ones) can beeffectively modeled with piecewise linear functions
MILP methods are used to optimize!
Gurobi/CPLEX: Specialized interface for specifying piecewise linearfunctions
Gurobi: specialiez linear programming (simplex) method for solvingpiecewise linear programs
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 10 / 17
Piecewise Linear
Decomposition
CPLEX v12.7 has an automatic mechanism for detecting whether otnot the instance has a structure that is amenable to Benders’Decomposition
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 11 / 17
Piecewise Linear
Cloud/Parallel
Most solvers are building/have built parallel solvers
Multi-core for some time, but more recently they have distributedsolvers.
Gurobi
1 Distributed MIP: Solve a single MIP model across multiplemachines. Manager machine passes problem data to a set ofworker machines and coordinates the overall solution process.
2 Distributed concurrent MIP: Doesn’t divide work, but rather eachmachine uses a different strategy. This seems stupid, but it can infact be very effective.
3 Distributed tuning: Tests model with variety of parameter settings
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 12 / 17
Piecewise Linear
Today’s Problem
Nonconvex (Box) QP. Q 6� 0
(boxqp) z∗ := max{x ∈ H | 1/2 xTQx+ cTx}
H = {x ∈ R[n] | `i ≤ xi ≤ ui ∀i ∈ [n]}
xTQx =∑
(i,j)∈EQijxixj +∑i∈[n] 1/2 Qiix
2i
Why Would Anyone Care?
As a relaxation of
max{x ∈ (H ∩ P) | 1/2 xTQx+ cTx}
for some polyhedron P
Improve on solver for nonconvex QP that is in CPLEX v12.6
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 13 / 17
Piecewise Linear
Gu Ro Bi
Zhonghao Gu Ed Rothberg Bob Bixby
Gurobi
A leading commercial provider of state-of-the-art software forsolving optimization problems
Estimated Company Value: $15,000,000
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 14 / 17
Piecewise Linear
Gu Bo Li
Oktay Gunluk Pierre Bonami Jeff Linderoth
GuBoLi
A made-up company consisting of three also-ran optimizers
Estimated Company Value: $0.15
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 15 / 17
Piecewise Linear
GuBoLi—Showing OffAvg. CPU Time on small (n ≤ 40) instances for well-known test setof Vandenbussche and Nemhauser (2005).
Couenne v0.4CPLEX v12.6.1GuBoLi (Gunluk, Bonami, Linderoth) v0.00001
Couenne CPLEX GuBoLi
0
1,000
2,000
3,000(2535)
(1670)
(1.57)
CP
UT
ime
“There’s only one thingto do at a moment like
this: strut!”
Bart’s Girlfriend, S6:E7
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 16 / 17
Piecewise Linear
Linderoth (UW ISyE) Quo Vadis MIP FOCAPO 17 / 17
top related