![Page 1: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/1.jpg)
mixed-integer convex optimization
Miles Lubinwith Emre Yamangil, Russell Bent, Juan Pablo Vielma, Chris CoeySeptember 16, 2016
MIT & Los Alamos National Laboratory
![Page 2: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/2.jpg)
Nonlinear (nonconvex) optimization,
minx
f(x)
subject to gj(x) ≤ 0, ∀j
- Example: AC power flow, PDEs- Global solution is hard- Be happy with local search, or- Approximate as another optimization problem we know how tosolve
- Search for global solutions via convex relaxations
1
![Page 3: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/3.jpg)
Which optimization problems can we solve to global optimality?
2
![Page 4: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/4.jpg)
Convex optimization
- All local optima are global- Typically efficient, polynomial-time algorithms
Image source: https://en.wikipedia.org/wiki/Convex_set#/media/File:Convex_polygon_illustration1.svg
3
![Page 5: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/5.jpg)
Mixed-integer linear programming (MILP),
minx
cTx
subject to Ax = b,x ≥ 0,xi ∈ Z, ∀i ∈ I
- Despite NP-Hardness, many problems of practical interest canbe solved to optimality or near optimality
- Algorithms are based on LP relaxations, branch & bound, cuts,preprocessing, heuristics, ...
- 50+ years of commercial investment in developing thesetechniques
4
![Page 6: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/6.jpg)
who uses milp?
http://www.gurobi.com/company/example-customers5
![Page 7: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/7.jpg)
example: unit commitment
Minimize unit-production costs + fixed operating costssubject to:
- Total generation = total demand, hourly over 24h- Generation and transmission limits
◦ If a generator is on, it produces within some interval [l,u],otherwise its production is zero
- Linear approximation of nonlinear powerflow laws
1% reduction in generation gives billions of dollars per year costreduction. Worth spending time to improve optimality.
6
![Page 8: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/8.jpg)
Mixed-integer convex programming (MICP),
minx
cTx
subject to gj(x) ≤ 0, ∀jxi ∈ Z, ∀i ∈ I
Objective function linear (WLOG) and constraints gj convex
- Maybe MICP can be a more useful approximation of nonconvexproblems than convex or MILP alone
7
![Page 9: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/9.jpg)
what can be modeled using micp?
Disjunctions/unions of compact convex sets (Stubbs and Mehrotra,1999)
{x : f1(x) ≤ 0 or f2(x) ≤ 0}
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
8
![Page 10: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/10.jpg)
The following set is not MICP representable
-3 -2 -1 0 1 2 3
-3
-2
-1
0
1
2
3
9
![Page 11: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/11.jpg)
State of the art for solving MICP
minx
cTx
subject to gj(x) ≤ 0, ∀jxi ∈ Z,∀i ∈ I
- gj convex, smooth- Bonmin, KNITRO, α-ECP, DICOPT, FilMINT, MINLP_BB, SBB, ...
10
![Page 12: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/12.jpg)
outer approximation algorithm
Idea: Reformulate as another optimization problem we know how tosolve (MILP)Given points x∗1 , . . . , x∗R, solve mixed-integer linear relaxation:
minx
cTx
subject to gj(x∗r ) +∇gj(x∗r )T(x− x∗r ) ≤ 0, ∀j, r = 1, . . . ,Rxi ∈ Z, ∀i ∈ I
Then solve convex problem with integer values fixed to get a newfeasible solution, and repeat with R = R+ 1.
11
![Page 13: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/13.jpg)
c
x′
x∗
x′
12
![Page 14: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/14.jpg)
not good enough...
[...] solution algorithms for [MICP] have benefit from thetechnological progress made in solving MILP and NLP.However, in the realm of [MICP], the progress has been farmore modest, and the dimension of solvable [MICP] bycurrent solvers is small when compared to MILPs and NLPs.
– Bonami, Kilinç, and Linderoth (2012)
13
![Page 15: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/15.jpg)
what can go wrong?
Classical approaches form polyhedral outer approximations by afinite collection of “gradient linearizations”. But a good polyhedralouter approximation might need too many linear constraints.Recall:
B1 := {x : ||x||1 ≤ 1} =
{x :
n∑i=1
sixi ≤ 1, s ∈ {−1,+1}n}
14
![Page 16: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/16.jpg)
B1 := {x : ||x||1 ≤ 1} =
{x :
n∑i=1
sixi ≤ 1, s ∈ {−1,+1}n}
Standard trick in linear programming is to introduce extra variables:
B1 ={x ∈ Rn : ∃ y ∈ Rn s.t. x ≤ y, x ≥ −y,
∑iyi ≤ 1
}The new formulation has 2n variables and 2n+ 1 constraints versusn variables and 2n constraints. Not bad.We call this an extended formulation.
15
![Page 17: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/17.jpg)
the same happens with smooth constraints
Hijazi et al. (2014):
Bn :=
x ∈ {0, 1}n :n∑j=1
(xj −
12
)2≤ n− 1
4
16
![Page 18: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/18.jpg)
what can go wrong?
Hijazi et al. (2014):
Bn :=
x ∈ {0, 1}n :n∑j=1
(xj −
12
)2≤ n− 1
4
- Bn is empty- No outer approximating hyperplane can exclude more than oneinteger lattice point
- So any polyhedral outer approximation which contains nointeger vertices needs at least 2n hyperplanes
17
![Page 19: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/19.jpg)
how to fix it?
Bn :=
x ∈ {0, 1}n :n∑j=1
(xj −
12
)2≤ n− 1
4
Consider the equivalent formulation in an extended set of variables:
B̂n :=
x ∈ {0, 1}n, z ∈ Rn :n∑j=1
zj ≤n− 14 ,
(xj −
12
)2≤ zj ∀j
- Bn = projx B̂n
- If you form a polyhedral relaxation of each(xj − 1
2)2 ≤ zj
individually, then 2n hyperplanes in R2n is sufficient to excludeall integer points
- Outer approximation algorithm now converges in 2 iterations- Why? Small polyhedron in higher dimension can haveexponentially many facets in lower dimension
18
![Page 20: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/20.jpg)
how to fix it?
Bn :=
x ∈ {0, 1}n :n∑j=1
(xj −
12
)2≤ n− 1
4
Consider the equivalent formulation in an extended set of variables:
B̂n :=
x ∈ {0, 1}n, z ∈ Rn :n∑j=1
zj ≤n− 14 ,
(xj −
12
)2≤ zj ∀j
- Bn = projx B̂n
- If you form a polyhedral relaxation of each(xj − 1
2)2 ≤ zj
individually, then 2n hyperplanes in R2n is sufficient to excludeall integer points
- Outer approximation algorithm now converges in 2 iterations- Why? Small polyhedron in higher dimension can haveexponentially many facets in lower dimension
18
![Page 21: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/21.jpg)
how to fix it?
Bn :=
x ∈ {0, 1}n :n∑j=1
(xj −
12
)2≤ n− 1
4
Consider the equivalent formulation in an extended set of variables:
B̂n :=
x ∈ {0, 1}n, z ∈ Rn :n∑j=1
zj ≤n− 14 ,
(xj −
12
)2≤ zj ∀j
- Bn = projx B̂n
- If you form a polyhedral relaxation of each(xj − 1
2)2 ≤ zj
individually, then 2n hyperplanes in R2n is sufficient to excludeall integer points
- Outer approximation algorithm now converges in 2 iterations
- Why? Small polyhedron in higher dimension can haveexponentially many facets in lower dimension
18
![Page 22: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/22.jpg)
how to fix it?
Bn :=
x ∈ {0, 1}n :n∑j=1
(xj −
12
)2≤ n− 1
4
Consider the equivalent formulation in an extended set of variables:
B̂n :=
x ∈ {0, 1}n, z ∈ Rn :n∑j=1
zj ≤n− 14 ,
(xj −
12
)2≤ zj ∀j
- Bn = projx B̂n
- If you form a polyhedral relaxation of each(xj − 1
2)2 ≤ zj
individually, then 2n hyperplanes in R2n is sufficient to excludeall integer points
- Outer approximation algorithm now converges in 2 iterations- Why? Small polyhedron in higher dimension can haveexponentially many facets in lower dimension
18
![Page 23: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/23.jpg)
“extended formulation for second-order cone”
SOCn := {(t, x) ∈ Rn : ||x||2 ≤ t}
Second order cone programming (SOCP) generalizes convexquadratic programming (QP).
Vielma, Dunning, Huchette, Lubin (2015):
- For MISOCP: Rewrite ∑ix2i ≤ t2
as ∑izi ≤ t, where zi ≥ x2i /t
- Implemented by CPLEX and Gurobi within months ofpublication
19
![Page 24: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/24.jpg)
“extended formulation for second-order cone”
SOCn := {(t, x) ∈ Rn : ||x||2 ≤ t}
Second order cone programming (SOCP) generalizes convexquadratic programming (QP).Vielma, Dunning, Huchette, Lubin (2015):
- For MISOCP: Rewrite ∑ix2i ≤ t2
as ∑izi ≤ t, where zi ≥ x2i /t
- Implemented by CPLEX and Gurobi within months ofpublication
19
![Page 25: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/25.jpg)
we know what to do...
Hijazi et al. (2014) propose to reformulate constraints of the formn∑i=1
gi(xi) ≤ 0
ton∑i=1
zi ≤ 0 and gi(xi) ≤ zi i = 1, . . . ,n,
where gi univariate, smooth, convex.
- Very impressive computational results- Paper submitted in 2011, but there’s still no solver whichautomates this transformation
20
![Page 26: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/26.jpg)
Why has nobody implemented this?
minx
f(x)
subject to gj(x) ≤ 0, ∀jxi ∈ Z, ∀i ∈ I
- MICP solvers view f and gj through “black-box” oracles forevaluations of values and derivatives
- Summation structure not accessible through the “black box”
21
![Page 27: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/27.jpg)
Why has nobody implemented this?
minx
f(x)
subject to gj(x) ≤ 0, ∀jxi ∈ Z, ∀i ∈ I
- MICP solvers view f and gj through “black-box” oracles forevaluations of values and derivatives
- Summation structure not accessible through the “black box”
21
![Page 28: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/28.jpg)
So what if we open the black box and access the algebraicstructure? Is it sufficient?
If h(x) = f(x) + g(x) is convex, not necessary for f and g to beconvex. (E.g., f(x) = x21 − x22 and g(x) = 2x22.)So if we have access to the algebraic representation, in principleneed to solve a subproblem of convexity detection beforeconstructing extended formulation. Convexity detection is a hardproblem.
22
![Page 29: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/29.jpg)
So what if we open the black box and access the algebraicstructure? Is it sufficient?If h(x) = f(x) + g(x) is convex, not necessary for f and g to beconvex. (E.g., f(x) = x21 − x22 and g(x) = 2x22.)
So if we have access to the algebraic representation, in principleneed to solve a subproblem of convexity detection beforeconstructing extended formulation. Convexity detection is a hardproblem.
22
![Page 30: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/30.jpg)
So what if we open the black box and access the algebraicstructure? Is it sufficient?If h(x) = f(x) + g(x) is convex, not necessary for f and g to beconvex. (E.g., f(x) = x21 − x22 and g(x) = 2x22.)So if we have access to the algebraic representation, in principleneed to solve a subproblem of convexity detection beforeconstructing extended formulation. Convexity detection is a hardproblem.
22
![Page 31: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/31.jpg)
Disciplined convex programming (DCP) as a solution
- Algebraic modeling concept proposed by Grant, Boyd, and Ye.Implemented in CVX, CVXPY, and Convex.jl.
- Users forced to prove convexity of their functions by followingbasic composition rules.
23
![Page 32: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/32.jpg)
Disciplined convex programming (DCP) as a solution
-√x2 + y2 not DCP valid, use norm(
[x1 x2
])
- log(exp(x1) + exp(x2)) not DCP valid, use logsumexp([x1 x2
])
- DCP library of atoms, plus valid compositions◦ Sum of convex functions is convex◦ f(g(x)) is convex when f is convex and nondecreasing and gis convex
24
![Page 33: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/33.jpg)
DCP composition rules correspond exactly to extendedformulations!!
- Sum of convex functions is convex◦ f(x) + g(x) ≤ 0 −→ f(x) ≤ t1,g(x) ≤ t2, t1 + t2 ≤ 0
- f(g(x)) is convex when f is convex and nondecreasing and g isconvex◦ f(g(x)) ≤ 0 −→ g(x) ≤ t, f(t) ≤ 0◦ Tawarmalani and Sahinidis (2005) show that polyhedralouter approximations that exploit composition structurehave extra strength.
25
![Page 34: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/34.jpg)
DCP composition rules correspond exactly to extendedformulations!!
Not a coincidence. To enforce f(x) + g(x) ≤ 0, it is sufficient to knowhow to optimize over the intersection of the epigraphs{(t, x) : f(x) ≤ t}, {(t, x) : g(x) ≤ t} and linear constraints.
DCP was designed to access state-of-the-art conic solvers, does notuse gradients.
26
![Page 35: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/35.jpg)
conic optimization
minx
cTx
s. t. Ax = bx ∈ K
- K is a cone if x ∈ K implies αx ∈ K∀α ≥ 0- Usually we consider K = K1 × · · · ×Ks where each Kj is one of asmall number of cones like Rn
+, SOCn, the cone of positivesemidefinite matrices...
27
![Page 36: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/36.jpg)
DCP
minx
f(x)
s. t. gj(x) ≤ 0, ∀j
Conic
minx
cTx
s. t. Ax = bx ∈ K
Optimal solution x∗
CVXCVXPYConvex.jl
ECOSSCS
Mosek
28
![Page 37: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/37.jpg)
Example: ∑iexp(cTi x+ bi) ≤ t
is expressed in conic form as∑izi ≤ t and (cTi x+ bi, 1, zi) ∈ EXP.
whereEXP = cl{(x, y, z) ∈ R3 : y exp(x/y) ≤ z, y > 0}
is the exponential cone.
The translation to conic form already produces a formulation withadditional variables ideal for polyhedral outer approximation.
29
![Page 38: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/38.jpg)
Example: ∑iexp(cTi x+ bi) ≤ t
is expressed in conic form as∑izi ≤ t and (cTi x+ bi, 1, zi) ∈ EXP.
whereEXP = cl{(x, y, z) ∈ R3 : y exp(x/y) ≤ z, y > 0}
is the exponential cone.The translation to conic form already produces a formulation withadditional variables ideal for polyhedral outer approximation.
29
![Page 39: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/39.jpg)
Example:exp(x2 + y2) ≤ t
is DCP compliant. CVX will generate equivalent formulation
exp(z) ≤ t where x2 + y2 ≤ z
which is then translated to EXP and second-order cones.
30
![Page 40: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/40.jpg)
- Mixed-integer second order cone programming (MISOCP) hasrapidly improving commercial support by Gurobi/CPLEX◦ Gurobi claims 2.8x improvement from version 6.0 to 6.5 (1year)
- Otherwise (exponential cones, SDP), mainly research codes(ecos_bb by Han Wang, 2014; SCIP-SDP by Mars, 2012)
- For MISOCP, no established open-source codes
31
![Page 41: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/41.jpg)
How general is MISOCP?MINLPLIB2 benchmark library designed for derivative-basedmixed-integer nonlinear solvers. We classified all of the convexinstances by conic representability.Of the 333 convex instances, 65% are MISOCP representable.
- tls6 instance previously unsolved. We translated it toConvex.jl and it was solved by Gurobi!
32
![Page 42: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/42.jpg)
What about the remaining 35% of problems?Of remaining 116 instances, in 107 the only nonlinear expressionsinvolve exp and log.
Recall the exponential cone:
EXP = cl{(x, y, z) ∈ R3 : y exp(x/y) ≤ z, y > 0}
- Closed, convex, nonsymmetric cone. Theoretically tractable andsupported by ECOS (Akle, 2015)
33
![Page 43: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/43.jpg)
What about the remaining 35% of problems?Of remaining 116 instances, in 107 the only nonlinear expressionsinvolve exp and log.Recall the exponential cone:
EXP = cl{(x, y, z) ∈ R3 : y exp(x/y) ≤ z, y > 0}
- Closed, convex, nonsymmetric cone. Theoretically tractable andsupported by ECOS (Akle, 2015)
33
![Page 44: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/44.jpg)
What about the remaining 9 of 333 instances?
- Seven representable by combination of EXP and second-ordercones (SOC)
- Two representable by the power cone
POWα = {(x, y, z) ∈ R3 : |z| ≤ xay1−a, x ≥ 0, y ≥ 0}
34
![Page 45: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/45.jpg)
Folklore: Almost all convex optimization problems of practicalinterest can be represented as conic programming problems usingsecond-order, positive semidefinite, exponential, and power cones.
35
![Page 46: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/46.jpg)
So cones are general and provide a good extended formulation ofthe original convex problem.How do we apply polyhedral outer approximation to mixed-integerconic problems?
- We’ll state the first finite-time outer approximation algorithmfor general mixed-integer conic optimization
36
![Page 47: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/47.jpg)
MIDCP
minx
f(x)
s. t. gj(x) ≤ 0, ∀jxi ∈ Z, ∀i ∈ I
MICONE
minx
cTx
s. t. Ax = bx ∈ K
xi ∈ Z, ∀i ∈ I
Optimal solution x∗
CVXCVXPYConvex.jl
???
37
![Page 48: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/48.jpg)
mixed-integer conic programming
minx,z
cTz
s.t. Axx+ Azz = b (MICONE)L ≤ x ≤ U, x ∈ Zn, z ∈ K,
- K = K1 ×K2 × · · · × Kr, where each Ki is a standard cone:nonnegative orthant, second-order cone, exponential cone,power cone, or even positive semidefinite cone
- WLOG, integer variables do not belong to cones, have zeroobjective coefficients
38
![Page 49: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/49.jpg)
Need to outer approximate cone K with finite number of linearconstraints...
Given a closed, convex cone K, the dual cone K∗ is the set such thatz ∈ K iff zTβ ≥ 0 ∀β ∈ K∗.
- Nonnegative orthant, second-order and positive semidefinitecone are self dual. Exponential and power cones are not
- Any finite subset of K∗ yields a polyhedral outer approximationof K!
39
![Page 50: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/50.jpg)
Need to outer approximate cone K with finite number of linearconstraints...Given a closed, convex cone K, the dual cone K∗ is the set such thatz ∈ K iff zTβ ≥ 0 ∀β ∈ K∗.
- Nonnegative orthant, second-order and positive semidefinitecone are self dual. Exponential and power cones are not
- Any finite subset of K∗ yields a polyhedral outer approximationof K!
39
![Page 51: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/51.jpg)
minx,z
cTz
s.t. Axx+ Azz = b (MIOA(T))L ≤ x ≤ U, x ∈ Zn,
βTz ≥ 0 ∀β ∈ T,
- If T = K∗, then equivalent to MICONE- If T ⊆ K∗ and |T| < ∞, then polyhedral outer approximation →MILP relaxation of MICONE
- Claim: ∃T ⊆ K∗, |T| < ∞ such that MIOA(T) is equivalent toMICONE
40
![Page 52: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/52.jpg)
Consider the subproblem with integer values x = x̂ fixed:
vx̂ = minz
cTz
s.t. Azz = b− Axx̂, (CP(x̂))z ∈ K.
The dual of CP(x̂) is
maxβ,λ
λT(b− Axx̂)
s.t. β = c− ATzλβ ∈ K∗.
41
![Page 53: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/53.jpg)
the conic outer approximation algorithm
- Start with T = ∅ (or small)- Until the optimal objective of MIOA(T) is ≥ the optimalobjective of the best feasible solution:◦ Let x̂ be the optimal integer solution of the MILP problemMIOA(T)
◦ Solve conic problem CP(x̂), add dual solution βx̂ to T◦ If CP(x̂) was feasible, record new feasible solution toMICONE
Finite termination assuming strong duality holds at subproblems
42
![Page 54: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/54.jpg)
the conic outer approximation algorithm
- Start with T = ∅ (or small)- Until the optimal objective of MIOA(T) is ≥ the optimalobjective of the best feasible solution:◦ Let x̂ be the optimal integer solution of the MILP problemMIOA(T)
◦ Solve conic problem CP(x̂), add dual solution βx̂ to T◦ If CP(x̂) was feasible, record new feasible solution toMICONE
Finite termination assuming strong duality holds at subproblems
42
![Page 55: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/55.jpg)
the conic outer approximation algorithm
- Easy to implement, requires black box MILP and conic solvers- Very often, number of iterations is < 10, sometimes 10-30.Worst observed is 171.
- Each MILP subproblem could be NP-Hard, but takes advantageof fast solvers
43
![Page 56: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/56.jpg)
recap
- Polyhedral approximations can be made much better byintroducing extra variables
- Summation structure and compositions of convex functionsshould be broken up
- Nobody automated this before! (Derivative-based viewprevents implementation)
- DCP to the rescue. Conic form is general and encodes theinformation we need as a solver
- Proposed first outer approximation algorithm for MICONE,based on conic duality
44
![Page 57: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/57.jpg)
recap
- Polyhedral approximations can be made much better byintroducing extra variables
- Summation structure and compositions of convex functionsshould be broken up
- Nobody automated this before! (Derivative-based viewprevents implementation)
- DCP to the rescue. Conic form is general and encodes theinformation we need as a solver
- Proposed first outer approximation algorithm for MICONE,based on conic duality
44
![Page 58: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/58.jpg)
recap
- Polyhedral approximations can be made much better byintroducing extra variables
- Summation structure and compositions of convex functionsshould be broken up
- Nobody automated this before! (Derivative-based viewprevents implementation)
- DCP to the rescue. Conic form is general and encodes theinformation we need as a solver
- Proposed first outer approximation algorithm for MICONE,based on conic duality
44
![Page 59: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/59.jpg)
recap
- Polyhedral approximations can be made much better byintroducing extra variables
- Summation structure and compositions of convex functionsshould be broken up
- Nobody automated this before! (Derivative-based viewprevents implementation)
- DCP to the rescue. Conic form is general and encodes theinformation we need as a solver
- Proposed first outer approximation algorithm for MICONE,based on conic duality
44
![Page 60: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/60.jpg)
recap
- Polyhedral approximations can be made much better byintroducing extra variables
- Summation structure and compositions of convex functionsshould be broken up
- Nobody automated this before! (Derivative-based viewprevents implementation)
- DCP to the rescue. Conic form is general and encodes theinformation we need as a solver
- Proposed first outer approximation algorithm for MICONE,based on conic duality
44
![Page 61: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/61.jpg)
Computational results
45
![Page 62: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/62.jpg)
Pajarito (PolyhedralApproximation in Julia...)New solver!
- 1500 lines of Julia- Input is in conic form, accessible through Convex.jl DCPlanguage and JuMP
- Works with any MILP and conic solvers supported in Julia- Pkg.add(”Pajarito”)
46
![Page 63: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/63.jpg)
Translated 194 convex instances from MINLPLIB2 from AMPL intoConvex.jl (heroic parsing work by E. Yamangil).Compare with Bonmin’s outer approximation algorithm using CPLEXas the MILP solver. We use the same version of CPLEX plus KNITROas NLP (conic) solver.
47
![Page 64: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/64.jpg)
(a) Solution time (b) Number of OA iterations
Figure: Comparison performance profiles for SOC representable instances(Bonmin >30 sec)
- Dominates on iteration count. Haven’t optimized Pajarito soBonmin is faster on the easier problems.
- CPLEX is usually the fastest on MISOCPs. Use that instead.
48
![Page 65: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/65.jpg)
(a) Solution time (b) Number of OA iterations
Figure: Comparison performance profiles for SOC representable instances(Bonmin >30 sec)
- Dominates on iteration count. Haven’t optimized Pajarito soBonmin is faster on the easier problems.
- CPLEX is usually the fastest on MISOCPs. Use that instead.
48
![Page 66: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/66.jpg)
(a) Solution time (b) Number of OA iterations
Figure: Comparison performance profiles for SOC representable instances(Bonmin >30 sec)
- Dominates on iteration count. Haven’t optimized Pajarito soBonmin is faster on the easier problems.
- CPLEX is usually the fastest on MISOCPs. Use that instead.
48
![Page 67: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/67.jpg)
Almost none of the non-MISOCP instances in MINLPLIB2 are hard!
49
![Page 68: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/68.jpg)
gams01
gams01 unsolved instance which is EXP+SOC representable
- 145 variables. 1268 constraints (1158 are linear)- Previous best bound: 1735.06. Best known solution: 21516.83- Pajarito solved in 6 iterations (< 10 hours). Optimal solution is21380.20.
50
![Page 69: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/69.jpg)
Claim: Pajarito is the fastest open-source MISOCP solver80 MISOCP instances from CBLIB, 20 minute timeout
Pajarito Bonmin OA Bonmin BB Bonmin OA-D CPLEX
Optimal 47 9 16 11 69Timeout 33 49 45 47 11Error 0 22 19 22 0
Pajarito using ECOS or ConicIP as SOCP solver. Bonmin uses Ipopt.Both use CBC as MILP solver.
51
![Page 70: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/70.jpg)
ongoing and future work
- Computational results with exponential cone solvers, branch &cut
- Extensions to semidefinite programming- What happens when strong duality fails?
52
![Page 71: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/71.jpg)
Pajarito Mountain, New Mexico53
![Page 72: Mixed-integer convex optimization - GitHub Pages · 2020. 6. 25. · Nonlinear (nonconvex) optimization, min x f(x) subject to gj(x) 0; 8j - Example: AC power flow, PDEs - Global](https://reader035.vdocument.in/reader035/viewer/2022071000/5fbcad2a24039a070832b893/html5/thumbnails/72.jpg)
Thanks! Questions?
https://github.com/mlubin/Pajarito.jlhttp://dx.doi.org/10.1007/978-3-319-33461-5_9http://arxiv.org/abs/1607.03566
54